3.9.2: panic during gitserver cleanup
Created by: slimsag
I intentionally overloaded my server instance with ~50 repos and large searches, it crashed during gitserver disk cleanup (my disk is at 90%). This is likely still an issue in master:
02:30:08 gitserver | t=2019-11-20T02:30:08+0000 lvl=eror msg="Failed to update" repo=github.com/ant-design/ant-design-pro error="exit status 128" output="fatal: Unable to read current working directory: No such file or directory\nfatal: index-pack failed\n"
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=122.3945ms
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=108.2038ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0004d0d50 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=972.4µs
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c6060 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=162µs
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=200 duration=17.8657173s
02:30:08 gitserver | t=2019-11-20T02:30:08+0000 lvl=eror msg="failed to get status of repo" repo=github.com/ant-design/ant-design-pro error="stat /var/opt/sourcegraph/repos/github.com/ant-design/ant-design-pro/.git/HEAD: no such file or directory"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c61d0 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=362.4µs
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=123.0072ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0003c2100 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=463.3µs
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c6360 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=174.3µs
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=125.7763ms
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=118.9796ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c66f0 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=293.6µs
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=100.331ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0003c2270 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=171.9µs
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=94.5557ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c68d0 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=154.7µs
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=120.4827ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0003c2460 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=226.8µs
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=129.0785ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c6a70 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=112.1395ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=398.9µs
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0001c6be0 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0004b07c0 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=5.8036ms
02:30:08 repo-updater | t=2019-11-20T02:30:08+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=1.7159ms
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=90.8145ms
02:30:08 frontend | t=2019-11-20T02:30:08+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=126.5914ms
02:30:09 gitserver | t=2019-11-20T02:30:09+0000 lvl=warn msg="cleanup: removed least recently used repo" repo=/var/opt/sourcegraph/repos/github.com/ant-design/ant-design-pro/.git how old=55.0374522s free space in GiB=11628.000 actual percent of disk space free=9.753 desired percent of disk space free=10.000 space freed in GiB=0.245 how much space to free in GiB=331.371
02:30:09 repo-updater | t=2019-11-20T02:30:09+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0003c2740 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:09 repo-updater | t=2019-11-20T02:30:09+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=361.6µs
02:30:09 frontend | t=2019-11-20T02:30:09+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=48.8218ms
02:30:09 repo-updater | t=2019-11-20T02:30:09+0000 lvl=dbug msg="TRACE handleStatusMessages" messages="[{Cloning:0xc0003c2930 ExternalServiceSyncError:<nil> SyncError:<nil>}]"
02:30:09 repo-updater | t=2019-11-20T02:30:09+0000 lvl=dbug msg=http.request method=GET route=/status-messages code=200 duration=116.2µs
02:30:09 frontend | t=2019-11-20T02:30:09+0000 lvl=dbug msg="TRACE HTTP" method=POST url=/.api/graphql?StatusMessages routename="graphql: StatusMessages" trace=#tracer-not-enabled userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.97 Safari/537.36" user=1 xForwardedFor=172.17.0.1 written=113 code=200 duration=54.1417ms
02:30:10 gitserver | panic: runtime error: invalid memory address or nil pointer dereference
02:30:10 gitserver | [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x9af8fd]
02:30:10 gitserver | goroutine 28878 [running]:
02:30:10 gitserver | github.com/sourcegraph/sourcegraph/cmd/gitserver/server.(*Server).cleanTmpFiles.func1(0xc0006c8a20, 0x85, 0x0, 0x0, 0xe4df40, 0xc0005b9770, 0x0, 0x0)
02:30:10 gitserver | github.com/sourcegraph/sourcegraph@/cmd/gitserver/server/cleanup.go:532 +0x18d
02:30:10 gitserver | path/filepath.walk(0xc001189980, 0x52, 0xe677c0, 0xc000cac8f0, 0xc000b6b680, 0x0, 0x52)
02:30:10 gitserver | path/filepath/path.go:378 +0x20c
02:30:10 gitserver | path/filepath.Walk(0xc001189980, 0x52, 0xc000b6b680, 0xc001189980, 0x52)
02:30:10 gitserver | path/filepath/path.go:404 +0xff
02:30:10 gitserver | github.com/sourcegraph/sourcegraph/cmd/gitserver/server.(*Server).cleanTmpFiles(0xc0003fe5a0, 0xc000a36b80, 0x40)
02:30:10 gitserver | github.com/sourcegraph/sourcegraph@/cmd/gitserver/server/cleanup.go:531 +0x182
02:30:10 gitserver | github.com/sourcegraph/sourcegraph/cmd/gitserver/server.(*Server).doRepoUpdate2(0xc0003fe5a0, 0xc000617080, 0x25, 0xc0006170b0, 0x2d, 0x0, 0x0)
02:30:10 gitserver | github.com/sourcegraph/sourcegraph@/cmd/gitserver/server/server.go:1355 +0x1088
02:30:10 gitserver | github.com/sourcegraph/sourcegraph/cmd/gitserver/server.(*Server).doRepoUpdate.func1.1()
02:30:10 gitserver | github.com/sourcegraph/sourcegraph@/cmd/gitserver/server/server.go:1060 +0x17d
02:30:10 gitserver | sync.(*Once).doSlow(0xc00041f890, 0xc00004f738)
02:30:10 gitserver | sync/once.go:66 +0xe3
02:30:10 gitserver | sync.(*Once).Do(...)
02:30:10 gitserver | sync/once.go:57
02:30:10 gitserver | github.com/sourcegraph/sourcegraph/cmd/gitserver/server.(*Server).doRepoUpdate.func1(0xc0009206c0, 0xc00041f890, 0xc0003c87e8, 0xc0003fe5a0, 0xc0003c6ab0, 0xc000617080, 0x25, 0xc0006170b0, 0x2d, 0xc0018d71f0)
02:30:10 gitserver | github.com/sourcegraph/sourcegraph@/cmd/gitserver/server/server.go:1052 +0x13a
02:30:10 gitserver | created by github.com/sourcegraph/sourcegraph/cmd/gitserver/server.(*Server).doRepoUpdate
02:30:10 gitserver | github.com/sourcegraph/sourcegraph@/cmd/gitserver/server/server.go:1050 +0x3cc
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=7.8346463s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2533406s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2509831s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/verekia/js-stack-from-scratch err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/styled-components/styled-components err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/GoogleChrome/lighthouse err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2574386s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=15.916019s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/jaywcjlove/awesome-mac err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/chenglou/react-motion err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2585345s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/zenorocha/clipboard.js err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2593327s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/ryanmcdermott/clean-code-javascript err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=9m31.468987s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/RocketChat/Rocket.Chat err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2542479s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/Tencent/wepy err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.261804s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/golang/go err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=2.395962s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/reduxjs/react-redux err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2593574s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/jiahaog/nativefier err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2631821s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/sourcegraph/about err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=5.4282295s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/Meituan-Dianping/mpvue err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=10m12.0702623s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/apache/incubator-superset err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2530068s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/NARKOZ/hacker-scripts err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2651907s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/leonardomso/33-js-concepts err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=13.2532152s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/parse-community/parse-server err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2683852s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/yangshun/tech-interview-handbook err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2679238s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/golang/tools err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2676774s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/zeit/next.js err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=9m31.4940834s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/storybooks/storybook err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2599621s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/airbnb/enzyme err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2694549s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/i0natan/nodebestpractices err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2650348s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/vuetifyjs/vuetify err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2692159s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/zeit/hyper err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2670922s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/vuejs/vue-cli err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2682072s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/thedaviddias/Front-End-Checklist err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2700949s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/gothinkster/realworld err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2682572s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/standard/standard err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2667218s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/dawnlabs/carbon err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2789798s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/Kickball/awesome-selfhosted err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 gitserver | Terminating gitserver
gitserver died. Shutting down...
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=18.3249833s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/jorgebucaran/hyperapp err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2808613s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/parcel-bundler/parcel err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2833611s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/gorilla/mux err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=22m29.2358269s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/gatsbyjs/gatsby err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2858908s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/gorilla/websocket err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2857543s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/react-boilerplate/react-boilerplate err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2862614s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/yarnpkg/yarn err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2842045s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/iamkun/dayjs err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2823895s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/tabler/tabler err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2817567s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/sentsin/layui err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2833407s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/yangshun/front-end-interview-handbook err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.2891306s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/juliangarnier/anime err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=dbug msg="TRACE gitserver" host=127.0.0.1:3178 path=/repo-update code=error duration=20.283791s
02:30:10 repo-updater | t=2019-11-20T02:30:10+0000 lvl=warn msg="error requesting repo update" uri=github.com/nuxt/nuxt.js err="Post http://127.0.0.1:3178/repo-update: EOF"
02:30:10 repo-updater | Terminating repo-updater
02:30:10 github-proxy | Terminating github-proxy
02:30:11 management-console | Terminating management-console
02:30:11 zoekt-indexserver | Terminating zoekt-indexserver
02:30:11 query-runner | Terminating query-runner
02:30:11 syntect_server | Terminating syntect_server
02:30:11 symbols | Terminating symbols
02:30:11 lsif-server | Terminating lsif-server
02:30:11 lsif-worker | Terminating lsif-worker
02:30:11 searcher | Terminating searcher
02:30:11 zoekt-webserver | Terminating zoekt-webserver
02:30:11 nginx | Terminating nginx
02:30:11 redis-cache | Terminating redis-cache
02:30:11 redis-store | Terminating redis-store
02:30:11 grafana | Terminating grafana
02:30:11 prometheus | Terminating prometheus
02:30:12 frontend | Terminating frontend
02:30:12 postgres | Terminating postgres