errors seen in indexed-search on bigdata cluster with 250k repos
Created by: uwedeportivo
- 3.16 or master insiders
- k8s GKE
these are error messages seen in the indexed-search pods on the pilot bigdata cluster this cluster has 250k repos in about 1k+ orgs
this one shows up sporadically (13 times), not often:
2020/05/28 19:09:50 watcher error: fsnotify queue overflowthis type showed up many times
2020/05/31 21:44:43 error indexing ghe.sgdev.org/awesome-dubinsky-308/achintp-Leetcode_solutions@HEAD=7f4e5f9670108b934491bc82cb488cb9d373c398: GET http://sourcegraph-frontend-internal/.internal/search/configuration giving up after 5 attemptsmaybe making frontend more responsive by giving it more mem helped with it
also seen is this type
ERR: 2020/06/01 10:44:35 main.go:237: archive/tar: invalid tar header
2020/06/01 10:45:31 error indexing ghe.sgdev.org/agitated-bouman-446/CloudMaster-Manoj-AC@HEAD=cfcc4223d2dfaa1516f1181880d689b2b5b2becc: command [zoekt-archive-index -name ghe.sgdev.org/agitated-bouman-446/CloudMaster-Manoj-AC -commit cfcc4223d2dfaa1516f1181880d689b2b5b2becc -incremental -branch HEAD -download-limit-mbps 1000 -file_limit 1048576 -parallelism 8 -index /data/index -require_ctags http://sourcegraph-frontend-internal/.internal/git/ghe.sgdev.org/agitated-bouman-446/CloudMaster-Manoj-AC/tar/cfcc4223d2dfaa1516f1181880d689b2b5b2becc] failed: exit status 1
OUT: this might be bum git repos (there are some in our collection of 111 million repos). worth checking here that the bum repos are not poison pills that prevent indexing of other repos (i don't think so, but worth double-checking).
there's also
er ERR: 2020/06/01 19:06:31 main.go:237: archive/tar: invalid tar header ││ zoekt-indexserver 2020/06/01 19:07:31 error indexing ghe.sgdev.org/agitated-bouman-446/CloudMaster-Manoj-AC@HEAD=cfcc4 ││ zoekt-indexserver OUT: ││ zoekt-indexserver ERR: 2020/06/01 19:07:31 main.go:237: archive/tar: invalid tar header
worth taking a look just to make sure. many thanks :-)