Core Services: 3.8 Tracking Issue & beyond

Created by: mrnugget

Planning 3.8 and beyond

What will break with 80k repositories?
What's the delta between the current state of Sourcegraph and the state that would close current deals in our pipeline?

Tomás: Improve ACL performance
- #4812 - Global Repository Permissions Caching (@tsenart)
Thorsten: Improve site-admin experience
- #5145 - Use streaming in repo-updater to make syncing process more responsive
- #5146 - Make external service syncing errors visible to site-admins
Keegan:
- Assisting and helping out the search team with architectural decisions & design process
- Improve Zoekt performance
  - Horizontally scaling Zoekt
  - (If time permits: single-node performance, see Beyond)

~~Fix Quinn's issue with EXTSVC_CONFIG_FILE setting: #5134~~
~~Allow configuring external service with orgs (do not require repos or repositoryQuery to be set if orgs is set): #4322~~
Customer issue: repo-updater "failed to run repository clone purge" (the <> bug)
Fix gitserver purging process: introduce "mark & sweep process". #5056 and comment
Querying repository by name with github.com prefix always returns empty string for name: #5125
Gitolite EOF: #5119
Gitolite not cloning: #5121
GitHub Oauth scope: #5111

Investigate, understand, tweak and improve concurrency model
Introduce shard compaction to improve read performance (Inspiration: Cassandra + SS Tables)
Bloom filters in main process to avoid looking into shards that won't yield results
Improve memory usage
Experiment with more efficient posting lists in Zoekt https://github.com/sourcegraph/zoekt/pull/10
Improve parsing of newlines in Zoekt

Improving gitolite performance: large-scale customer reported that it takes gitolite 27 min to list all repositories (that is: without Sourcegraph being involved at all, it's pure gitolite)
Determine resource allocation "equation" and update docs for admins to provision them accordingly for their Sourcegraph installation (e.g. https://confluence.atlassian.com/bitbucketserver/scaling-bitbucket-server-776640073.html)