Querying for cloning repositories is very slow

Created by: eseliger

• Sourcegraph version: master, sourcegraph.com
• Platform information: apparently any, macOS 1.15.3 locally and sourcegraph.com as well

Initial:

While setting up some more repositories locally for some testing, I noticed that the repository page in site-admin is quite slow when having many repositories (okr/distribution/dev-productivity) and a filter for the clone status is applied, and saw in the jaeger trace that it’s due to gitserver doing many commands to git remote get-url origin sequentially - perhaps room for parallelization and/or some cache? cmd/gitserver/server/serverutil.go:257 is the line in question

Since we found out it is not slow but broken on sourcegraph.com (504s consistently), I moved this from perf enhancement to bug.

Thread: https://sourcegraph.slack.com/archives/CHPC7UX16/p1583155977114600

Steps to reproduce:

1. Go to <SG_INSTANCE>/site-admin/repositories?filter=cloning. eg https://sourcegraph.com/site-admin/repositories?filter=cloning
2. See it loads for a long time (on instances with a larger number of repositories)

Expected behavior:

It is somewhat fast

Actual behavior:

It takes very long to load or even fails with a 504