Loading extensions takes a really long time

Created by: mrnugget

(Just dumping debugging information for now)

Here are the problems I run into:

1. I need to load extensions more often than I think I should (multiple times per week, easily vs. code comments saying we cache extension infinitely if they're not updated)
2. Extensions are loaded sequentially
3. Time-to-first-byte is >500ms for each extension
4. Downloading an extension also takes >400ms
5. Loading my 8 extensions delays the point at which the site becomes fully available to me by ~8s

Example URL: https://sourcegraph.com/github.com/sourcegraph/sourcegraph@af89c14bfeaa3718334fcccf2ce1d6b675f58ba2/-/blob/enterprise/cmd/frontend/internal/registry/extension_connection_graphql_test.go?L89

Timeline of extension requests:

Details of two of these requests:

Debugging TTFB with curl

We used curl to record TTFB for a single extension (and replaced the sourcegraph.com in the URL below with the load balancer URL that Cloudflare talks to to exclude cloudflare)

command.sh:

curl -w "@curl-format.txt" -o /dev/null -s  -k 'https://sourcegraph.com/-/static/extension/12272-sourcegraph-rust.js?ccn0trohjr28--sourcegraph-rust&trace=1' \
  -H 'authority: sourcegraph.com' \
  -H 'pragma: no-cache' \
  -H 'cache-control: no-cache' \
  -H 'sec-ch-ua: " Not;A Brand";v="99", "Google Chrome";v="91", "Chromium";v="91"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'upgrade-insecure-requests: 1' \
  -H 'user-agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36' \
  -H 'accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9' \
  -H 'sec-fetch-site: none' \
  -H 'sec-fetch-mode: navigate' \
  -H 'sec-fetch-user: ?1' \
  -H 'sec-fetch-dest: document' \
  -H 'accept-language: en,en-US;q=0.9' \
  -H 'host: sourcegraph.com' \
  --compressed

curl-format.txt:

     time_namelookup:  %{time_namelookup}s\n
        time_connect:  %{time_connect}s\n
     time_appconnect:  %{time_appconnect}s\n
    time_pretransfer:  %{time_pretransfer}s\n
       time_redirect:  %{time_redirect}s\n
  time_starttransfer:  %{time_starttransfer}s\n
                     ----------\n
          time_total:  %{time_total}s\n

Output:

$ ./command.sh
     time_namelookup:  0.002002s
        time_connect:  0.125511s
     time_appconnect:  0.396055s
    time_pretransfer:  0.396162s
       time_redirect:  0.000000s
  time_starttransfer:  0.580356s
                     ----------
          time_total:  0.947820s