Skip to content

Speed up fetching/loading of extensions via `graphql?Extensions`

Warren Gifford requested to merge sqs/faster-ext-loading into main

Created by: sqs

NOTE: This PR is based on a branch that includes both #24627 and #24626. When those are merged, I'll rebase it onto main. You can review the diff now if you'd like.

These 2 perf improvements:

  • Cut the graphql?Extensions response size from ~800kb to ~5kb (compressed), or ~1.3MB to ~86kb (uncompressed).
  • Eliminate the need to JSONC-parse the result (which is slower than parsing JSON)
  • Speed up the graphql?Extensions processing on the server from ~220ms to ~140ms.

Because this introduces 2 new things to the GraphQL API, it handles backcompat. Initially it tries to query graphql?Extensions with the new query. If that returns an error that indicates that the server is old, then it uses the old query. (This is important for, e.g., browser extensions updated from the Chrome Web Store with the new code running against a Sourcegraph instance that hasn't yet been updated.)


See commit messages:

  • support fetching a subset of extension manifest fields for faster loads

    This cuts the graphql?Extensions response

    Previously, the graphql?Extensions result was very large and slow to load/parse (~800kb compressed, ~1.3MB uncompressed, for the default set of extensions) because it included the full JSON package.json manifest for each extension. This file contains base64-encoded images and the full Markdown README contents, neither of which are necessary to execute extensions.

    Now, only a subset of the manifest's fields are fetched (url, contributes, activationEvents). TypeScript types are used to enforce that no callers are using fields that aren't fetched.

  • add extensionIDs param to GraphQL extensions queries

    This significantly speeds up the graphql?Extensions query that needs to run on each page load. Previously, the query to get the user's list of enabled extensions used the prioritizeExtensionIDs param (and a first param of the length), which means that only their enabled extensions are included in the result. That sounds right, but it doesn't "hint" the intent of the query enough to make it fast. Because the intent is just to sort ("prioritize"), the GraphQL backend still needs to process all possible extensions. (As it turns out, processing them is slow because it must sort a fairly large list based on properties that sometimes require parsing a lot of JSON to determine.)

    Using the new extensionIDs param (instead of prioritizeExtensionIDs) tells the GraphQL backend that it can ignore extensions not listed, so it doesn't need to sort them or otherwise process them.

    This speeds up the graphql?Extensions timing from ~220ms to ~140ms. It is also a simpler API that more closely matches the intent of this popular way of filtering the extensions results.

    Adds tests for GraphQL and DB interactions.

Merge request reports

Loading