Skip to content

search: paginated repository resolution (take 2)

Warren Gifford requested to merge backend-integration/ts/paginated-repo-resolution into main

Created by: tsenart

This PR is the last of series to introduce paginated repo resolution in search.

  • We remove backend.ListSearchable and its use in repository resolution because global searches on Sourcegraph.com no longer depend on it. This in turn simplifies the code in preparation for cursor based pagination.
  • In order to be able to remove the blocking resolveRepositories in doResults, repository search had to be migrated to the new Job interface. This is implemented by run.RepoSearch.
  • We migrate all existing Job implementations to use searchrepos.Resolver.Paginate.
  • Alerts for missing repo revs and no resolved repos errors are now handled by searchResolver.errorToAlert.

New stuff in take 2

  • We optimize run.RepoSearch to leverage the database for regex matching on repo name, rather than brute-forcing it through all paginated repos. This should address the massive extra DB load we saw in production yesterday.
  • After thinking more about and talking with @stefanhengl, we change the semantics of repository count stats to mean any repositories with matches, rather than "searched" or "searchable". Searched repositories and searchable repositories aren't as useful as the number of repositories that have matches.

Part of #26995 (closed)

Merge request reports