Skip to content

codeintel: Update candidate repos for auto-indexing

Warren Gifford requested to merge ef/autoindex-repo-candidates into main

Created by: efritz

Fixes https://github.com/sourcegraph/sourcegraph/issues/26852.

This PR makes the following high-level changes:

  • The repositories we choose to auto-index now come from indexing policies and does not use search contexts as a back-door way to collect sets of repositories.
  • We now have global limits (configured in site settings) to restrict the blast radius of indexing patterns like *
  • Update the PreviewRepositoryFilter graphql resolver to take multiple patterns and return a paginate-able response
  • Better UI!

Disallow global policies:

Screen Shot 2021-11-15 at 3 50 43 PM Screen Shot 2021-11-15 at 3 50 48 PM

Allowed global policies:

Screen Shot 2021-11-15 at 3 51 31 PM

This PR makes the following lower-level changes:

  • Replace RepoIDsByGlobPattern with RepoIDsByGlobPatterns that takes multiple glob patterns and handles pagination
  • Update UpdateReposMatchingPatterns to takes a optional maximum number of matching repositories per pattern set (presumably belonging to the same policy)
  • Update SelectRepositoriesForIndexScan to scan the repos matching configuration policies instead of using search contexts. We'll be able to delete our large corpus search context in prod after this PR becomes effective.
  • Adds two new site config flags under codeIntelAutoIndexing
  • Exposes the site config value codeIntelAutoIndexingAllowGlobalPolicies to the UI context

Merge request reports