Skip to content

search: Reduce time to lazy sync new dependency repos

Warren Gifford requested to merge main-dry-run/faster-package-host-lazy-syncing into main

Created by: tsenart

In #31577, we introduced the ability to search dependency repos with the new repo:dependencies() predicate.

As part of that, we enabled lazy syncing of new dependency repos that are parsed from lockfiles in such a search — this was done by inserting rows into the lsif_dependency_repos table for new dependency repos and triggering a new sync job for all package hosts, which then use those rows as the source of truth for what to sync.

However, when a full background sync is already happening, it can take minutes until a user sees their (new) dependency repos show up.

This commit makes us lazy sync individual dependency repos to solve that issue, both on-prem and dotcom, rather than trigger a background sync job (when there isn't one already running). It changes the repo-updater repo lookup endpoint to always call the Syncer.SyncRepo method, which is now modified to:

  1. Simply return a repo that already exists for any external service
  2. Lazy sync repos from public cloud default external services.
  3. Lazy sync repos from package host external services (i.e. NPM, Maven, etc).

Part of #28932

Test plan

Unit, integration and manual end to end testing.

Merge request reports