Skip to content

codeintel: Consolidate periodic jobs into frontend

Warren Gifford requested to merge ef/consolidate-background-jobs into main

Created by: efritz

The precise-code-intel-indexer, precise-code-intel-bundle-manager, and precise-code-intel-worker all preform periodic tasks in the background. As two of these services are going away in the immediate future, these background tasks should be consolidated into the frontend.

Summary of background jobs (all copied from another source, does not need full review):

  • The abandoned uploads pass was copied from the bundle manager janitor
  • The commit updater pass was copied from the worker
  • The deleted repositories pass was copied from the bundle manager janitor
  • The hard delete pass was copied from the bundle manager janitor
  • The indexability scheduler was copied from the indexer
  • The index scheduler was copied from the indexer
  • The record expirer was copied from the bundle manager janitor
  • The enterprise/cmd/frontend/internal/codeintel/inference package was copied from the indexer services

The indexer is being removed in https://github.com/sourcegraph/sourcegraph/pull/15135, and the bundle manager will be removed in 3.23. It is ok for these background jobs to be run multiple times and concurrently. Any critical section is already guarded by postgres advisory locks.

Additional changes:

  • Refactored the codeintel service setup during init of enterprise frontend
  • Added Config object for all code-intel specific envvar validation
  • Collapsed metrics for background jobs (need to update monitoring)

I fully intend to refactor the frontend startup pass to make this flow less of a kludge.

Merge request reports