Skip to content

codeintel: Rewrite janitor to focus on postgres data

Warren Gifford requested to merge ef/update-janitor into main

Created by: efritz

Critical for 3.21 release, otherwise we will have no control of disk usage in customer environments.

Closes https://github.com/sourcegraph/sourcegraph/issues/14566 and https://github.com/sourcegraph/sourcegraph/issues/14569.

The janitor is now updated to not care about bundle file cleanup (as they should all be migrated), and cares about upload file maintenance less (as we'll be trying to replace the bundle manager entirely in the near future).

We've added two passes in the janitor flow:

  • Find all of the lsif uploads that are older than some configurable threshold (30 days by default) that are also not visible from the tip of the default branch for that repository and mark them as deleted.
  • Remove any data from the codeintel db that does not have a corresponding upload record in the frontend db.

This should give us an immediate delete avalanche on the dotcom environment.

Merge request reports