Skip to content

codeintel: Add diagnostic count to lsif_data_documents

Administrator requested to merge ef/18248 into main

Created by: efritz

Fixes https://github.com/sourcegraph/sourcegraph/issues/18248. This effort is doing double duty to both aid in better performance in precise diagnostic resolvers as well as test out the RFC-311 out-of-band migration infrastructure.

We make two alterations to the databases:

  • in the frontend, we add an out-of-band migration record with an ID=1
  • in the codeintel-db, we add a schema_version and a num_diagnostic field to the documents table

For all new records to the documents table, we set the newest schema version and denormalize the diagnostic counts to the new field.

We are not yet reading the new field, so its absence does not yet affect us. In order to move all records to the new version over time, we add a migrator instance that reads the old payloads, extracts a count, and updates the record, bumping the schema version as it does so.

Note that when we do read the new field, we have several paths forward:

  • Do not read new fields until the version in which we mark this migration as deprecated (not before 3.27), then only read the new version and remove support for the old version in one go, or
  • Read new fields optimistically fall back to interpreting the record in the previous way - note that fallbacks should reduce as the migration occurs in the background.

This PR does not register this migration to run in the background yet, but does add the (tested) migrator code. A subsequent PR will register the migrator to run in the frontend.

Merge request reports