Skip to content
Snippets Groups Projects

LSIF: Randomize the names of temp files during conversion.

Merged Warren Gifford requested to merge lsif-random-temp-files into master

Created by: efritz

When an LSIF index is uploaded it is placed into lsif-storage/uploads. During conversion there is a SQLite database created at lsif-storage/temp, and once it's complete it is renamed into /lsif-storage/dbs where it can be queried by the API.

If a conversion job fails without an explicit exception (which is written to the upload record in postgres), it will eventually be retried by another worker. This may happen on pod crash/restart. In this case, the temp file needs to be different than the previous retry, because there could already be data in that SQLite file. This causes constraint violations as a metadata node already exists with a canned ID. Randomizing the filename will make sure that the retry isn't writing into a database file with partially computed data.

There should be no reason why this was chosen to be a deterministic filename (that I can remember). Nothing seems to construct the temp filename for any use, and abandoned temp files will be cleaned up by one of the cron tasks in the lsif-server (anything over a day old in uploads or temp are deleted periodically).

I think the original reason we made it a deterministic filename was so we could debug stuck files when writing some of the cleanup/data-retention things.

Merge request reports

Merged by avatar (Jul 13, 2025 10:32am UTC)

Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading