LSIF: Randomize the names of temp files during conversion.
Created by: efritz
When an LSIF index is uploaded it is placed into lsif-storage/uploads
. During conversion there is a SQLite database created at lsif-storage/temp
, and once it's complete it is renamed into /lsif-storage/dbs
where it can be queried by the API.
If a conversion job fails without an explicit exception (which is written to the upload record in postgres), it will eventually be retried by another worker. This may happen on pod crash/restart. In this case, the temp file needs to be different than the previous retry, because there could already be data in that SQLite file. This causes constraint violations as a metadata node already exists with a canned ID. Randomizing the filename will make sure that the retry isn't writing into a database file with partially computed data.
There should be no reason why this was chosen to be a deterministic filename (that I can remember). Nothing seems to construct the temp filename for any use, and abandoned temp files will be cleaned up by one of the cron tasks in the lsif-server (anything over a day old in uploads or temp are deleted periodically).
I think the original reason we made it a deterministic filename was so we could debug stuck files when writing some of the cleanup/data-retention things.