RFC 629: Update lifecycle of `lsif_indexes`
Created by: efritz
Implements a portion of RFC 629: Cold-storing older worker execution logs.
Because index records can be deleted, we also want to delete the execution logs stashed in cold storage when that record goes away. For lsif_uploads
table, we have extra state transitions, specifically: completed
-> deleting
-> deleted
-> (hard delete). We should add a deleting
state for indexes as well so that we can mark the records to remove from the database without having to synchronously delete files in cold storage.
The sink interface should expand to include a remove or delete method that operates on a forwarding address identifying some specific piece of data. All places where index records are hard deleted should be vetted to move them into the deleting
state instead when a forwarding address is set. A new background janitor routine should be added to delete data associated with an index in this state, then hard delete the records.
Query methods should likely exclude any index records in deleting state - this is an implementation detail that doesn't need to leak to the client. (Unlike upload records, where deleting vs deleted actually have different meanings in the UI: deleting uploads are still accessible for code intelligence; index records are just metadata and don't affect query paths).