Skip to content

codeintel: Reorganize bundle dir

Administrator requested to merge reorganize-bundle-dir into master

Created by: efritz

This PR reorganizes the bundle directory layout so that part files are separated from real uploads and dbs, and allow us to swap out the SQLite backend for something that does not write to a single file, but a directory, in the future if we find a backend with better performance or storage characteristics (I'm currently running experiments on a few).

Old layout:

  • db files are ./dbs/{id}.lsif.db
  • db part files are ./dbs/{id}.{index}.lsif.db
  • upload files are ./uploads/{id}.lsif.gz
  • upload part files are ./uploads/{id}.{index}.lsif.gz

New layout:

  • db files are ./dbs/{id}/sqlite.db
  • db part files are ./db-parts/{id}.{index}.gz
  • upload files are ./uploads/{id}.gz
  • upload part files are ./upload-parts/{id}.{index}.gz

A migration runs at application startup to rename these files if necessary. The new and old filenames do not overlap, so the migration is idempotent. This change messes with the janitor quite a bit in that it now has to support deleting and determining sizes of directory trees instead of just files.

Merge request reports