Skip to content

LSIF: RFC127-3: Move fs-based tasks into dump manager

Administrator requested to merge lsif-dump-manager-3 into lsif-dump-manager-2

Created by: efritz

This is the first step to reduce the dependency on local disk in lsif-server. Currently lsif-server and lsif-worker share a persistent volume, which makes it impossible to scale or separate either.

RFC 127 (summarized) introduces a dump-manager process with which the two communicate. The dump manager is (currently singleton) service with a persistent volume and will be the only one with persistent disk that is used for anything other than scratch space.

This PR stops the server's periodic tasks from trying to delete persistent files (which it no longer has access to). These tasks are to be run on the dump manager where the persistent volume is actually mounted.

The resulting APIs are minimally changed but not very nice. These will be reworked in a future PR to minimize data transfer and coupling.

Summary of changes:

  • Move fs-based periodic tasks over to dump manager (with dependence on db)
  • Add internal routes to get the state of uploads and prune/delete an old upload to the server
  • Replace db accesses in dump manager tasks with calls to internal API routes

Merge request reports