Skip to content

LSIF: RFC127-2: Do not persist data locally in server or processor

Administrator requested to merge lsif-dump-manager-2 into lsif-dump-manager-1

Created by: efritz

This is the first step to reduce the dependency on local disk in lsif-server. Currently lsif-server and lsif-worker share a persistent volume, which makes it impossible to scale or separate either.

RFC 127 (summarized) introduces a dump-manager process with which the two communicate. The dump manager is (currently singleton) service with a persistent volume and will be the only one with persistent disk that is used for anything other than scratch space.

This PR stops the server and processor from using disk space in a persistent way. All persistent data on disk is sent/retrieved from the dump manager via an API call.

The resulting APIs are minimally changed but not very nice. These will be reworked in a future PR to minimize data transfer and coupling.

Summary of changes:

  • When uploading a dump to lsif-server, immediately send the data to the dump manager for persistence
  • When converting an upload, pull the raw dump from the dump manager
  • When finished converting an upload, send the converted dump to the dump manager

Merge request reports