Something went wrong on our end. Please try again.
Created by: efritz
Following up from https://github.com/sourcegraph/sourcegraph/pull/18511, we ensure that we keep the memory usage within postgres itself low by not having to have it load a bunch of data into a cache for us to stream. I think this should reduce some kinds of worst-case scenario where we balloon data in the codeintel-db because we have to open 99,999 documents in a very large dump.
The change here looks nastier than it is. There's no big logic changes except that when we are querying the lsif_data_result_chunks and lsif_data_documents tables, we do it in maximum sized batches. All the other changes are breaking things into different functions.