codeintel: Group code intel data for serialization on-demand
Created by: efritz
This changes the grouping layer that converts correlated raw LSIF data into the format that is written to disk from being pre-computed to being computed when needed by the batch writer.
This saves us from grouping everything in memory all at once (which requires that we have both the correlated data in memory, AND the grouped data before beginning to write it to disk). We now correlate everything in memory, and return only the batches that are going to be immediately serialized, keeping our resident memory down.
This will help us with concurrent serialization, as more processors can run at once with a lower memory requirement.