Skip to content

codeintel: More intelligently page/paginate lsifstore locations results

Warren Gifford requested to merge ef/18250 into main

Created by: efritz

Fixes https://github.com/sourcegraph/sourcegraph/issues/18250. This PR makes a change to the signature of readLocationsFromResultChunks, which now returns a total count rather than a sorted/deduplicated list of paths. The paths are constructed only by one caller, which constructs them manually now.

The bigger change is that we "push down" the pagination from definition/references into the locations call, which allows us to cut a number of documents we would otherwise need to open. We do this by performing the same basic steps:

  • translating each definition/reference result ID into a result chunk
  • read each result chunk (in batches) and construct a map of things we need to look up
  • new step remove things from the mapping we just constructed based off of limit/offset parameters
  • resolve the range ids in the mapping we just constructed/modified and return that

This buys us a fewer number of range ids to resolve and documents to open. There may be additional benefits in opening fewer result chunks, which we will cover in https://github.com/sourcegraph/sourcegraph/issues/18620.

Merge request reports