Skip to content
Snippets Groups Projects

codeintel: Avoid unbounded data retrieval from lsifstore

Merged Administrator requested to merge ef/avoid-unbounded-payload-requests into main

Created by: efritz

Following up from https://github.com/sourcegraph/sourcegraph/pull/18511, we ensure that we keep the memory usage within postgres itself low by not having to have it load a bunch of data into a cache for us to stream. I think this should reduce some kinds of worst-case scenario where we balloon data in the codeintel-db because we have to open 99,999 documents in a very large dump.

The change here looks nastier than it is. There's no big logic changes except that when we are querying the lsif_data_result_chunks and lsif_data_documents tables, we do it in maximum sized batches. All the other changes are breaking things into different functions.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
Please register or sign in to reply
Loading