Skip to content
Snippets Groups Projects

codeintel: LSIF line reader

Merged Administrator requested to merge lsif-line-reader into master

Created by: efritz

Refactor the function that reads and unmarshalls lines of LSIF data from a source. This is done in a generic (focus on reading, not unmarshalling) way so we can replace/supplment JSON as an encoding in the future.

There are three significant changes:

  • Add a buffered channel between the bufio Scanner and the consumer so that IO can be done ahead of the consumer
  • Introduce a pool of unmarshal workers that can work on a batch of lines concurrently
  • Buffer the output channel so that unmarshalling can be done ahead of the correlator process

Running the precise-code-intel integration suite's upload phase on master:

3m37.597s
3m35.858s
3m42.591s

And after this change:

2m29.395s
2m32.812s
2m30.429s

Merge request reports

Merged by avatar (Jul 6, 2025 1:05am UTC)

Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading