Skip to content
Snippets Groups Projects

codeintel: Keep api doc search current asynchronously

Merged Warren Gifford requested to merge ef/fast-lsif-data-upsert into main

Created by: efritz

Changes:

  • Add a new table that keeps track of the most current dump_id for a (repo, root, language)
  • Remove deletions from the data insertion hot-path. We don't want lock contention clearing old records, so we only append to the table when processing an LSIF record. We now expect that the query path uses the condition: dump_id = (SELECT dump_id FROM lsif_data_doc_search_public WHERE repo_id=outer.repo_id AND dump_root=outer.dump_root AND lang_name_id=outer.lang_name_id) to remove non-relevant records that may exist between cleanup runs.
  • Create a background process that periodically removes the records that would be filtered out by the condition above.

I hope that this restores the PCI worker throughput on Cloud.

Helps #21938

Merge request reports

Merged by avatar (Jun 9, 2025 10:40am UTC)

Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading