Skip to content

search: Add support for unindexed structural search

Administrator requested to merge master-dry-run/cc/unindexed-structural into main

Created by: camdencheek

This adds support for searching unindexed revisions with a structural search.

Depends on #17924 Depends on #17930 Fixes #8164

General flow:

  • Frontend sends searcher a repo/revision to search with indexed == false
  • Searcher retrieves a zip of that revision from gitserver or from its cache as normal (no change here)
  • Searcher unzips the zip, skipping any files that do not match filters
    • We unzip because ripgrep is used to pre-filter the list of files for comby, and it doesn't support searching a zip file.
    • We skip files that don't match filters because ripgrep only supports glob patterns, and our filters are regex patterns. Additionally, this keeps us from having to extract huge archives when we're only interested in a couple of files.
  • Comby searches the unzipped directory, pre-filtering with ripgrep for the sake of performance
  • Clean up the unzipped directory
  • Matches are returned as normal

Merge request reports