Skip to content

search: shortcircuit indexed search for structural search

Administrator requested to merge rvt/structural-search-shortcircuit-v2 into master

Created by: rvantonder

Stacked on #8369. This PR introduces a functional change to how structural search calls indexed search. From previous PRs, I use indexed search to get a list of candidate files that likely contain content that will be found by structural search. Previously, indexed search would always be called once, with fixed parameters. This PR modifies it so that indexed search is possibly called twice, with different parameters which I found useful for returning results faster for some cases.

Basically:

  1. First call indexed search with the default count (30) and a regexp pattern that approximates the structural pattern and disallows newline matching across .*-like patterns. Disallowing newline matching may return true hits faster.

  2. If the number of candidate files in (1) is low (less than 10), or count was overridden in the query, then perform the expensive search including newline matching across program fragments. This can be pretty slow, but more complete.


The only unfortunate side-effect is that doing (1) always assumes that the returned results will be incomplete. This is because the pattern without newlines may capture all file candidates containing all true matches, but we don't know for sure unless (2) runs. Thus, when only (1) runs, the user will always see a 'show more' prompt. Clicking the 'show more' will force (2) to run, which may return the same results, or more. I highly prefer this minor usability issue (which I'll address once I think more about it) in favor of the slow results that may happen due to (2).

The slowness before this PR has been a blocker for me releasing the blog post, and this PR unblocks it. This functionality was tested heavily against example patterns used in the blog post for speed and accuracy. I will add some e2e tests but want to put this up for review now.

See inline comments for details.

Merge request reports

Loading