Skip to content
Snippets Groups Projects

search: basic and/or search pattern evaluation

Created by: rvantonder

This is the barebones evaluator code for and/or expressions on content search. Example screenshot below. It is barebones because it implements the bare minimum and naive way to ensure that we can run and/or queries. There are many (many!) opportunities to optimize this functionality and we should consider them all. But we should first add the essential logic that can always serve as a naive fallback guaranteeing a correct evaluation. Here are some details to make that concrete:

  • The way the evaluator works is by breaking down the search pattern into leaf nodes, running our search, and then merging the results (intersect or union). For each leaf node search, we tack on the scopeParameters which are global with respect to searching over file content currently. In future the evaluator will take into account expressions in scope too.

  • One part to still refine and test is around merge operations for things other than file matches (i.e., common search results, result counts, timing info and so on). Most of this is taken care of naturally with the update function, but there are other parts that I'm not so sure of (e.g., what is the "merge" operation for two search pagination cursors?). This is not immediately important to solve; just pointing it out.

  • Things that are done naively that can be optimized in future:

    • Expressions like (foo or bar) map directly to regex expressions like foo|bar. This is a simple rewrite that we can do on Parameter nodes as a preprocess step.
    • Translate our and/or more directly into a Zoekt expression tree on the indexed code path.
    • Reorder the tree using heuristics that so that we evaluate patterns that are likelier to shortcircuit early (applies to both and and orbranches)
    • Goroutines for evaluating separate branches, maybe.
    • And other things I probably am not thinking of right now.

I'll add tests, either in this PR, or follow up, based on what I want to tackle. The PR is good for review; think of it more as scaffolding than the final product.


Screenie: with this PR, the following kinds of queries mostly work:

Screen Shot 2020-03-27 at 7 26 14 PM

It also carries over to structural search, which is pretty neat. There are still a ton of gaps to fill in around things like quoting, parentheses interpretation, and other stuff. But this is the basics.

Merge request reports

Approval is optional

Merged by avatar (Sep 4, 2025 1:35am UTC)

Merge details

  • Changes merged into master with 4af36697.
  • Deleted the source branch.

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Created by: codecov[bot]

    Codecov Report

    Merging #9381 into master will decrease coverage by 0.04%. The diff coverage is 0.00%.

    @@            Coverage Diff             @@
    ##           master    #9381      +/-   ##
    ==========================================
    - Coverage   41.40%   41.35%   -0.05%     
    ==========================================
      Files        1330     1330              
      Lines       72367    72442      +75     
      Branches     6581     6581              
    ==========================================
      Hits        29962    29962              
    - Misses      39643    39718      +75     
      Partials     2762     2762              
    Impacted Files Coverage Δ
    cmd/frontend/graphqlbackend/search_results.go 46.92% <0.00%> (-4.31%) :arrow_down:
  • Created by: rvantonder

    Also feel free to slam the "request changes", next time. It would have been appropriate here :-)

  • Created by: rvantonder

    Merging since this is low risk. Please comment post merge to tie up loose ends. Tests to follow.

Please register or sign in to reply
Loading