Skip to content

search: convert structural search patterns to match newlines in index

Warren Gifford requested to merge rvt/structural-search-newline-regex-index into master

Created by: rvantonder

Summary is the same as the comment for the added StructuralPatToRegexpQuery function:

Converts comby a structural pattern to a Zoekt regular expression query. This
conversion conceptually performs the same conversion as
StructuralPatToConjunctedLiteralsQuery, except that whitespace in the pattern
is converted so that content across newlines can be matched in the index. The
function produces a conjunction of regular expressions where whitespace is
converted to \s+, rather than a conjunction of literal strings.

Example:
"ParseInt(:[args]) if err != nil" -> "ParseInt(" AND ")\s+if\s+err!=\s+nil"

The regex-zoekt query is combined with the existing substring query using an OR clause. I don't want to combine the regex-zoekt queries and substring query generation even though it's possible, because (a) keeping the functionality isolated may help for reusing this in unindexed search and (b) the existing tests do not need to change (the internal query representation would change otherwise, meaning the output would change).

Test plan: Tests added.

Merge request reports