Skip to content

proposal: nested search query syntax

Created by: kzh

This proposal is for a robust search query syntax that we can easily integrate nested logic into.

Current: repo:^github\.com/sourcegraph/ repohascommitafter:"1 month ago" lang:go lang:typescript file:search (console\.log|fmt\.Println) After: repo:[hascommitsafter:"1 month ago" name:^github.com/sourcegraph/] file:[(lang:go or lang:typescript) name:search] (console\.log|fmt\.Println)

This proposal is still very early along and hasn't been fully flushed out. There isn't a definite syntax decided yet but there needs to be support for ORs and ANDs.

Context:

An everyday sourcegraph query (https://docs.sourcegraph.com/user/search/queries) can be broken up into multiple semantic parts.

<query modifiers> <repo selectors> <filters> <query>
  • query modifiers include search flags such as count, timeout, symbol, etc.
  • repo selectors indicates which repositories to search in (ex repo:^github\.com/sourcegraph/sourcegraph$)
  • filters (ie repohascommitafter, repohasfile, archive, fork, etc) specify the scope (primarily repositories and files) of the search results.

The idea is to logically group flags and be able to mix and mash filters together in order to support nested search.

Current: timeout:10s repo:^github\.com/sourcegraph/ -repogroup:frontend file:\.go$ -file:vendor/ error count:1000 archive:no fork:no type:symbol repohascommitafter:"5 months ago" New: q:[timeout:10s count:1000 type:symbol] repo:[^github\.com/sourcegraph/ hascommitafter:"5 months ago" archive:no fork:no -group:frontend] file:["\.go$" "-vendor/"] error

Here are some discussions we've had previously on this topic (still very new!). @ijt (https://sourcegraph.slack.com/archives/CHEKCRWKV/p1561486352014100)

We could get a clear and unambiguous version of nested search by adopting a variant of the earlier proposal about query fields. For example, queries could look like this: repo:[hascommitsafter:"1 month ago" name:^github.com/sourcegraph/] file:[(lang:go or lang:typescript) name:search] (console\.log|fmt\.Println). I suspect that most of what we need this sort of boolean logic for is not the textual query itself, but for properties of files and repos, where this syntax would be far clearer. So the nesting logic would be isolated within these specific fields, and the rest of the queries would be unchanged. We already have an OR operator with regular expressions foo|bar, and it’s not clear what the use case would be for an AND operator at the level of textual matches, apart from the usual way of having sequential matches like func main. If people really want func AND main they can just write (func main|main func). I believe it addresses most or all of the real use cases we were thinking of for Nested Search, but without the ambiguous and confusing syntax that was proposed before.

@kzh (https://sourcegraph.slack.com/archives/CHEKCRWKV/p1560897701015100)

Thoughts on logical grouping of search filters/flags by scopes in the search query with [] (set), & (and), and , (or)? This would avoid long filter names such as repohascommitafter (-> repo[commited: “5 days ago”]).

This is an old syntax proposed which is filled with ambiguity and inconsistency.

Scopes
======
- Query (q)
- Repository (repo)
- Files (file)

Examples
========
- repo:[commited: "5 days ago" & fork:no & archived:no, "^github\.com/sourcegraph/sourcegraph$"] file:["*.go", "*.java"] <query>
  * Repositories: (1) all repositories that has been commited to in the last five days, is not a fork, and is not archived 
                  (2) github.com/sourcegraph/sourcegraph.
  * Files: results only from Go or Java files
  
- repo:[-group: "libs"] file:["*_test.go" & changed: "2 days ago"] <query>
  * Repositories: (1) all repositories that are not in the repo group "libs"
  * Files: results only from Go testing files and changed in the last two days

- q:[type: symbol, timeout: 10s]  <query>
  * Query: (1) symbol search (2) ten second timeout
  * Repositories: all
  * Files: all

- file:["vendor/*.go" & changed:"1 day ago"] <query>
  * Repositories: all
  * Files: all Go files changed in the last day and is in the `vendor` directory

Goal
====
- Logical grouping of filters (organized)
- Queries look nicer