Skip to content

insights: add a new insightsCount compute command to support insights work in aggregating search results

Administrator requested to merge cw/insights-compute-cmd into main

Created by: chwarwick

This adds a new compute command for use by Code Insights to support on going work to provide aggregates of search results.

This initial pr adds the command and the ability to return counts that can be summed by a calling client for Repo, Path, Author, and Language. Additional work will extend this command's return type to optionally include a repo id so it can be used for insights that are persisted as well as add the ability to count capture group matches like the MatchOnly command currently does.

typical usage:

repo:^github.com/org/myrepo$ lang:Go content:insights.count(my search -> $REPO) count:all patterntype:standard

I choose to add a new command because I wanted to constrain the behaviors of the command to the minimum needed to support the code insights functionality under development.

Known issues:

  • Currently search queries passed to compute are initially parsed as patterntype:regexp, this can lead to parsing errors if the search pattern contains an invalid regex. For example i++. I think this is something that can be resolved in followup work.
  • Currently search queries passed to compute without a patterntype run as a regexp patterntype and do not follow the same default logic that search does which would run as patterntype:standard

There is a very similar issue raised in https://github.com/sourcegraph/sourcegraph/issues/40125

Test plan

Unit tests pass

GET - q= content:insights.count(todo -> $REPO) patterntype:literal count:10

event: results
data: [{"value":"github.com/sourcegraph/sourcegraph","count":10}]

event: progress
data: {"done":false,"matchCount":0,"durationMs":33,"skipped":[{"reason":"shard-match-limit","title":"result limit hit","message":"Not all results have been returned due to hitting a match limit. Sourcegraph has limits for the number of results returned from a line, document and repository.","severity":"info","suggested":{"title":"increase limit","queryExpression":"count:1000"}}]}

event: progress
data: {"done":true,"matchCount":0,"durationMs":33,"skipped":[{"reason":"shard-match-limit","title":"result limit hit","message":"Not all results have been returned due to hitting a match limit. Sourcegraph has limits for the number of results returned from a line, document and repository.","severity":"info","suggested":{"title":"increase limit","queryExpression":"count:1000"}}]}

event: done
data: {}

Closes https://github.com/sourcegraph/sourcegraph/issues/39776

Merge request reports