Insights/map type query builder
Created by: coury-clark
This adds functionality such that any insight serires with a groupBy will be treated as a compute query, and will convert the regexp in the content of the query to a valid output.extra compute query.
Example
Input insights query
lang:go func(.*) group by repo
Will convert to
lang:go output.extra(func(.*) -> $repo)
Test plan
Record in insight_series:
| id | series_id | query | created_at | oldest_historical_at | last_recorded_at | next_recording_after | deleted_at | backfill_queued_at | last_snapshot_at | next_snapshot_after | repositories | sample_interval_unit | sample_interval_value | generated_from_capture_groups | generation_method | just_in_time | num_repos_at_backfill | group_by | backfill_attempts |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 9 | 27keWBK8rY1DpefN0gUvgDtkd2Q | historical lang:go | 2022-04-13 16:48:59.842378 | 2021-10-13 16:48:59.842380 | 0001-01-01 00:00:00.000000 | 2022-07-13 16:48:59.842379 | NULL | 2022-06-30 13:47:06.000000 | 2022-07-05 19:56:16.709733 | 2022-07-06 00:00:00.000000 | {github.com/sourcegraph/sourcegraph} | MONTH | 1 | true | mapping-compute | false | 1 | lang | 0 |
Generate a snapshot by setting the next_snapshot_after field to a previous date.
Here is the generated job:
| id | series_id | search_query | state | failure_message | started_at | finished_at | process_after | num_resets | num_failures | execution_logs | record_time | worker_hostname | last_heartbeat_at | priority | cost | persist_mode | queued_at |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 7857 | 27keWBK8rY1DpefN0gUvgDtkd2Q | fork:yes archived:yes lang:go count:99999999 repo:^(github\.com/sourcegraph/sourcegraph)$ content:output.extra(historical -> $lang) | completed | NULL | 2022-07-05 16:42:47.385135 +00:00 | 2022-07-05 16:42:47.527202 +00:00 | NULL | 0 | 0 | NULL | NULL | Courys-MacBook-Pro.local | 2022-07-05 16:42:47.385135 +00:00 | 10 | 500 | snapshot | 2022-06-30 21:15:22.211132 +00:00 |
Example job after changing the group field
| id | series_id | search_query | state | failure_message | started_at | finished_at | process_after | num_resets | num_failures | execution_logs | record_time | worker_hostname | last_heartbeat_at | priority | cost | persist_mode | queued_at |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 7931 | 27keWBK8rY1DpefN0gUvgDtkd2Q | fork:yes archived:yes lang:go count:99999999 repo:^(github\.com/sourcegraph/sourcegraph)$ content:output.extra(historical -> $path) | completed | NULL | 2022-07-05 20:01:59.780500 +00:00 | 2022-07-05 20:02:04.678393 +00:00 | NULL | 0 | 0 | NULL | NULL | Courys-MacBook-Pro.local | 2022-07-05 20:01:59.780500 +00:00 | 10 | 500 | snapshot | 2022-07-05 20:01:59.935622 +00:00 |