a8n: Implement `previewCampaignPlan`
Created by: mrnugget
This is part of https://github.com/sourcegraph/sourcegraph/issues/6085 and implements the GraphQL previewCampaign
mutation.
It validates arguments, creates a CampaignPlan
and multiple CampaignJob
s (one for each repo yielded by the CampaignType
s search query).
The single CampaignType
that's currently implemented is "comby"
, which does search&replace across repositories.
What's in this PR already works: a plan is created, jobs are launched, executed and their Diff
field is updated.
There are a few things missing that I want to address in follow-up PRs in order to make reviewing easier and to unblock @felixfbecker and @eseliger:
-
See comment below
The diff produced by reading json-lines from thereplacer
service is not a valid multi-file diff. While it can be parsed again bydiff.ParseMultiFile
the produced[]*diff.FileDiff
only has a single element. That's enough to use the API for now, which is why I don't want it to block the merging of this PR. I will address this in a separate PR that also adds tests forcomby
(which right now only does a single HTTP request which we likely need to change) -
CampaignPlan
s are not cleaned up yet. I already discussed this with @tsenart. - The
CampaignPlan.status
is still a dummy implementation.
Merge request reports
Activity
Created by: codecov[bot]
Codecov Report
Merging #6265 into master will increase coverage by
0.06%
. The diff coverage is51.76%
.@@ Coverage Diff @@ ## master #6265 +/- ## ========================================== + Coverage 39.32% 39.38% +0.06% ========================================== Files 1192 1194 +2 Lines 61289 61419 +130 Branches 5838 5838 ========================================== + Hits 24101 24189 +88 - Misses 34983 35008 +25 - Partials 2205 2222 +17
Impacted Files Coverage Δ internal/a8n/types.go 21.12% <ø> (ø)
cmd/frontend/graphqlbackend/codemod.go 10.05% <0%> (ø)
enterprise/pkg/a8n/resolvers/resolver.go 38.81% <0%> (+4.36%)
enterprise/pkg/a8n/run/campaign_type.go 44.82% <44.82%> (ø)
enterprise/pkg/a8n/run/runner.go 60.78% <60.78%> (ø)
Created by: mrnugget
I added the last commit 41f8d0e after our discussion in Slack about skipping repositories without a default branch.
Created by: mrnugget
Just added 374fc893fcf0a46b7290e2cd97fecf504e923787 to temporarily fix the problem of multi-file diffs.
The problem is that the
go-diff
package doesn't parse multi-file diffs correctly if they're missing adiff ...
separator line between the single file diffs.git apply
andpatch
, though, can handle these perfectly fine. So that is a "bug" ingo-diff
that I want to fix.Until that's fixed and to make working with this branch easier, this temporary fix injects a
diff ...
line between the file diffs.The resulting multi-file diff can then be parsed by
go-diff
in the GraphQL backend and return multiple file diffs per repository. That gives us the correct response. Here's an example from a campaign that ran across multiple repositories and replaced text in zero or more files in each repository:{ "data": { "previewCampaignPlan": { "__typename": "CampaignPlan", "id": "Q2FtcGFpZ25QbGFuOjEx", "type": "comby", "arguments": "{\"scopeQuery\":\"repo:gorilla\",\"matchTemplate\":\"example.com\",\"rewriteTemplate\":\"sourcegraph.com\"}", "status": { "completedCount": 0, "pendingCount": 99, "state": "ERRORED", "errors": [ "this is just a skeleton api" ] }, "changesets": { "nodes": [ { "diff": { "fileDiffs": { "totalCount": 0, "nodes": [] } }, "repository": { "name": "ghe.sgdev.org/sourcegraph/gorilla-sessions", "url": "/ghe.sgdev.org/sourcegraph/gorilla-sessions" } }, { "diff": { "fileDiffs": { "totalCount": 4, "nodes": [ { "oldPath": "example_cors_method_middleware_test.go", "newPath": "example_cors_method_middleware_test.go", "hunks": [ { "body": " \tfmt.Println(rw.Header().Get(\"Access-Control-Allow-Origin\"))\n \t// Output:\n \t// GET,PUT,PATCH,OPTIONS\n-\t// http://example.com\n+\t// http://sourcegraph.com\n }\n", "oldRange": { "startLine": 33 }, "newRange": { "startLine": 33 } } ] }, { "oldPath": "README.md", "newPath": "README.md", "hunks": [ { "body": " \n Setting the same matching conditions again and again can be boring, so we have a way to group several routes that share the same requirements. We call it \"subrouting\".\n \n-For example, let's say we have several URLs that should only match when the host is `www.example.com`. Create a route for that host and get a \"subrouter\" from it:\n+For example, let's say we have several URLs that should only match when the host is `www.sourcegraph.com`. Create a route for that host and get a \"subrouter\" from it:\n \n ```go\n r := mux.NewRouter()\n", "oldRange": { "startLine": 150 }, "newRange": { "startLine": 150 } }, { "body": " s.HandleFunc(\"/articles/{category}/{id:[0-9]+}\", ArticleHandler)\n ```\n \n-The three URL paths we registered above will only be tested if the domain is `www.example.com`, because the subrouter is tested first. This is not only convenient, but also optimizes request matching. You can create subrouters combining any attribute matchers accepted by a route.\n+The three URL paths we registered above will only be tested if the domain is `www.sourcegraph.com`, because the subrouter is tested first. This is not only convenient, but also optimizes request matching. You can create subrouters combining any attribute matchers accepted by a route.\n \n Subrouters can be used to create domain or path \"namespaces\": you define subrouters in a central place and then parts of the app can register its paths relatively to a given subrouter.\n \n", "oldRange": { "startLine": 165 }, "newRange": { "startLine": 165 } } ] }, { "oldPath": "route.go", "newPath": "route.go", "hunks": [ { "body": " // For example:\n //\n // r := mux.NewRouter()\n-// r.Host(\"www.example.com\")\n+// r.Host(\"www.sourcegraph.com\")\n // r.Host(\"{subdomain}.domain.com\")\n // r.Host(\"{subdomain:[a-z]+}.domain.com\")\n //\n", "oldRange": { "startLine": 284 }, "newRange": { "startLine": 284 } }, { "body": " // It will test the inner routes only if the parent route matched. For example:\n //\n // r := mux.NewRouter()\n-// s := r.Host(\"www.example.com\").Subrouter()\n+// s := r.Host(\"www.sourcegraph.com\").Subrouter()\n // s.HandleFunc(\"/products/\", ProductsHandler)\n // s.HandleFunc(\"/products/{key}\", ProductHandler)\n // s.HandleFunc(\"/articles/{category}/{id:[0-9]+}\"), ArticleHandler)\n", "oldRange": { "startLine": 455 }, "newRange": { "startLine": 455 } } ] }, { "oldPath": "mux_test.go", "newPath": "mux_test.go", "hunks": [ { "body": " \t\t\tpath: \"\",\n \t\t\tquery: \"foo=bar&baz=ding\",\n \t\t\tpathTemplate: `/api`,\n-\t\t\thostTemplate: `www.example.com`,\n+\t\t\thostTemplate: `www.sourcegraph.com`,\n \t\t\tqueriesTemplate: \"foo=bar,baz=ding\",\n \t\t\tqueriesRegexp: \"^foo=bar$,^baz=ding$\",\n \t\t\tshouldMatch: true,\n", "oldRange": { "startLine": 782 }, "newRange": { "startLine": 782 } }, { "body": " \t\t\tpath: \"\",\n \t\t\tquery: \"foo=bar&baz=ding\",\n \t\t\tpathTemplate: `/api`,\n-\t\t\thostTemplate: `www.example.com`,\n+\t\t\thostTemplate: `www.sourcegraph.com`,\n \t\t\tqueriesTemplate: \"foo=bar,baz=ding\",\n \t\t\tqueriesRegexp: \"^foo=bar$,^baz=ding$\",\n \t\t\tshouldMatch: true,\n", "oldRange": { "startLine": 796 }, "newRange": { "startLine": 796 } } ] } ] } }, "repository": { "name": "ghe.sgdev.org/sourcegraph/gorilla-mux", "url": "/ghe.sgdev.org/sourcegraph/gorilla-mux" } }, { "diff": { "fileDiffs": { "totalCount": 1, "nodes": [ { "oldPath": "x_net_proxy.go", "newPath": "x_net_proxy.go", "hunks": [ { "body": " \n // AddFromString parses a string that contains comma-separated values\n // specifying hosts that should use the bypass proxy. Each value is either an\n-// IP address, a CIDR range, a zone (*.example.com) or a host name\n+// IP address, a CIDR range, a zone (*.sourcegraph.com) or a host name\n // (localhost). A best effort is made to parse the string and errors are\n // ignored.\n func (p *proxy_PerHost) AddFromString(s string) {\n", "oldRange": { "startLine": 94 }, "newRange": { "startLine": 94 } } ] } ] } }, "repository": { "name": "ghe.sgdev.org/sourcegraph/gorilla-websocket", "url": "/ghe.sgdev.org/sourcegraph/gorilla-websocket" } }, { "diff": { "fileDiffs": { "totalCount": 0, "nodes": [] } }, "repository": { "name": "ghe.sgdev.org/sourcegraph/gorillalabs-sparkling", "url": "/ghe.sgdev.org/sourcegraph/gorillalabs-sparkling" } } ] } } } }