Implement ApplyCampaign and changeset reconciler
Created by: mrnugget
IMPORTANT! Note for reviewers: There still a lot of TODOs, but I don't want to tackle all of them in this PR, otherwise the PR grows too large and our changes to the API diverge. And I want to now get a review as early as possible. See the two checklists below for what I want to do in this PR and what not before you review.
@efritz can you take a look at workers.go
to see whether I'm doing anything wrong?
This PR implements the majority of the new workflow's backend logic:
- Changesets are now state machines that can exist in Sourcegraph's database before they've been created on the code host.
- A new changeset reconciler (see
reconciler.go
) is aworkerutils.Worker
that runs in the background and picks up enqueued changesets to reconcile their current state (in the Sourcegraph DB or on the code host) with the desired state the user expressed in a changeset spec. - The
applyCampaign
mutation now correctly creates changesets for a given campaign so that the user can apply a campaign spec and immediately see unpublished/published changesets in the UI that the reconciler then picks up and, well, reconciles.
In order to do that, a few changes to previous invariants were necessary:
- Changesets can now exist in the database without an
external_id
. That change required me to get rid of our batch-insertion/update logic that we used for changesets in theStore
. But that's alright, since we didn't use it anyway. Seestore.go
for that. - Changesets now have a
PublicationState
and aReconcilerState
in the API, seeschema.graphql
.
What's still missing in this PR
-
Tests for reconciler
-
Repository permissions test for ApplyCampaign
-
Add support for new PublicationState
andReconcilerState
toListChangesets
What I will address in follow-up PRs
-
The logic in the changesets resolvers -
Maybe: refactor applyCampaign
to split it up into more pieces -
Refactor the code in the reconciler so that delta
is nicer to use -
Moving CloseOpenChangesets
to the reconciler -
Updating the GraphQL API integration test in resolver_test.go
-
Moving SyncChangesets
to the reconciler (which will require makingtitle
andbody
on changeset nullable in the API since tracked changesets can then exist in an unsynced state) https://github.com/sourcegraph/sourcegraph/issues/12645 -
Remove the wrapping UpdateChangesets
,CreateChangesets
(note: plural!) methods -
Retrying of errors in the reconciler. -
All the other TODOs