campaigns: add GitLab webhook support
Created by: LawnGnome
This PR closes #12171 (closed) by implementing GitLab webhook support. It includes the UI and schema changes required for GitLab external services to support webhooks, the documentation for a user to configure the services, and then all the bits we need to handle those webhook requests.
Random points of interest are enumerated below.
Testing
About half of this PR is unit and integration tests for the new webhooks_gitlab.go
file, which I've put a lot of effort into testing as thoroughly as possible, since we don't control what comes into the HTTP handler. To shamelessly rip off one of Knuth's great lines, I can't promise that it's bug free, but I can promise it has 100% code coverage.
Approvals and unapprovals
GitLab uses a completely different mechanism in webhooks to indicate that a merge request has been approved or unapproved to what the API normally exposes. In the REST and GraphQL APIs, we get the approval state by parsing the list of notes (comments) that are attached to the merge request: there's a concept of a "system note" that we can string parse to figure out the state. (Alas, more structured approval and unapproval data relies on the project (a) using an enterprise licence, and (b) being configured with approval pipelines, which isn't required for normal use.)
In webhooks, we just get the merge request as a "merge request event" with an "action" field set to "approved" or "unapproved", and no further details. (I'd point to the documentation, but I couldn't find any. This was all trial and error.) The lack of detail means we can't deduplicate the approval later when we do a full sync, since we don't have the note ID. To make matters worse, although GitLab does support note events, note events aren't sent for system notes.
So I've taken a slightly different approach here: we don't create changeset events based on approval or unapproval webhook invocations. Instead, we take those as a hint that we should sync the merge request via repo-updater
as soon as possible, and let the normal sync process handle it as if the user had clicked that lovely refresh button in the UI.
Other merge request events are handled more normally.
Time
GitLab's REST and GraphQL APIs use RFC 3339 dates. As I'm sure you all know, Go can parse these directly out of JSON into time.Time
instances.
GitLab's webhook events are... less consistent. Some fields are RFC 3339, whereas others are not.
As a result, I've updated the structures in the GitLab package to use a new gitlab.Time
instead of time.Time
that can handle them both. There's a bit of extra boilerplate here around type assertions and casts, but it means we can then use the same structures for types that are shared between the REST and webhook APIs, so I'm calling it a win.
Pipelines
Pipeline events are supposed to include the merge request they were triggered by. In practice, they generally do not.
I've speculatively implemented pipeline event support based on the GitLab docs in the hopes that the bug will eventually be addressed. For now, however, I've been unable to actually trigger it in practice because I haven't been able to generate a pipeline event that includes the merge request. Again, we'll fall back to repo-updater
figuring this out when it syncs, since we can get the pipeline events through the REST (and, eventually, GraphQL) API.