Merged GitHub changeset shown as "closed" until next sync when webhooks are activated

Created by: mrnugget

I thought this was fixed, but it seems there's another bug.

I merged some changesets in this campaign and noticed that they showed up as "closed" until I manually enqueued another sync.

So I picked out another PR (https://github.com/sourcegraph/go-lsp/pull/9) to observe what happens.

Here's the data in the database before I merged the PR:

changeset before merge:

SELECT changesets.* from changesets JOIN repo ON repo.id = repo_id where repo.name = 'github.com/sourcegraph/go-lsp' AND campaign_ids ? '8';
id                    | 21
campaign_ids          | {"8": null}
repo_id               | 156375
created_at            | 2020-03-03 18:04:20.90578+00
updated_at            | 2020-04-20 23:50:43.240533+00
metadata              | {"ID": "MDExOlB1bGxSZXF1ZXN0MzgzMDk2Njg1", "URL": "https://github.com/sourcegraph/go-lsp/pull/9", "Body": "This campaign is part of our [Campaigns delivery plan](https://github.com/sourcegraph/customer/issues/13).\n\nIt runs `go mod tidy` in every project in our `sourcegraph` GitHub organization if it has a `go.mod` file.\n\nFind this campaign [here](https://k8s.sgdev.org/campaigns/Q2FtcGFpZ246OA==)", "State": "OPEN", "Title": "Run go mod tidy over Go projects", "Author": {"URL": "https://github.com/sourcegraph-bot", "Login": "sourcegraph-bot", "AvatarURL": "https://avatars1.githubusercontent.com/u/15893131?u=a138ed8274844e043250363a9efb971e133c0936&v=4"}, "Labels": {"Nodes": []}, "Number": 9, "Commits": {"Nodes": [{"Commit": {"OID": "d578150d6b7d225ac98eeec05a13b02669a44763", "Status": {"State": "", "Contexts": null}, "CheckSuites": {"Nodes": [{"ID": "MDEwOkNoZWNrU3VpdGU2MDY0NDg2MTY=", "Status": "QUEUED", "CheckRuns": {"Nodes": []}, "Conclusion": "", "ReceivedAt": "0001-01-01T00:00:00Z"}, {"ID": "MDEwOkNoZWNrU3VpdGU2MDY0NDg2MTc=", "Status": "QUEUED", "CheckRuns": {"Nodes": []}, "Conclusion": "", "ReceivedAt": "0001-01-01T00:00:00Z"}, {"ID": "MDEwOkNoZWNrU3VpdGU2MDY0NDg2Mzk=", "Status": "COMPLETED", "CheckRuns": {"Nodes": [{"ID": "MDg6Q2hlY2tSdW41OTU3NTM2NjI=", "Status": "COMPLETED", "Conclusion": "SUCCESS", "ReceivedAt": "0001-01-01T00:00:00Z"}]}, "Conclusion": "SUCCESS", "ReceivedAt": "0001-01-01T00:00:00Z"}]}, "CommittedDate": "2020-03-03T18:04:13Z"}}]}, "CreatedAt": "2020-03-03T18:04:19Z", "UpdatedAt": "2020-04-17T15:11:40Z", "BaseRefOid": "b19bb38222e24b57a93b803229b83f1eb113095d", "HeadRefOid": "d578150d6b7d225ac98eeec05a13b02669a44763", "BaseRefName": "master", "HeadRefName": "go-mod-tidy-sourcegraph", "Participants": [{"URL": "https://github.com/sourcegraph-bot", "Login": "sourcegraph-bot", "AvatarURL": "https://avatars1.githubusercontent.com/u/15893131?u=a138ed8274844e043250363a9efb971e133c0936&v=4"}], "TimelineItems": [{"Item": {"Commit": {"OID": "d578150d6b7d225ac98eeec05a13b02669a44763", "URL": "https://github.com/sourcegraph/go-lsp/commit/d578150d6b7d225ac98eeec05a13b02669a44763", "Message": "Run go mod tidy over Go projects", "Committer": {"Name": "Sourcegraph Bot", "Email": "[email protected]", "AvatarURL": "https://camo.githubusercontent.com/33d4c509ae479643c65ba8967c16b869c1558e90/68747470733a2f2f322e67726176617461722e636f6d2f6176617461722f34366333653536396163366530356264326136366339313661396666343934633f643d68747470732533412532462532466769746875622e6769746875626173736574732e636f6d253246696d6167657325324667726176617461727325324667726176617461722d757365722d3432302e706e6726723d67"}, "PushedDate": "2020-04-17T15:11:39Z", "CommittedDate": "2020-03-03T18:04:13Z", "MessageHeadline": "Run go mod tidy over Go projects"}}, "Type": "PullRequestCommit"}, {"Item": {"Actor": {"URL": "https://github.com/sourcegraph-bot", "Login": "sourcegraph-bot", "AvatarURL": "https://avatars1.githubusercontent.com/u/15893131?u=a138ed8274844e043250363a9efb971e133c0936&v=4"}, "CreatedAt": "2020-04-17T15:11:39Z", "CurrentTitle": "Run go mod tidy over Go projects", "PreviousTitle": "Go mod tidy sourcegraph"}, "Type": "RenamedTitleEvent"}]}
external_id           | 9
external_service_type | github
external_deleted_at   | [NULL]
external_branch       | go-mod-tidy-sourcegraph
external_updated_at   | 2020-04-17 15:11:40+00
external_state        | OPEN
external_review_state | PENDING
external_check_state  | PASSED

changset_events before merge:

SELECT id, kind, created_at, updated_at FROM changeset_events where changeset_events.changeset_id = 21 ORDER BY updated_at ASC;
   id   |      kind      |          created_at           |          updated_at
--------+----------------+-------------------------------+-------------------------------
 781648 | github:commit  | 2020-03-15 06:39:51.976868+00 | 2020-04-17 11:15:25.9431+00
 793152 | github:commit  | 2020-04-17 15:11:40.872979+00 | 2020-04-20 23:50:43.244705+00
 793153 | github:renamed | 2020-04-17 15:11:40.872979+00 | 2020-04-20 23:50:43.244705+00
(3 rows)

Now the data after merging the PR (which showed up as "closed"):

After merge:

changeset:

[email protected]:3333 sg# SELECT changesets.* from changesets JOIN repo ON repo.id = repo_id where repo.name = 'github.com/sourcegraph/go-lsp' AND campaign_ids ? '8';
-[ RECORD 1 ]---------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
id                    | 21
campaign_ids          | {"8": null}
repo_id               | 156375
created_at            | 2020-03-03 18:04:20.90578+00
updated_at            | 2020-04-21 05:23:24.770811+00
metadata              | {"ID": "MDExOlB1bGxSZXF1ZXN0MzgzMDk2Njg1", "URL": "https://github.com/sourcegraph/go-lsp/pull/9", "Body": "This campaign is part of our [Campaigns delivery plan](https://github.com/sourcegraph/customer/issues/13).\n\nIt runs `go mod tidy` in every project in our `sourcegraph` GitHub organization if it has a `go.mod` file.\n\nFind this campaign [here](https://k8s.sgdev.org/campaigns/Q2FtcGFpZ246OA==)", "State": "OPEN", "Title": "Run go mod tidy over Go projects", "Author": {"URL": "https://github.com/sourcegraph-bot", "Login": "sourcegraph-bot", "AvatarURL": "https://avatars1.githubusercontent.com/u/15893131?u=a138ed8274844e043250363a9efb971e133c0936&v=4"}, "Labels": {"Nodes": []}, "Number": 9, "Commits": {"Nodes": [{"Commit": {"OID": "d578150d6b7d225ac98eeec05a13b02669a44763", "Status": {"State": "", "Contexts": null}, "CheckSuites": {"Nodes": [{"ID": "MDEwOkNoZWNrU3VpdGU2MDY0NDg2MTY=", "Status": "QUEUED", "CheckRuns": {"Nodes": []}, "Conclusion": "", "ReceivedAt": "0001-01-01T00:00:00Z"}, {"ID": "MDEwOkNoZWNrU3VpdGU2MDY0NDg2MTc=", "Status": "QUEUED", "CheckRuns": {"Nodes": []}, "Conclusion": "", "ReceivedAt": "0001-01-01T00:00:00Z"}, {"ID": "MDEwOkNoZWNrU3VpdGU2MDY0NDg2Mzk=", "Status": "COMPLETED", "CheckRuns": {"Nodes": [{"ID": "MDg6Q2hlY2tSdW41OTU3NTM2NjI=", "Status": "COMPLETED", "Conclusion": "SUCCESS", "ReceivedAt": "0001-01-01T00:00:00Z"}]}, "Conclusion": "SUCCESS", "ReceivedAt": "0001-01-01T00:00:00Z"}]}, "CommittedDate": "2020-03-03T18:04:13Z"}}]}, "CreatedAt": "2020-03-03T18:04:19Z", "UpdatedAt": "2020-04-17T15:11:40Z", "BaseRefOid": "b19bb38222e24b57a93b803229b83f1eb113095d", "HeadRefOid": "d578150d6b7d225ac98eeec05a13b02669a44763", "BaseRefName": "master", "HeadRefName": "go-mod-tidy-sourcegraph", "Participants": [{"URL": "https://github.com/sourcegraph-bot", "Login": "sourcegraph-bot", "AvatarURL": "https://avatars1.githubusercontent.com/u/15893131?u=a138ed8274844e043250363a9efb971e133c0936&v=4"}], "TimelineItems": [{"Item": {"Commit": {"OID": "d578150d6b7d225ac98eeec05a13b02669a44763", "URL": "https://github.com/sourcegraph/go-lsp/commit/d578150d6b7d225ac98eeec05a13b02669a44763", "Message": "Run go mod tidy over Go projects", "Committer": {"Name": "Sourcegraph Bot", "Email": "[email protected]", "AvatarURL": "https://camo.githubusercontent.com/33d4c509ae479643c65ba8967c16b869c1558e90/68747470733a2f2f322e67726176617461722e636f6d2f6176617461722f34366333653536396163366530356264326136366339313661396666343934633f643d68747470732533412532462532466769746875622e6769746875626173736574732e636f6d253246696d6167657325324667726176617461727325324667726176617461722d757365722d3432302e706e6726723d67"}, "PushedDate": "2020-04-17T15:11:39Z", "CommittedDate": "2020-03-03T18:04:13Z", "MessageHeadline": "Run go mod tidy over Go projects"}}, "Type": "PullRequestCommit"}, {"Item": {"Actor": {"URL": "https://github.com/sourcegraph-bot", "Login": "sourcegraph-bot", "AvatarURL": "https://avatars1.githubusercontent.com/u/15893131?u=a138ed8274844e043250363a9efb971e133c0936&v=4"}, "CreatedAt": "2020-04-17T15:11:39Z", "CurrentTitle": "Run go mod tidy over Go projects", "PreviousTitle": "Go mod tidy sourcegraph"}, "Type": "RenamedTitleEvent"}]}
external_id           | 9
external_service_type | github
external_deleted_at   | [NULL]
external_branch       | go-mod-tidy-sourcegraph
external_updated_at   | 2020-04-17 15:11:40+00
external_state        | CLOSED
external_review_state | PENDING
external_check_state  | PASSED

changeset_events:

[email protected]:3333 sg# SELECT id, kind, created_at, updated_at FROM changeset_events where changeset_events.changeset_id = 21 ORDER BY updated_at ASC;
   id   |      kind      |          created_at           |          updated_at
--------+----------------+-------------------------------+-------------------------------
 781648 | github:commit  | 2020-03-15 06:39:51.976868+00 | 2020-04-17 11:15:25.9431+00
 793152 | github:commit  | 2020-04-17 15:11:40.872979+00 | 2020-04-20 23:50:43.244705+00
 793153 | github:renamed | 2020-04-17 15:11:40.872979+00 | 2020-04-20 23:50:43.244705+00
 794117 | github:closed  | 2020-04-21 05:23:24.767655+00 | 2020-04-21 05:23:24.768239+00
(4 rows)

The changeset is "correctly" reported as "closed" because we've only received a github:closed event. And since the changeset is then marked as "closed", we the next sync is in 8 hours.

Multiple questions:

Why do we receive only a github:closed event?
Does this github:closed event include an updated changeset that we can use to update the changeset?
Should we stop delaying the sync if the "closed" event is super fresh? (The chance that it might be merged, reopened, etc. shortly after decreased with age of the the "closed" event.)

cc @sourcegraph/campaigns-core