Make changeset syncer a dbworker
Created by: eseliger
We currently have the stateful changeset syncer in repo-updater that can be reached via RPC from frontend to prioritize certain changesets with a mutation.
The problems:
- We don't want this to run in repo-updater, and in worker we don't have a fixed hostname to talk to, so RPC is not possible for prioritized dequeues
- We want to benefit from other worker things like pre-built metrics, and in the future the #26895
- There is no insight into the syncer because its entire queue is held in memory
Not 100% sure how this will look in the end, but I assume we will have a "sync scheduler" service that creates changset_sync_jobs
rows that are attached to a dbworker. That way, we can also just add a new sync record with high_priority TRUE
and have an order by statement for the dequeues that looks like high_priority DESC, id DESC
. I assume this will also give better ability to show in the UI when a changeset was actually synced last and when a current user-triggered sync job has been enqueued (as it'll live in the DB, not repo-updaters memory). Also, we can scale the changeset syncers across multiple machines or have more simultaneous workers, which is currently not possible with the architecture.