Skip to content

Make changeset syncer a dbworker

Created by: eseliger

We currently have the stateful changeset syncer in repo-updater that can be reached via RPC from frontend to prioritize certain changesets with a mutation.

The problems:

  • We don't want this to run in repo-updater, and in worker we don't have a fixed hostname to talk to, so RPC is not possible for prioritized dequeues
  • We want to benefit from other worker things like pre-built metrics, and in the future the #26895
  • There is no insight into the syncer because its entire queue is held in memory

Not 100% sure how this will look in the end, but I assume we will have a "sync scheduler" service that creates changset_sync_jobs rows that are attached to a dbworker. That way, we can also just add a new sync record with high_priority TRUE and have an order by statement for the dequeues that looks like high_priority DESC, id DESC. I assume this will also give better ability to show in the UI when a changeset was actually synced last and when a current user-triggered sync job has been enqueued (as it'll live in the DB, not repo-updaters memory). Also, we can scale the changeset syncers across multiple machines or have more simultaneous workers, which is currently not possible with the architecture.