Skip to content

a8n: Use a persistent queue for CampaignJobs and ChangesetJobs

Created by: mrnugget

Runner executes CampaignJobs in goroutines, a8n.Service executes ChangesetJobs in goroutines.

Sooner or later, we probably want to use a "persistent queue" instead of simply scheduling goroutines.

Why? If the frontend crashes while CampaignJobs/ChangesetJobs are executed, progress is lost and we'd be in a state we can't get out of (at least for previewCampaignPlan).

By separating the enqueuing of a job (i.e. creating a ChangesetJob/CampaignJob in the database) and its execution (i.e. a separate automation-workers process that checks Postgres for new jobs) we could

  1. offload work from frontend to another process, making it easier to scale/run
  2. make it more resilient