Skip to content

worker: refactor packages to allow for OSS usage

Administrator requested to merge aharvey/allow-worker-use into main

Created by: LawnGnome

Defining workers outside of the enterprise package is theoretically supported right now: you can add your worker to the builtin global variable, and then it will be loaded regardless of which enterprise workers are configured.

In practice, though, this is infeasible, for two reasons:

  1. Workers that access the database first do so by invoking shared.InitDatabase(), which is going to result in a circular dependency, as shared.builtin has to import the package defining the worker. You end up with something like shared.builtin -> awesome.Worker -> shared.InitDatabase, and Go rightly complains.
  2. Most workers are going to want to validate that the job they're returning is valid: the pattern used in enterprise workers is to have their constructor return shared.Job instead of the concrete struct type, which may well not even be exported. Again, this will result in a circular import loop for similar reasons to the above.

Instead, we can move the genuinely reusable bits of code — the database handling, the memoisation helper, and the Job interface — out of the main shared package. This means that the imports become acyclic: now the example above looks like shared.builtin -> awesome.Worker -> workerdb.Init.

All that said, I don't particularly love the new structure of cmd/worker: it leaves cmd/worker/shared as a bit of a dumping ground, albeit a slightly more focused one than before.

Merge request reports

Loading