worker: refactor packages to allow for OSS usage
Created by: LawnGnome
Defining workers outside of the enterprise package is theoretically supported right now: you can add your worker to the builtin
global variable, and then it will be loaded regardless of which enterprise workers are configured.
In practice, though, this is infeasible, for two reasons:
- Workers that access the database first do so by invoking
shared.InitDatabase()
, which is going to result in a circular dependency, asshared.builtin
has to import the package defining the worker. You end up with something likeshared.builtin -> awesome.Worker -> shared.InitDatabase
, and Go rightly complains. - Most workers are going to want to validate that the job they're returning is valid: the pattern used in enterprise workers is to have their constructor return
shared.Job
instead of the concrete struct type, which may well not even be exported. Again, this will result in a circular import loop for similar reasons to the above.
Instead, we can move the genuinely reusable bits of code — the database handling, the memoisation helper, and the Job
interface — out of the main shared
package. This means that the imports become acyclic: now the example above looks like shared.builtin -> awesome.Worker -> workerdb.Init
.
All that said, I don't particularly love the new structure of cmd/worker
: it leaves cmd/worker/shared
as a bit of a dumping ground, albeit a slightly more focused one than before.