insights: split query runner into a separate worker job in the worker service
Created by: coury-clark
Code insights starts multiple background jobs in the worker
service. The worker
service supports distributing jobs across multiple nodes. The Code Insights query runner
job can support horizontally scaling and would benefit from being an independent worker
job. This task is to split out the worker / work_handler
background goroutine into a separate and independently scalable job installed on the worker
service.
To clarify a few things:
worker
service is a piece of the Sourcegraph architecture (it's a separate pod)
worker / workerutil
is an abstraction of a job running mechanism backed by a queue stored on postgres
It's a confusing collision of names, for sure.