insights: split query runner into a separate worker job in the worker service
Created by: coury-clark
Code insights starts multiple background jobs in the worker service. The worker service supports distributing jobs across multiple nodes. The Code Insights query runner job can support horizontally scaling and would benefit from being an independent worker job. This task is to split out the worker / work_handler background goroutine into a separate and independently scalable job installed on the worker service.
To clarify a few things:
worker service is a piece of the Sourcegraph architecture (it's a separate pod)
worker / workerutil is an abstraction of a job running mechanism backed by a queue stored on postgres
It's a confusing collision of names, for sure.