repo-updater: Stream inserts
Created by: keegancsmith
Syncing requires fetching from all sources before it can update the database. For large instances this leads to a long time (minutes) from the time an admin makes a change to an external service to seeing the effects. The most common complaint is adding repositories to the external service, and not seeing them. This commit will eagerly add new repositories into the store before running the full sync. This is a much smaller change than making the sync engine fully stream based.
This is an alternative approach to making repo-updater completely stream based. The important observation is that users complain that they add a repository to an external service and don't see it. This change doesn't adjust how repo-updater's core sync algorithm works. Instead it will eagerly insert any repository from a source which is not already in the store.
Alternative to https://github.com/sourcegraph/sourcegraph/pull/5526
Fixes https://github.com/sourcegraph/sourcegraph/issues/5145