Skip to content

packages: don't abort syncing process if one package fails

Warren Gifford requested to merge mrn/package-host-sync-err into main

Created by: mrnugget

We ran into this after discovering that an invalid name inserted into the lsif_dependency_repos table could cause the whole syncing process to stop.

What would happen:

  1. Closure does return err
  2. The errgroup.WithContext would cancel the ctx since one goroutine returned an error
  3. All already running goroutines would finish (maximum 32 - size of semaphore)
  4. Next call to ListDependencyRepos would fail, because the context is cancelled.
  5. Whole loop is stopped and we end up with two errors: the one actual error from the invalid package and the "context canceled" from ListDependencyRepos.

This changes the behaviour to make use of the infrastructure and report an error only via SourceResult. That causes the error to be recorded and the sync job to be marked as failed, but it still syncs the other packages.

Test plan

  • Tested this manually by adding an NPM package host with multiple valid packages to my instance, then doing an insert into lsif_dependency_repos (name, version, scheme) values ('@automapper/classes/transformer-plugin', '0.0.0', 'npm'); to add the invalid package from production. That way I could check what happens and how my change here fixes it.

Merge request reports

Loading