Skip to content

search: dependencies search prototype

Warren Gifford requested to merge main-dry-run/dependency-search into main

Created by: tsenart

Context

This PR introduces dependency search of package repos as a prototype of RFC 593.

It introduces a new repo:dependencies predicate (and its alias deps) that receive a repo-regex@revA:revB:... argument which will be an input to resolving the set of repos to search over. We handle it by finding all matching repos, fetching supported lockfiles from gitserver, parsing them and finally converting those dependencies to repos to search over, triggering a sync of all existing package host external services at the end.

This means the first time we use r:deps with repos that contain new dependencies in the instance, we won't be able to search those immediately, but they should be shortly available thereafter (i.e. lazy syncing).

We only support searching over NPM package repos, since we only have a package-lock.json parser implemented. Once more parsers are implemented (e.g. yarn.lock) for package hosts we already support, we will expand the set of searchable dependency repos.

For now we are not storing the relationships in a way that we can query directly from the database — every single search with dependencies in it needs to re-parse lockfiles.

Test plan

Unit, integration and manual end to end tests.

Merge request reports

Loading