extsvc: Change default rate limits of npm and Go external services (!34042) · Merge requests · Warren Gifford / sourcegraph

Something went wrong while fetching comments. Please try again.

Merged Warren Gifford requested to merge main-dry-run/ts/change-package-repos-rate-limits into main 3 years ago

Created by: tsenart

This commit changes the default rate limit configs of npm and Go external services to be unlimited by default, and for a higher default of 16 req/s to be filled in when auto-completing in the Monaco editor of the UI.

Unlike the GitHub or GitLab APIs, neither the public npm registry nor the public Go modules proxy (i.e proxy.golang.org) document an enforced req/s rate limit. For private registries / proxies, site-admins can manually configure their desired rate limits.

The request pattern for npm and Go sources is also different than GitHub and GitLab sources in that they execute a lot more individual requests since there are no batch endpoints used (except for multiple versions of a single package). This means we need a far higher rate limit in order for these external services to sync in a reasonable amount of time, as well as leave some quota for lazy-syncing individual packages via dependencies search.