Skip to content

Configurable sync limits + rate limiter wait metrics

Warren Gifford requested to merge rg/CLOUD-304-sync-limits into main

Created by: rafax

This PR adds site-config options for configuring the number of users / repos for which we sync permissions in a single iteration of PermsSyncer (currently hardcoded to 10). To keep permissions fresh-enough, we'd need to schedule

batch_size=(number_of_permission_owners)/(age_window_minutes*sync_per_minute)

syncs in a single iteration - 565 repos / 236 users per sync with 30 min SLA, 47 repos / 20 users per sync with 6h SLA.

This is a large increase over what we do today, so this PR adds a metric tracking time spent waiting for rate limiter (not to add more work than we can get through).

Test plan

Updated unit tests Verified locally (by reconfiguring parameters / verifying rate limiter metric is updated).

Merge request reports