Skip to content
Snippets Groups Projects

monitoring(repo-updater): add duration on critical rate limit alerts, update next steps

Merged Warren Gifford requested to merge monitoring-update-ratelimit-critical-alerts into main

Created by: bobheadxi

For https://github.com/sourcegraph/sourcegraph/issues/36434#issuecomment-1148000738

  1. @michaellzc noted that the pod restart advice is not great:

Restarting the pod doesn't guarantee a new public IP, and it's possible customers are using a public gateway (e.g. Cloud NAT) with VM without a public IP address

  1. It's also noted that this comes up quite frequently - I think on larger instances running up against the limit is quite common and even expected, and Sourcegraph should for the most part continue working even if it exhausts its rate limits. It becomes a critical issue only if the rate limit is exhausted immediately after a rate limit reset. We can detect this by checking to see if the rate limit is below the threshold for ~most of a window.

Test plan

n/a

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
Please register or sign in to reply
Loading