Skip to content

monitoring: configure alert notifications from site config

Warren Gifford requested to merge distribution/alert-notifications into master

Created by: bobheadxi

closes #10641 (closed)

This PR introduces a new workflow:

  1. admin goes to site-admin/configuration and adds:
 "observability.alerts": [
    {
      "id": "email-notifier-1",
      "level": "warning",
      "notifier": {
        "type": "email",
        "addresses": "[email protected]"
      }
    },
    {
      "id": "email-notifier-2",
      "level": "critical",
      "notifier": {
        "type": "email",
        "addresses": "[email protected]"
      }
    },
    {
      "id": "email-notifier-3",
      "level": "critical",
      "notifier": {
        "type": "email",
        "addresses": "[email protected]"
      }
    }
  ]
  1. the Overview dashboard (duplicate of the home) is updated to fire alerts to the above channels when x alerts by service dashboard is >0

This is done by a new grafana-wrapper that subscribes to site configuration updates and makes API calls to grafana to manipulate the alerts overview dashboard and notification channels.

TODO

Merge request reports