Code monitor webhooks & Slack notifications support

Created by: dadlerj

Product document can be found here but was a point-in time document used primarily at the start of the project in FY22 Q3.

Problem to solve

Enterprises need to communicate important code changes to stakeholders in an efficient, timely manner. Currently, they’re only enabled to notify the Code Monitor creator, and then the onus is on them to communicate any pertinent alert information to the wider company after they receive the alert.

With Code Monitor integrations, we aim to make that dissemination as frictionless as possible by providing multiple simple, intuitive ways to integrate Sourcegraph into enterprise communication channels.

Measure of success

Code monitor creation utilizing the new integrations is activated significantly in FY23 Q3, with 30% of existing Code monitor customers enabling at least 1 new integration by end of FY23 Q3
5 code monitors enabled that have a scale of larger than 50 repositories by end of FY23 Q3
3 code monitors enabled that have a scale of larger than 10,000 repositories by end of FY23 Q3

Solution summary

Allow users to manage their own Sourcegraph integrations via a “standard” webhook, and provide UI in the app to set their Sourcegraph webhook URLs and possibly special HTTP headers. Our Slack integration approach will use a webhook and have the user create their own Slack app (it’s now just a few clicks) that takes a JSON payload. As long as users are allowed to create their own Slack apps (Sourcegraph employees can, for example) this allows bottom-up adoption of Code Monitors at the user and team level.

Additionally, attempt to reduce time to value by providing a starter library of a few Code Monitors. We have examples from the team that could be easily customized. We will also provide a searchable list of the Code Monitors that were attempted, a timestamp, and their status (success/failure).

What specific customers are we iterating on the solution with?

This project was done in tandem with Code monitors performance improvements because the 50 repository limit that existed on code monitors at the time of this project's inception had been identified as a major blocker to adoption in addition to the lack of webhook or Slack integration. Because of that, are not yet actively iterating on this project with any customers. However, the frontend changes to support it were relatively minimal, and we took an MVP approach to the Slack integration.

Product gaps and their customers/prospects for these projects

Code monitoring webhook/custom email

Unlimited Repo Limit for Code Monitoring

Impact on use cases

This work (and critically, the supporting performance work) will have an impact on our Code security and Incident response use cases especially. Our internal security team at Sourcegraph is already using the Slack integration to send monitor events when certain monitors are triggered. Additionally, webhook integration will allow users to send the monitor output to any internal alerting system that can consume webhooks.

Delivery plan

Reach out to current users of Code monitors to notify them of the integrations and performance improvements and offer trainings
Monitor new pings created to measure the impact of these integrations on usage

Archived ticket description

Currently saved searches can only send email updates. We previously supported Slack updates, but that was removed when Slack's API changed.

Multiple users have requested webhooks as a simple solution to this need:

There will be two types of webhooks: generic, and Slack notifications. Generic webhooks will be called with a generic schema, while Slack webhooks will send a message formatted for display in Slack. For the initial release, no authentication is supported. Generic webhooks will be called without any authentication. Slack users are required to create an application and add webhooks for the channels they want notifications for.