Distribution 3.12 tracking issue

Created by: beyang

Mark	Description
🏃	Work in progress
✋	Blocked task
💪	Stretch goal for this iteration
⏱️💪	Status changed to stretch mid-iteration
⌛	Waiting for information from user
🎁	Unexpected addition to milestone
🔴	Missing issue reference
🔵	More investigation required to remove uncertainty
⚫️	Under discussion within the team
crossed out	No longer expect to achieve in iteration.

The work we set out to do in 3.10 was: https://github.com/sourcegraph/sourcegraph/issues/6038

Planned dev work

OKR 1: Critical customer features

• General
  • Enforce pod security policies https://github.com/sourcegraph/deploy-sourcegraph/issues/482 @beyang
  • 💪 GraphQL timeout on very large repositories https://github.com/sourcegraph/sourcegraph/issues/6878 @beyang
  • Version number is wrong for 3.11.0, causing update checker to not work #7442 (closed) @ggilmore @beyang
  • 🏃⌛(anticipated) Forbidden - CSRF token invalid #7368 (closed) @slimsag (identify workaround at least) @slimsag
• Monitor recently reported search issues @slimsag
  • Search indexing monitoring: https://github.com/sourcegraph/deploy-sourcegraph-docker/issues/55
  • search: add prometheus metric(s) about how many repos are indexed/nonindexed #4197 (closed)
  • metrics/alerts for gitserver slow execs and fetches/clones #6675 (closed)
  • 🏃Searcher dashboard incorrectly reports on query-runner #7531 (closed)
  • 🏃Frontend dashboard: "Search responses by status" query is not aggregated #7627 (closed)
• (anticipated) Make it clear to admins how to configure alerting @slimsag
  • Add documentation for setting up Grafana alerting #7523 (closed)
• (anticipated?) Monitor GitLab integration @uwedeportivo
  • alerts for code intelligence latency
  • alert gitserver latency spikes https://github.com/sourcegraph/sourcegraph/issues/6651
  • metric labeling https://github.com/sourcegraph/sourcegraph/issues/6650
  • disk space alerts https://github.com/sourcegraph/sourcegraph/issues/5994 @uwedeportivo @slimsag

OKR 2: Features and docs that accelerate customer journey

• Document scaling recommendations https://github.com/sourcegraph/sourcegraph/issues/6274 @beyang
• Monitoring: insight into instance over/under-provisioned-ness @slimsag
  • Add unified set of container / pod Prometheus metrics #5547 (closed)
  • Add monitoring/alerts to signal whether instance is underprovisioned or not #7529 (closed)
• catch-all: Improve onboarding / initial setup documentation, auth documentation, etc @slimsag
  • ✅18 issues identified from running a user onboarding test: https://github.com/sourcegraph/sourcegraph/issues?q=is%3Aissue+is%3Aopen+label%3Aonboarding-test-1

OKR 3: Improve developer productivity

• Move e2e tests into regression test suite https://github.com/sourcegraph/sourcegraph/issues/5721 @beyang
• Add regression tests that ensure diff, commit, and repo search work properly https://github.com/sourcegraph/sourcegraph/issues/7102 @beyang
• Eliminate ephemeral alerts RFC 35 @uwedeportivo