Skip to content

A/B testing philosophy and process

Warren Gifford requested to merge add-ab-testing-philosophy into main

Created by: malomarrec

Guidelines about how we chose, track, run, analyse and cleanup A/B tests.

Needs

  • cleanup and generalization @malomarrec
  • conventions for flags, and tracking that in tickets @malomarrec
  • create a "tracking issue" template for A/B tests @malomarrec
  • link to relevant "how to" material, and how our A/B testing infra works @camdencheek

Future work

  • how to use Amplitude to analyze A/B tests @ebrodymoore
  • define a source of truth for A/B tests results (moved to: https://github.com/sourcegraph/about/issues/3591) Discussion
  • Should A/B test conclusions be documented somewhere (in a source of truth)? Should that be an issue "A/B test report" that we close? Something else?

Merge request reports