Add feature flag tooling
Created by: arussellsaw
Right now we don't have any canonical way of managing feature flags, and are using a few tools that don't quite meet our needs, which are user tags and site-wide config options. As we start to ship sourcegraph.com and work towards increasing adoption, we'll want to be spending more time running experiments and A/B testing our releases, so having some tooling in place to make this easier would be really useful.
Ideally the first version of feature flagging would support things like:
- percentage rollouts
- manual overrides
- user group rollouts
- analytics & reporting
percentage rollouts would apply to new and existing users via something like a hash ring, a 50% rollout would be hash(userid+flagname)%100 <= 50 so that when we apply a rollout we're not manually assigning flags to users, rather evaluating rollout when we check the flag against a given user ID. as far as events go, we would want to emit events when the rollout is changed, and have an easy way of attaching feature flag rollout status to other analytics events to help with A/B testing.
Using a single package to manage feature flags would also give us an easy way to get an overview of all feature flags currently in use, along with rollout status, which will allow us to keep on top of trimming any completed or cancelled experiments.
As far as managing feature flag state goes i'd probably go for a postgres table rather than config, as it allows us to change manual overrides and rollouts more quickly, rather than needing to edit config.