Skip to content

In-product site configuration

Created by: sqs

Site configuration is how site admins configure things for their Sourcegraph instance:

  • repository mirroring (from GitHub, GitLab, and other code hosts)
  • user authentication providers
  • email (SMTP credentials)
  • and other things

The full list of things is in site.schema.json.

There are 3 problems with this that we want to solve, in order of importance:

  • When running a Sourcegraph cluster, you can't edit the site config on the web. You need to edit a k8s configmap and then wait for a reload (or manually kill pods).
  • Sourcegraph can't automatically migrate the site config content when fields change. (For example, the auth.saml -> auth.provider -> auth.providers change was painful.) This is true in both sourcegraph/server and cluster, because the config file is "owned" by the site admin, not by Sourcegraph.
  • Tech debt: Propagating frontend values to other services (such as the list of gitservers) is weird.
  • (Out of scope right now) Site config contains secrets, such as TLS keys, GitHub tokens, etc., that are sourced from central config management systems and should not be present in unencrypted form on disk (for some customers).

The solution must abide by the following:

  • Currently a site admin on sourcegraph/server can make a change (on sourcegraph/server, and on cluster when we solve the 1st problem) that takes down their site, such as changing appURL. Right now, they can fix the problem by manually editing the site config file, but when we solve the 2nd problem, this might no longer be possible. This needs to not become a problem.
  • It is OK to make certain fields in site configuration harder to edit (such as appURL and auth.providers) in exchange for making most others easier to edit.

Proposed solution:

  1. Move site config that can't take down the site (i.e., all except appURL, auth.providers, etc.) out of the site config file and into "core site config", and make both be stored in the DB.
  2. Make a new management console UI that allows editing the core config (the site admin page in-app will only allow editing the site config, the management console UI must be used to edit core config).
  3. The management console UI has extra work needed around authentication, TBD at a later date once initial management console UI is complete.
  4. (later?) Get https://github.com/sourcegraph/sourcegraph/pull/929 finished to remove more things from the "dangerous site config".

Decisions:

  • "Deployment config" (PG*, REDIS_*, SRC_GITSERVERS) is renamed to "service connections".

Levels:

  • User settings
  • Org settings
  • Global settings
  • Site configuration
  • Core configuration -> Critical configuration
  • Deployment configuration -> Cluster service connections

Discussion:

  • Is it important to store the "dangerous" site config outside the database? That lets you store "deployment configuration" in the config file as well.
    • Deployment configuration: PG*, REDIS_*, which gitserver instances to talk to

How other products solve this:

  • GitLab's gitlab.rb config file
    • GitLab deciding on configurations: "When we have no choice, the secondary priority is to configure something in the GitLab interface. A configuration should only appear in a file (gitlab.rb or gitlab.yml) as a last resort."
  • GitHub Enterprise's ini file

Draft announcement (from before kickoff)

Project status:

  • Last week I moved significant portions of #966 into smaller PRs that I can land tomorrow with a strong guarantee of no regressions. (see PRs linked to from that one)
  • I still need to address some TODOs in #966 before it itself can be merged, I am hoping to have that completed by Tue/Wed.
  • The management-console itself still needs a UX (even if primitive / just a textarea for now). I am hoping to have the primitive version done late Wed / mid Thur.