Skip to content

Add an interactive docker-compose and Kubernetes resource estimator

Administrator requested to merge sg/resource-estimator into master

Created by: slimsag

This adds an interactive resource estimator to our docs:

  • You punch in your best guesses for # of users, repositories, monorepos, engagement rate that you expect to have on Sourcegraph.
  • We tell you the best deployment type (Docker-compose/Kubernetes), a really accurate starting point for replicas/CPUs/memory for all services, etc. based on deployments we know work well but tailored towards your inputs.

Prior to having this, when customers asked "What resources should I start out with?" I would manually and tediously consider these same factors in a less well-define manner to provide a starting point (I can list several customers i have done this for in the past and each time it has taken several hours). Now this process is automated and self-service, so anyone can do it.

Note this doesn't replace the need for admins to understand how Sourcegraph scales -- it just answers that initial question of "where do I start?"

How it looks

Demo video coming soon, screenshot to appease you:

image

The results

The overwhelming majority of my time was spent making sure this would produce valid, and actually useful, results. I will explain in proper documentation exactly how this does works -- but in short it uses a series of reference points from known-good-deployments along with some educated understandings about how each of our services scale, and some projections/speculation in some cases, in order to identify two reference points we can interpolate the user input onto to get a final answer that is quite good.

Technology

Originally I wrote the resource estimator in JS because I just wanted to KISS and just dump the script inline in a Markdown file in our docsite. However, quickly I realized that the logic needed here was more complex than I could sanely do in JS without types. This left me with two options:

  • Write it in TS+React
  • Write it as a Go CLI

Short on time and fearing I would waste time on stupid things or get stuck due to my lack of TS knowledge if I went with the first option, I opted for the second option.

But this being a CLI felt weird and overly tedious. It felt like something we could use to advise users -- but not something we could easily tell them to play around with and self-service. It needed to be on the docsite. So I scrapped the CLI idea and chucked together a Go WebAssembly app that takes some basic form inputs and generates+renders the markdown (effectively all the CLI did) and displays it on the page. I know TS+React is definitely the right approach for more serious things, but given this was a one-off project and small/isolated, this felt okay to do and it meant I could have it in the web whereas otherwise I would've had to settle on just the CLI.

The dev experience is really nice, it works really well, the only downside to this approach is that I didn't spend any time optimizing the WASM bundle and as such it is large (~3.8MB).

Try it out

  1. Checkout this branch.
  2. Launch ./enterprise/dev/start.sh
  3. Navigate to http://localhost:5080/admin/install/resource_estimator

Reviewing

The most useful thing to review here is just: as a consumer of this, what works the way you expect, and what doesn't? What is confusing for you?

I have some further improvements (specifically w.r.t. testing and documentation) for the code in https://github.com/sourcegraph/resource-estimator which will come soon, but.there is an immediate need to give this to a customer tomorrow so this may be merged prematurely.

Merge request reports