Skip to content

ci: move e2e tests to separate buildkite pipeline

Created by: ggilmore

Since e2e test failures no longer block commits for PRs, we can save a lot of time on CI builds if we moved the e2e steps to a separate buildkite pipeline.

More specifically,

  • Building the sourcegraph/server:*_candidate images takes ~8 minutes (example)
  • Running the e2e tests on the dedicated e2e test machine takes ~3 minutes (example)

So these steps collectively take >10+minutes to run. This is further exacerbated since only 1 e2e test can be running at a time across all sourcegraph/sourcegraph builds. Having to wait that long for non-blocking steps in PRs is painful if we only want quick feedback about whether or not unit tests/linters are passing, etc.

Using a separate "e2e" pipeline should still give us a separate (non-blocking) commit status for the e2e results on PRs.

For the master branch, we can use "trigger" steps, to have the "e2e" pipeline explicitly kick off the Docker builds for each of our services (frontend etc) when the e2e tests pass.