Skip to content

RFC ci: Only E2E run for releases

Warren Gifford requested to merge ci/remove-e2e into master

Created by: keegancsmith

I expect this to be a bit controversial. This is a request for comment in the form of a PR. Please state your opinions as comments. cc @sourcegraph/code-intel @sourcegraph/core-services @sourcegraph/distribution

E2E test reliability only seems to be getting worse. Just today I have had to hit retry 3 times on builds I landed in master. This has been an ongoing issue, and I believe we have reached the point where running E2E on PRs/master provides more bad than good. This is one possible short-term solution.

PRs and master will no longer run E2E tests. This should improve reliability and speed of CI for the general case, while still ensuring e2e tests pass on tagged releases.

The downside is we won't notice severe regressions to e2e, so will have to deal with the issues once we do a release. But as it stands the e2e tests are in a bad shape and no one is fixing them. We are building a pavlovian response to e2e failing and the retry button.

Merge request reports

Loading