Skip to content

ci: use patch check as informational only

Administrator requested to merge kr/codecov-patch-nonblocking into main

Created by: courier-new

As a follow up to some conversation our team had around not treating failed codecov checks as blockers for merging a PR, I've modified the codecov config so that the codecov/patch check runs in informational mode, which is explicitly recommended for this purpose:

Informational mode is great to use if you want to expose codecov information to other developers in your pull request without necessarily gating PRs on that information.

Screen Shot 2021-06-23 at 3 42 57 PM

For many developers, especially those who are newer to Sourcegraph (and including myself), a failed CI check means a PR is not ready to be merged, regardless of whether or not that check is "required". Knowing when it's safe to ignore a failed check is tribal knowledge and subject to frequent change.

The intention of the codecov/patch check is to encourage developers to write more tests. Anecdotally, however, it seems to fail more regularly on small PRs, where the code changed may not be covered by existing tests, and where writing a new test just to hit the arbitrary 5% target for coverage is not necessarily in scope of the changes. It makes more sense, then, to defer to the author's and reviewers' judgments on test coverage and only use codecov as a suggestion.

As it happens, all the other project-specific codecov checks were already configured to work in informational mode anyway. Ideally, if we fail to hit the 5% target, we would grab attention with a "warning" status on the check or an automatic comment on the PR, but since neither of these are options at present, communicating the information without failing the check seems to be the best way to use this tool.

Merge request reports

Loading