Skip to content

devx: Measure time from commit push to CI build feedback

Created by: marekweb

Why:

We want to measure how long it takes for CI to go from the moment that a commit is pushed to the moment that the author of the commit has feedback (a build success or a build failure).

This helps us know how long engineers are blocked waiting for feedback on their pushed commits, in real time (including any latency in getting the build started or in getting the result back).

Questions

  • Can we rely on the point in time that we get the result from the Buildkite build? Or do we need to take other GitHub PR checks into consideration?
    • It looks like we can just use the Buildkite build result, because that's the main one that a PR author cares about, and that one is usually the one that takes the longest (in the case of success). In the case of build failures that happen fast, though, the checks may wait on some other check that may take longer, in theory, but I'm not sure how important that is in practice.