Skip to content

Implement step-wise caching for batch spec execution

Warren Gifford requested to merge mrn+kr/step-wise-caching into main

Created by: mrnugget

This adds step-wise caching for the execution of batch specs.

In short, given a batch spec that contains the following

on:
  - repository: github.com/sourcegraph/automation-testing
  - repository: github.com/sourcegraph-testing/mkcert

steps:
  - run: echo "this is step 1" >> caching.txt
    container: alpine:3
  - run: echo "this is step 2" >> README.md
    container: alpine:3
  - run: echo "this is step 3" >> README.md
    container: alpine:3
    outputs:
      myOutput:
        value: "what is up"
  - run: echo "this is step 4" >> caching.txt
    if: ${{ eq repository.name "github.com/sourcegraph/automation-testing" }}
    container: alpine:3

this PR would cause src to cache the results _per step and per repository.

In practice, this means that if we would change the batch spec from the above so it would look like this

steps:
  - run: echo "this is step 0" >> caching.txt
    container: alpine:3
  - run: echo "this is step 1" >> README.md
    container: alpine:3
    #                         vvvv LOOK HERE vvv
  - run: echo "this is step 2 WITH A CHANGE" >> README.md
    container: alpine:3
    outputs:
      myOutput:
        value: "what is up"
  - run: echo "this is step 3" >> caching.txt
    if: ${{ eq repository.name "github.com/sourcegraph/automation-testing" }}
    container: alpine:3

and we'd then re-execute the batch spec, the steps 0 and 1 would not need to be re-executed. Only step 2 would need to be re-executed for both repositories and step 3 would only need to be re-executed for both repositories.

Merge request reports

Loading