web: make web integration tests great again!
Created by: valerybugakov
TL;DR: shave off 50% of the time from the Pupetteer build step.
DISCLAIMER Diff in the PR is quite small. yarn.lock update adds most of the lines changed
Context
Our Puppeteer integration tests are slow. This PR relies on Percy's ability to parallelize visual testing on multiple Builkite agents. How it works:
- Puppeteer preparation agent builds web application and uploads an artifact to be shared between agents.
- We find all web integration test files, split them in chunks by 3 (empirically found number), and start a separate Buildkite agent for each chunk.
- Chunk agent downloads pre-built artifact and runs integrations tests on a small subset of test files.
- When all tests are completed, the final Buildkite agent kicks in to finalize Percy build.
Changes
- Split web integration tests in chunks to leverage Percy's parallelization.
- Percy CLI is updated to the latest version.
- Chromium version used by Puppeteer is updated to one of the latest versions.
- Added missing methods to the Buildkite golang package to add dependent steps to the pipeline.
Results
- ~8 mins to execute all integration tests.
- This can be scaled indefinitely with more integration tests being added to the web application.
Pipeline generated for integration tests
Related work from @jhchabran https://github.com/sourcegraph/sourcegraph/pull/25158.