Skip to content

web: make web integration tests great again!

Warren Gifford requested to merge vb/apollo-cache-persist-test into main

Created by: valerybugakov

TL;DR: shave off 50% of the time from the Pupetteer build step. DISCLAIMER Diff in the PR is quite small. yarn.lock update adds most of the lines changed 🙂

Context

Our Puppeteer integration tests are slow. This PR relies on Percy's ability to parallelize visual testing on multiple Builkite agents. How it works:

  1. Puppeteer preparation agent builds web application and uploads an artifact to be shared between agents.
  2. We find all web integration test files, split them in chunks by 3 (empirically found number), and start a separate Buildkite agent for each chunk.
  3. Chunk agent downloads pre-built artifact and runs integrations tests on a small subset of test files.
  4. When all tests are completed, the final Buildkite agent kicks in to finalize Percy build.

Changes

  • Split web integration tests in chunks to leverage Percy's parallelization.
  • Percy CLI is updated to the latest version.
  • Chromium version used by Puppeteer is updated to one of the latest versions.
  • Added missing methods to the Buildkite golang package to add dependent steps to the pipeline.

Results

  1. ~8 mins to execute all integration tests.
  2. This can be scaled indefinitely with more integration tests being added to the web application.
Screenshot 2021-09-21 at 15 34 26

Pipeline generated for integration tests

Screenshot 2021-09-21 at 15 34 41

Related work from @jhchabran https://github.com/sourcegraph/sourcegraph/pull/25158.

Merge request reports

Loading