Lighthouse CI currently only runs on the sourcegraph-async pipeline. This is because it is not currently asserting against anything so should not block any builds. This is also a good opportunity to test it for any flakiness without risking breaking builds
Building:
Build a production bundle of the Sourcegraph web app
Upload bundle to Buildkite so we can parallelize upcoming Lighthouse runs for perf improvements
Running:
Download prebuilt production bundle artifact
Run Lighthouse through Puppeteer.
2.1. Config derived from lighthouse.js and some additional flags that are only relevant in CI
Reports back to the Lighthouse GitHub app using the LHCI_GITHUB_APP_TOKEN set in Buildkite.
3.1. Status checks are attached to PRs with the results of each performance run
3.2. If running on main, we store the results of the build so future runs can generate comparison reports.
Local flow
Locally there is no need to upload reports and update PR status checks. You can still configure Lighthouse through the CLI, or by updating lighthouse.js. Just run yarn test-lighthouse and a report will be opened automatically when the tests have finished
Notes
It is quite difficult to get our local 'production' server to mimic how it would be on a deployed site. There are some caveats to address:
Local reports have a lower SEO score than in production. This is because the meta-description tag is attached dynamically in the actual production server, we don't mimic this locally. It is also because we do not serve our robots.txt file locally.
Local reports have a lower 'Best practices' score than in production. This is mainly because we do not support HTTPS locally.
The 'First Contentful Paint' is consistently around 0.6s on local reports. In production it averages between 1.0s - 1.6s. Need to investigate further what is happening here.
Aside from this, it is quite accurate compared to production results.
Next steps
Improve accuracy by making the local production server closer to actual production server.
Enforce minimum performance requirements by creating assertions and a performance budget.
Best implementation here would likely be to initially set our performance budget low, and slowly raise it as we improve various performance metrics until we hit our targets.
Enforce other assertions available through Lighthouse checks (e.g. accessibility, best practices, SEO)
