dev/ci: roll QA pipeline into core pipeline
Created by: bobheadxi
Currently, the QA pipeline is effectively just a few steps that runs after all the other tests, alongside the deploy so as to be non-blocking. This PR just pulls in those steps into the core pipeline for simplicity so that we can see everything in one place, while keeping its non-blockingness (as far as deploys go), and integrates it with the rest of the tests. Other advantages:
- enables the QA pipeline to integrate with pipeline generator features and
sg ci
integration, and also simplifiessg ci
implementation by reducing the need to support multiple pipelines.- this is also a bit of a prerequisite for https://github.com/sourcegraph/sourcegraph/issues/25562 to work better when pulling logs from all relevant jobs
- there are a lot of obscure configuration options in the build steps UI of QA: https://buildkite.com/sourcegraph/qa/steps , this unifies all our configuration into the generator and moves the secrets to a better home (https://github.com/sourcegraph/infrastructure/pull/2821)
- implements the concrete proposal to unify our pipelines in #25348 (closed)
This change also "promotes" all QA pipeline jobs into the main block before images get published so that it can start earlier, as soon as the server candidate image is done. This also means that QA tests are now required for deployment.
❯ sg ci preview
Detected run type: Main dry run
Detected Changed files (5):
.buildkite/pipeline.qa.yml
dev/release/src/release.ts
enterprise/dev/ci/images/images.go
enterprise/dev/ci/internal/ci/operations.go
enterprise/dev/ci/internal/ci/pipeline.go
Detected changes:
Affects Go: true
Affects Client: false
Affects Docs: false
Affects Dockerfiles: false
Affects GraphQL: false
Affects SG: false
Computed Build Steps:
:snail: Trigger async
:docker: :construction: alpine-3.12
:docker: :construction: cadvisor
:docker: :construction: codeinsights-db
:docker: :construction: codeintel-db
:docker: :construction: frontend
:docker: :construction: github-proxy
:docker: :construction: gitserver
:docker: :construction: grafana
:docker: :construction: indexed-searcher
:docker: :construction: jaeger-agent
:docker: :construction: jaeger-all-in-one
:docker: :construction: minio
:docker: :construction: postgres-12.6-alpine
:docker: :construction: postgres_exporter
:docker: :construction: precise-code-intel-worker
:docker: :construction: prometheus
:docker: :construction: query-runner
:docker: :construction: redis-cache
:docker: :construction: redis-store
:docker: :construction: redis_exporter
:docker: :construction: repo-updater
:docker: :construction: search-indexer
:docker: :construction: searcher
:docker: :construction: symbols
:docker: :construction: syntax-highlighter
:docker: :construction: worker
:docker: :construction: server
:lipstick: Prettier
:clipboard: Misc Linters
:puppeteer::electric_plug: Puppeteer tests prep
:puppeteer::electric_plug: Puppeteer tests chunk #1
→ depends on puppeteer:prep
:puppeteer::electric_plug: Puppeteer tests chunk #2
→ depends on puppeteer:prep
:puppeteer::electric_plug: Puppeteer tests chunk #3
→ depends on puppeteer:prep
:puppeteer::electric_plug: Puppeteer tests chunk #4
→ depends on puppeteer:prep
:puppeteer::electric_plug: Puppeteer tests chunk #5
→ depends on puppeteer:prep
:puppeteer::electric_plug: Puppeteer tests chunk #6
→ depends on puppeteer:prep
:puppeteer::electric_plug: Puppeteer tests finalize
→ depends on puppeteer:chunk:1 puppeteer:chunk:2 puppeteer:chunk:3 puppeteer:chunk:4 puppeteer:chunk:5 puppeteer:chunk:6
:chromatic: Upload Storybook to Chromatic
:jest: Test shared client code
:jest: Test wildcard client code
:webpack::globe_with_meridians: Build
:webpack::globe_with_meridians::moneybag: Enterprise build
:jest::globe_with_meridians: Test
:chrome: Puppeteer tests for chrome extension
:jest::chrome: Test browser extension
:jest: Test branded client code
:eslint: Typescript eslint
:stylelint: Stylelint
:go: Test
:go: Build
:lipstick: :graphql:
:docker: Lint
:memo: Check and build docsite
:chains: Backend integration tests
→ depends on server:candidate
:docker::brain: Code Intel QA
→ depends on server:candidate
:chromium: Sourcegraph E2E
→ depends on server:candidate
:docker::chromium: Sourcegraph QA
→ depends on server:candidate
:docker::arrow_double_up: Sourcegraph Upgrade
→ depends on server:candidate
:docker: :truck: alpine-3.12
:docker: :truck: cadvisor
:docker: :truck: codeinsights-db
:docker: :truck: codeintel-db
:docker: :truck: frontend
:docker: :truck: github-proxy
:docker: :truck: gitserver
:docker: :truck: grafana
:docker: :truck: indexed-searcher
:docker: :truck: jaeger-agent
:docker: :truck: jaeger-all-in-one
:docker: :truck: minio
:docker: :truck: postgres-12.6-alpine
:docker: :truck: postgres_exporter
:docker: :truck: precise-code-intel-worker
:docker: :truck: prometheus
:docker: :truck: query-runner
:docker: :truck: redis-cache
:docker: :truck: redis-store
:docker: :truck: redis_exporter
:docker: :truck: repo-updater
:docker: :truck: search-indexer
:docker: :truck: searcher
:docker: :truck: symbols
:docker: :truck: syntax-highlighter
:docker: :truck: worker
:docker: :truck: server