Skip to content

ci: parallelize docker build scripts

Warren Gifford requested to merge add-parallel into master

Created by: ggilmore

Some parts of our docker build scripts can be sped up by running some independent steps in parallel:

  • server: the frontend and management console pre-build scripts can be run in parallel
  • frontend: the phabricator extension and web assets can be built in parallel
  • server: all of the go binaries can be built in parallel

By using GNU Parallel to concurrently run these steps, we can shave ~2-3 minutes off of the server's docker build step.

Notes:

  • This introduces a new dev dependency on GNU Parallel (already added to our buildkite agents)
  • Buildkite's timestamp UI for certain sections can no longer give you detailed timing information since things are no being run in parallel. However, part of the output of parallel_run (utility function that I introduced) can display this same information. Example:
build_frontend_enterprise	Seq	Host	Starttime	JobRuntime	Send	Receive	Exitval	Signal	Command
build_frontend_enterprise	1	:	1574719866.609	   117.459	0	1627	0	0	build_browser
build_frontend_enterprise	2	:	1574719866.612	   232.769	0	1475	0	0	build_web

Merge request reports