ci: add shellcheck linter for shell scripts
Created by: ggilmore
Overview
This PR adds the popular https://github.com/koalaman/shellcheck to our CI pipeline. Shellcheck helps prevent a lot of shell footguns and pitfalls and will make our scripts more-robust overall.
I fixed all of these scripts by hand following shellcheck's suggestions. I'm reasonably confident that the critical paths work (CI pipelines + docker builds pass, dev/start.sh). There might be some lesser-used scripts that are slightly broken by these changes, but I'm confident that those be fixed up after they're discovered by follow up PRs.
Riskier edits
I tagged people here so that they can sign off on the more extensive / riskier script fixes that I had to make.
I'd like for these to signed off on before merging.
-
@sourcegraph/core-services https://github.com/sourcegraph/sourcegraph/blob/shellcheck/dev/ci/ci-db-backcompat.sh (Note: https://github.com/sourcegraph/sourcegraph/issues/9921 will be fixed in a separate PR) -
@sourcegraph/core-services https://github.com/sourcegraph/sourcegraph/blob/shellcheck/dev/ci/db-backcompat.sh (Note: https://github.com/sourcegraph/sourcegraph/issues/9921 will be fixed in a separate PR) -
@sourcegraph/core-services https://github.com/sourcegraph/sourcegraph/blob/shellcheck/dev/go-install.sh -
@sourcegraph/core-services https://github.com/sourcegraph/sourcegraph/blob/shellcheck/dev/handle-change.sh -
@efritz https://github.com/sourcegraph/sourcegraph/blob/shellcheck/dev/squash_migrations.sh -
@sourcegraph/distribution @ryan-blunden https://github.com/sourcegraph/sourcegraph/blob/shellcheck/dev/src-expose/entry.sh (Note: user-facing script) -
@sourcegraph/distribution @uwedeportivo https://github.com/sourcegraph/sourcegraph/blob/shellcheck/docker-images/prometheus/entry.sh (Note: user-facing script / https://github.com/koalaman/shellcheck/wiki/SC2039)
Scripts sorted by code owner
This PR touches a lot of scripts. This section conveniently lists all of the edited files group by their code owner so that each team can take a look at the scripts that they're supposed to maintain.
Note: I'm not asking each team to review every one of the files here before I merge this PR (the high-risk files I flagged are listed above). Most of the changes here are fairly straightforward. This section is just an easy way to navigate the PR. I'm confident that issues that slip through here can be easily fixed in a follow up PR.
@sourcegraph/web
@sourcegraph/nobody
- dev/Procfile @sourcegraph/nobody
- dev/add_migration.sh @sourcegraph/nobody
- dev/api/delete_user.sh @sourcegraph/nobody
- dev/api/delete_user_mutation.json @sourcegraph/nobody
- dev/api/user_id_query.json @sourcegraph/nobody
- dev/auth-provider/keycloak.sh @sourcegraph/nobody
- dev/auth-provider/scripts/common.sh @sourcegraph/nobody
- dev/auth-provider/scripts/configure-keycloak.sh @sourcegraph/nobody
- dev/caddy.sh @sourcegraph/nobody
- dev/changewatch.sh @sourcegraph/nobody
- dev/check/all.sh @sourcegraph/nobody
- dev/check/bash-syntax.sh @sourcegraph/nobody
- dev/check/broken-urls.bash @sourcegraph/nobody
- dev/check/check-owners.sh @sourcegraph/nobody
- dev/check/docsite.sh @sourcegraph/nobody
- dev/check/go-dbconn-import.sh @sourcegraph/nobody
- dev/check/go-enterprise-import.sh @sourcegraph/nobody
- dev/check/go-lint.sh @sourcegraph/nobody
- dev/check/gofmt.sh @sourcegraph/nobody
- dev/check/licenses.sh @sourcegraph/nobody
- dev/check/no-localhost-guard.sh @sourcegraph/nobody
- dev/check/shellcheck.sh @sourcegraph/nobody
- dev/check/yarn-deduplicate.sh @sourcegraph/nobody
- dev/ci/ci-db-backcompat.sh @sourcegraph/nobody
- dev/ci/db-backcompat.sh @sourcegraph/nobody
- dev/ci/e2e.sh @sourcegraph/nobody
- dev/ci/parallel_run.sh @sourcegraph/nobody
- dev/ci/reset-test-db.sh @sourcegraph/nobody
- dev/ci/yarn-build.sh @sourcegraph/nobody
- dev/ci/yarn-run.sh @sourcegraph/nobody
- dev/ci/yarn-test-separate.sh @sourcegraph/nobody
- dev/ci/yarn-test.sh @sourcegraph/nobody
- dev/comby-install-or-upgrade.sh @sourcegraph/nobody
- dev/dev-sourcegraph-server.sh @sourcegraph/nobody
- dev/foreach-ts-project.sh @sourcegraph/nobody
- dev/generate.sh @sourcegraph/nobody
- dev/git-diff-no-enterprise.sh @sourcegraph/nobody
- dev/go-install.sh @sourcegraph/nobody
- dev/handle-change.sh @sourcegraph/nobody
- dev/jaeger.sh @sourcegraph/nobody
- dev/owners @sourcegraph/nobody
- dev/owners.sh @sourcegraph/nobody
- dev/phabricator/e2e.sh @sourcegraph/nobody
- dev/phabricator/install-sourcegraph.sh @sourcegraph/nobody
- dev/phabricator/restart.sh @sourcegraph/nobody
- dev/phabricator/shared.sh @sourcegraph/nobody
- dev/phabricator/start.sh @sourcegraph/nobody
- dev/postgres_exporter.sh @sourcegraph/nobody
- dev/prune-pick.sh @sourcegraph/nobody
- dev/run-server-image.sh @sourcegraph/nobody
- dev/start.sh @sourcegraph/nobody
- dev/syntect_server @sourcegraph/nobody
- dev/syntect_server.sh @sourcegraph/nobody
- enterprise/dev/dev-sourcegraph-server.sh @sourcegraph/nobody
- enterprise/dev/generate.sh @sourcegraph/nobody
- enterprise/dev/start.sh @sourcegraph/nobody
@sourcegraph/distribution
- .buildkite/hooks/pre-command @sourcegraph/distribution @ggilmore
- .tool-versions @sourcegraph/distribution
- cmd/frontend/build.sh @sourcegraph/distribution
- cmd/github-proxy/build.sh @sourcegraph/distribution
- cmd/gitserver/build.sh @sourcegraph/distribution
- cmd/loadtest/build.sh @sourcegraph/distribution
- cmd/query-runner/build.sh @sourcegraph/distribution
- cmd/replacer/build.sh @sourcegraph/distribution
- cmd/repo-updater/build.sh @sourcegraph/distribution
- cmd/searcher/build.sh @sourcegraph/distribution
- cmd/server/build.sh @sourcegraph/distribution
- cmd/server/jaeger.sh @sourcegraph/distribution
- cmd/server/pre-build.sh @sourcegraph/distribution
- dev/check/build.sh @sourcegraph/distribution
- dev/grafana.sh @sourcegraph/distribution
- dev/libsqlite3-pcre/build.sh @sourcegraph/distribution
- dev/prometheus.sh @sourcegraph/distribution
- dev/src-expose/build.sh @sourcegraph/distribution
- docker-images/alpine/build.sh @sourcegraph/distribution
- docker-images/alpine/release.sh @sourcegraph/distribution
- docker-images/grafana/build.sh @sourcegraph/distribution
- docker-images/indexed-searcher/build.sh @sourcegraph/distribution
- docker-images/jaeger-agent/build.sh @sourcegraph/distribution
- docker-images/jaeger-all-in-one/build.sh @sourcegraph/distribution
- docker-images/postgres-11.4/build.sh @sourcegraph/distribution
- docker-images/prometheus/build.sh @sourcegraph/distribution
- docker-images/prometheus/entry.sh @sourcegraph/distribution
- docker-images/redis-cache/build.sh @sourcegraph/distribution
- docker-images/redis-store/build.sh @sourcegraph/distribution
- docker-images/search-indexer/build.sh @sourcegraph/distribution
- docker-images/syntax-highlighter/build.sh @sourcegraph/distribution
- enterprise/cmd/frontend/build.sh @sourcegraph/distribution
- enterprise/cmd/repo-updater/build.sh @sourcegraph/distribution
- enterprise/cmd/server/build.sh @sourcegraph/distribution
@sourcegraph/core-services
- enterprise/dev/check/build.sh @sourcegraph/distribution @sourcegraph/core-services
- cmd/symbols/go-build.sh @sourcegraph/core-services
@slimsag
- cmd/frontend/pre-build.sh @slimsag
- enterprise/cmd/frontend/pre-build.sh @slimsag
- enterprise/cmd/server/pre-build.sh @slimsag
@keegancsmith
- dev/src-expose/docker-publish.sh @keegancsmith
- dev/src-expose/entry.sh @keegancsmith
@felixfbecker
- .vscode/extensions.json @felixfbecker
@efritz
- dev/squash_migrations.sh @efritz