codeintel: Store index job commands and output in the database
Created by: efritz
This change is to help me diagnose issues with the auto-indexer without having to hand-correlate logs by time range in stackdriver and the records in the DB.
A logfile that consists of each command run on the indexer host as well as the stdout and stderr stream content is written to the index record after each attempt. This will let me see relevant logs for failed jobs, and will eventually let us empower users with better transparency on why their indexer configuration may not be correct. (I would refuse to use this product if all I got back was exit status 1, which is all we get back today.)
id | 637
commit | feeb9a050b31b40eec6f2470e7599eeeadfe5bdd
queued_at | 2020-10-06 19:25:48.724828-05
state | errored
failure_message | failed to perform docker step: exit status 2
started_at | 2020-10-06 19:25:49.844921-05
finished_at | 2020-10-06 19:25:50.830732-05
repository_id | 154
process_after | [NULL]
num_resets | 0
num_failures | 1
docker_steps | {"{\"root\": \"\", \"image\": \"sourcegraph/lsif-go:latest\", \"commands\": [\"go\", \"mod\", \"downsload\"]}"}
root |
indexer | sourcegraph/lsif-go:latest
indexer_args | {lsif-go,--no-animation}
outfile |
log_contents | git -C /Users/efritz/.sourcegraph/indexer-temp/234402727 init +
| stdout: Initialized empty Git repository in /Users/efritz/.sourcegraph/indexer-temp/234402727/.git/ +
| +
| git -C /Users/efritz/.sourcegraph/indexer-temp/234402727 -c protocol.version=2 fetch http://indexer:******@localhost:3080/.internal-code-intel/git/github.com/sourcegraph-testing/zap feeb9a050b31b40eec6f2470e7599eeeadfe5bdd+
| stderr: From http://localhost:3080/.internal-code-intel/git/github.com/sourcegraph-testing/zap +
| stderr: * branch feeb9a050b31b40eec6f2470e7599eeeadfe5bdd -> FETCH_HEAD +
| +
| git -C /Users/efritz/.sourcegraph/indexer-temp/234402727 checkout feeb9a050b31b40eec6f2470e7599eeeadfe5bdd +
| stderr: Note: switching to 'feeb9a050b31b40eec6f2470e7599eeeadfe5bdd'. +
| stderr: +
| stderr: You are in 'detached HEAD' state. You can look around, make experimental +
| stderr: changes and commit them, and you can discard any commits you make in this +
| stderr: state without impacting any branches by switching back to a branch. +
| stderr: +
| stderr: If you want to create a new branch to retain commits you create, you may +
| stderr: do so (now or later) by using -c with the switch command. Example: +
| stderr: +
| stderr: git switch -c <new-branch-name> +
| stderr: +
| stderr: Or undo this operation with: +
| stderr: +
| stderr: git switch - +
| stderr: +
| stderr: Turn off this advice by setting config variable advice.detachedHead to false +
| stderr: +
| stderr: HEAD is now at feeb9a050b31 Prepare release v1.15.0 (#819) +
| +
| docker run --rm --cpus 4 --memory 12G -v /Users/efritz/.sourcegraph/indexer-temp/234402727:/data -w /data sourcegraph/lsif-go:latest go mod downsload +
| stderr: go mod downsload: unknown command +
| stderr: Run 'go help mod' for usage. +
| +
|