Skip to content

Use native executor git repo support in SSBC

Warren Gifford requested to merge es/native-workspaces into main

Created by: eseliger

This PR turns on repo cloning in executors for batch changes. It clones the repo into the workspace, adds cache files and the input and spawns the execution from that. This requires a change in src-cli and makes the repo fetching process faster, more resilient (in my testing) and enables to run on the largest repos without bigger problems. Also, we have a grafana dashboard for executor cloning performance.

Stats for nerds

Ran on my Mac:

120 workspaces total, some of them are workspaces in subdirectories with onlyFetchWorkspaces on. Total execution time: Before: 03:35 minutes, after: 01:09 minutes. Also, in the 03:35 minute run the mac would start heating up a lot, it stayed mostly cool in the 01:09 minute run.

sgtest/megarepo: Setting up the workspace before: ~8 minutes, now 2 minutes.

Also, see this: https://github.com/sourcegraph/sourcegraph/pull/37299 PR; it's the follow-up for post-3.41 cut to remove the access_token_id column from the database as well.

Test plan

Lots of validation testing. Will throw a giant execution in k8s at it too once it's out.

Merge request reports