SSBC-EM2.3-CACHE1: Implement caching of full changeset specs
Created by: mrnugget
Part of https://github.com/sourcegraph/sourcegraph/issues/26142
-
Extract createChangesetSpecs
fromsrc-cli
intosourcegraph/lib
and use in src-cli -1d
-
Add implementation of ExecutionCache
that writes JSON LogLines on write-events0.5d
-
Make src batch exec
use this cache0.5d
-
Server-side: extract cache-write events from log entries and save to new batch_spec_execution_cache_entries
table with K/V layout2d
-
key
is cache key -
value
is text/jsonb/binary/whatever -
version
is a number that signifies format used in cached value last_used_at
created_at
- https://github.com/sourcegraph/sourcegraph/pull/27125
- https://github.com/sourcegraph/sourcegraph/pull/27212
-
-
Extract implementation of TaskCacheKey
from src-cli tosourcegraph/lib
2d
-
In workspace resolution: build cache key, check against database, set boolean cached_result_found
onbatch_spec_workspace
, if cache hit: build changeset specs, persist, wire up correctly1d
-
In workspace resolver: if cached result found, return CachedResultFound: true
andState: completed
0.5d
-
When creating batch spec workspace execution jobs: skip workspaces with where cached_result_found
boolean is set to true0.5d
-
UI: Show workspace.CachedResultFound
0.5d
-
UI: All workspace.Steps
are shown asskipped
0.5d
-
Ensure that steps.env
forwarding isn't usable in SSBC because it would break caching.0.5d
-
Periodic background job that cleans up old cache entries 1d
- If possible: use table size (max X gigabyte) and evict LRU entries first
- https://github.com/pulls?q=is%3Apr+author%3Amrnugget+archived%3Afalse+user%3Asourcegraph+is%3Aclosed
-
In src batch exec
: disable full-spec-caching in coordinator0.5d
- https://github.com/sourcegraph/src-cli/pull/653
- Remove
-clear-cache
flag https://github.com/sourcegraph/sourcegraph/pull/27338
-
Ensure that recreated-from-cache changeset specs are also cleaned up in replaceBatchSpecInput
0.5d