Skip to content

feat(cache): resolve remote cache configuration - #727

Merged
wan9chi merged 14 commits into
mainfrom
remote-cache-config
Sep 25, 2026
Merged

wan9chi merged 14 commits into
mainfrom
remote-cache-config

Conversation

@wan9chi

@wan9chi wan9chi commented Sep 17, 2026 •

Copy link
Copy Markdown
Member

Motivation

Each planned execution needs a resolved remote cache endpoint and access mode, so execution can read from or upload to the remote cache without working out the settings itself. Resolving them during planning also makes them visible in plan snapshots.

Summary

  • The workspace root's cache: { remote: { url } } sets the endpoint. A task opts out with cache: { remote: false }.
  • --remote-cache=off|read|read-write and VP_REMOTE_CACHE select access, and the flag overrides the env. VP_REMOTE_CACHE_URL overrides the configured endpoint. Without either, access is read when an endpoint is configured and off otherwise. Empty values count as unset.
  • Each vp run level resolves its own settings from the envs it sees. A level's --remote-cache reaches nested vp run through VP_REMOTE_CACHE, and a prefix such as VP_REMOTE_CACHE=off vp run build applies only to that command.
  • Disabling local caching also disables remote access.
  • The two env vars reach task processes and follow env and untrackedEnv like other variables. By default they pass through untracked with other VP_* variables, so runs with different remote access share cache entries.
  • The endpoint is carried as a string and validated when remote caching uses it.

@wan9chi wan9chi changed the title remote cache config feat(cache): resolve remote cache configuration Sep 17, 2026
@wan9chi
wan9chi force-pushed the remote-cache-config branch from 7855149 to e96231c Compare September 17, 2026 05:57
@github-actions

github-actions Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.22%  [ -6.53% ..  +6.47%]  overhead  +291.41%
dynamic/access             change  -0.31%  [ -2.87% ..  +2.58%]  overhead   +14.31%
dynamic/access-relative    change  +0.20%  [ -5.93% ..  +4.54%]  overhead   +60.83%
dynamic/access-contended   change  -2.46%  [-12.38% ..  +4.67%]  overhead   +15.63%
static/launch              change  -0.08%  [ -7.73% ..  +7.06%]  overhead  +727.11%
static/access              change  -0.07%  [ -9.02% .. +14.26%]  overhead  +813.43%
static/access-relative     change  -0.75%  [ -3.83% ..  +2.67%]  overhead +1175.75%
static/access-contended    change  +0.45%  [ -4.46% ..  +6.20%]  overhead +1994.32%

macos

dynamic/launch             change  -0.11%  [ -3.98% ..  +3.93%]  overhead  +218.65%
dynamic/access             change  +0.67%  [ -3.11% ..  +6.00%]  overhead    +2.78%
dynamic/access-relative    change  -0.68%  [ -4.57% ..  +3.02%]  overhead  +278.69%
dynamic/access-contended   change  +1.49%  [ -4.58% ..  +8.88%]  overhead    +2.19%

windows

dynamic/launch             change  +1.93%  [-15.21% .. +20.57%]  overhead   +26.50%
dynamic/access             change  +2.68%  [ -7.61% .. +27.21%]  overhead    +6.29%
dynamic/access-relative    change  +0.17%  [-21.38% .. +18.05%]  overhead    +2.95%
dynamic/access-contended   change  -2.25%  [-19.17% ..  +8.66%]  overhead    -0.09%

@wan9chi
wan9chi force-pushed the remote-cache-config branch from e96231c to 67e02e8 Compare September 21, 2026 10:45
@wan9chi
wan9chi changed the base branch from main to remote-cache-plan September 21, 2026 10:45
@wan9chi
wan9chi added this pull request to stack #741 September 21, 2026 10:45
Base automatically changed from remote-cache-plan to main September 21, 2026 10:51
@wan9chi
wan9chi force-pushed the remote-cache-config branch 3 times, most recently from 65b48e0 to e455c62 Compare September 22, 2026 14:03
@wan9chi
wan9chi removed this pull request from stack #741 September 22, 2026 14:05
@wan9chi
wan9chi changed the base branch from main to remote-cache-model September 22, 2026 14:05
@wan9chi
wan9chi added this pull request to stack #745 September 22, 2026 14:05
@wan9chi
wan9chi force-pushed the remote-cache-config branch from e455c62 to a631878 Compare September 22, 2026 14:10
@wan9chi
wan9chi force-pushed the remote-cache-config branch from a631878 to 96e4492 Compare September 22, 2026 14:19
Base automatically changed from remote-cache-model to main September 22, 2026 14:24
@wan9chi
wan9chi force-pushed the remote-cache-config branch 5 times, most recently from 21cd645 to adbd1b9 Compare September 22, 2026 15:29
@wan9chi
wan9chi removed this pull request from stack #745 September 22, 2026 15:31
@wan9chi
wan9chi changed the base branch from main to fix/windows-env-name-comparison September 22, 2026 15:38
@wan9chi
wan9chi added this pull request to stack #748 September 22, 2026 15:38
@wan9chi
wan9chi force-pushed the remote-cache-config branch from adbd1b9 to 4256d4a Compare September 22, 2026 15:40
@wan9chi
wan9chi force-pushed the remote-cache-config branch from 4256d4a to a30cc6e Compare September 23, 2026 06:56
@wan9chi
wan9chi removed this pull request from stack #748 September 23, 2026 06:58
@wan9chi
wan9chi changed the base branch from fix/windows-env-name-comparison to main September 23, 2026 06:58
@wan9chi
wan9chi force-pushed the remote-cache-config branch from a30cc6e to c408288 Compare September 23, 2026 11:52
@wan9chi
wan9chi changed the base branch from main to feat/group-task-cache-fields September 23, 2026 11:52
@wan9chi
wan9chi added this pull request to stack #751 September 23, 2026 11:52
Base automatically changed from feat/group-task-cache-fields to main September 24, 2026 02:11
@wan9chi
wan9chi force-pushed the remote-cache-config branch 3 times, most recently from 03e11af to 9218032 Compare September 24, 2026 09:11
wan9chi and others added 13 commits September 25, 2026 09:55
Co-authored-by: GPT-6 <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: GPT-6 <codex@openai.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Carry the endpoint from `remoteCache.url` or `VP_REMOTE_CACHE_URL` as a
string, so an endpoint can't fail planning before remote caching uses it.
Treat empty control values as unset, as a CI secret unavailable to the job
expands to an empty string.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`VP_REMOTE_CACHE` and `VP_REMOTE_CACHE_URL` reach task processes like other
`VP_*` variables, including the mode set by `--remote-cache`, so `vp`
processes started by a task inherit the invocation's remote cache settings.
They stay out of cache fingerprints, including envs matched by `cache.env`,
command prefix envs, and runner-aware env queries, so runs with different
remote access share cache entries.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Configure the workspace endpoint as `cache: { remote: { url } }` in the
workspace root config, next to the task-level `cache: { remote }` setting.
`cache` is already limited to the workspace root, so the separate
`remoteCache` check is no longer needed.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`VP_REMOTE_CACHE` and `VP_REMOTE_CACHE_URL` now match env names by the
platform's rules, so on Windows a differently spelled control selects the
remote cache and stays out of fingerprints.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Resolve the `cache.remote.url` endpoint into `ResolvedGlobalCacheConfig`
with the other workspace cache settings, instead of keeping it in a
separate field on the task graph. `--cache` and `--no-cache` keep the
configured endpoint.

Store endpoints as `Arc<str>`: they are usually too long for an inline
`Str`, and each cacheable execution carries a copy.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- Report invalid remote cache envs as plan errors, like
  `VP_RUN_CONCURRENCY_LIMIT`, instead of through a separate error type.
- Keep the invocation's remote cache config in `PlanContext` next to the
  resolved global cache config, instead of passing it through task planning.
- Serve runner-aware env queries the same envs as before remote caching,
  without copying the env map for each spawn to hide the controls.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Leave each package's loaded config intact.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…evels

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
`VP_REMOTE_CACHE` and `VP_REMOTE_CACHE_URL` follow the task's `env` and
`untrackedEnv` settings, and command prefixes, like any other variable. By
default they pass through untracked with other `VP_*` variables, so runs
with different remote access still share cache entries.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
- `PlanOptions.remote_cache_mode`: the `--remote-cache` flag.
- `ResolvedRemoteCacheConfig` with `access`: resolved for a `vp run` level.
- `PlanContext::resolved_remote_cache`: the level's resolved value.
- `CacheConfig.remote_cache_allowed`: the task's `cache.remote`.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the remote-cache-config branch from 02bab8e to c2aa084 Compare September 25, 2026 01:56
@wan9chi
wan9chi marked this pull request as ready for review September 25, 2026 01:56
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi merged commit db55b64 into main Sep 25, 2026
19 checks passed
@wan9chi
wan9chi deleted the remote-cache-config branch September 25, 2026 02:13
wan9chi added a commit that referenced this pull request Sep 27, 2026
## Motivation

#727 resolves a remote cache endpoint and access mode for each
execution, but nothing uses them yet. This is the first client PR for
the server API in #713: in `read-write` mode, `vp run` uploads each
successful, cacheable new execution after recording it locally, so the
following PRs can restore it elsewhere.

The new `vt_remote_cache` crate implements `POST {endpoint}/store` over
opaque bytes. It appends `/store` to the endpoint's path, keeping any
namespace path, and sends a CBOR `metadata` part with the key, secondary
key, and value, plus the output archive streamed as the `blob` part.
Only HTTP 200 counts as success, and the response isn't decoded. The
client uses reqwest with rustls and the ring provider, verifies HTTPS
certificates with the operating system's verifier, and sets 10-second
connect and 60-second read timeouts.

`ExecutionCache` creates one client per endpoint on first use, and
uploads after a successful local update, awaiting the upload inline.
Hits never upload. The key is a header plus wincode(`CacheEntryKey`),
the secondary key is the header plus wincode(`ExecutionCacheKey`), and
the value is wincode(`CacheEntryValue`). The header contains the cache
schema version and the target OS and architecture.

An invalid endpoint or a failed upload never fails the task or changes
the exit status. The run summary shows a warning instead, even when a
single task ran, such as `vp run: build not uploaded to the remote
cache: network error.` The reason names only the kind of failure, such
as an invalid endpoint, a network error, or an HTTP status, so it's the
same on every platform. `--last-details` adds the underlying details for
each task, such as the connection error behind a network error, the
parse error for an endpoint that isn't a URL, or the message in an error
response's body.

After the wrapped command exits, `remote-cache-server` prints one line
for each request it served, such as `[remote-cache] POST /store 200` or
`[remote-cache] POST /fetch 200 not_found`. The `remote_cache_backend`
snapshots gain these lines. A new ignored `remote_cache` fixture shows
that a `read-write` run sends one store request, a rerun is a local hit
with no requests, and a `read` run after an input change reruns the task
without requests. Cases in the default suite cover an invalid endpoint
and port 0, where nothing can listen, on every platform.

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant