test(executor): run the executor tests against an unreleased valgrind

[lvaroqui]

Run the Docker executor tests against an unreleased valgrind-codspeed:

CODSPEED_VALGRIND_REF=<branch | tag | full commit sha> just test-integ [filter]

The tests install the pinned valgrind-codspeed release through codspeed setup, so a runner change that depends on an unreleased valgrind-codspeed change could not be tested end to end until that release shipped and the pin was bumped.

• tests/docker/run.sh resolves the ref to a commit on the host (git ls-remote; a full sha is used as-is) and hashes it into the setup image tag, so a push to the branch rebuilds the image and an unchanged ref reuses it.
• tests/docker/setup.sh builds and installs that commit in the setup container, adds libc6-dbg (without it codspeed setup reinstalls the package), then runs codspeed setup, which keeps an installed build at or above the pinned version. It fails when setup replaced the build, i.e. the ref is older than the pin. Test containers are unchanged.
• The executor-tests CI job gets a commented-out env: example, to enable while a PR waits on a valgrind-codspeed release.

I first considered teaching codspeed setup itself to build a ref, but only the tests need it, and since every codspeed run runs setup it would have needed per-run commit tracking to avoid rebuilding.

Short shas are not supported: GitHub only serves fetches by full sha.

Verified locally with CODSPEED_VALGRIND_REF=700eb1edb4393e1e964acb4a04c1cc6a7f499911: the setup container built and kept valgrind-3.26.0.codspeed7 from that commit, and simulation_exec_harness_declares_benchmark_pid passed against it. Branch and annotated tag names resolve to their commits, and an unknown name fails before any build.

Closes COD-3781

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds optional testing against unreleased valgrind builds.

The PR appears safe to merge; no new actionable issues remain.

What we checked:

• Moving branches use fresh builds: run.sh resolves the branch on each run and includes its commit in the setup image tag.
• Older builds stop the run: The source build now installs under /usr. Setup replaces an older build there, and the version check stops the run.

Summary

Lets Docker executor tests use an unreleased valgrind-codspeed commit through CODSPEED_VALGRIND_REF.

• Resolves branches and tags before builds and includes the selected commit in the setup image tag.
• Builds Valgrind under /usr, installs libc debug symbols, and checks that codspeed setup kept the build.
• Adds clean-integ and documents the override.
• Both previous unnumbered findings are addressed: older builds no longer hide under /usr/local, and bad branch or tag names fail before builds.

_{Reviews (2) · Last reviewed commit: "test(executor): run the executor tests a..." · Reviewed by Greptile}

[codspeed]

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 31 untouched benchmarks
⏩ 6 skipped benchmarks¹

---

_{Comparing cod-3781-run-the-executor-tests-against-an-unreleased-valgrind (8470cc5) with main (8c603c2)}

Footnotes

1. 6 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[GuillaumeLagrange]

olgtm, maybe we can add a just clean-integ that would be in charge of clearing all docker volumes, images and so on easily? Dont go too complicated on this