Repository navigation
Test PyPI hosted mode under rules_python on real Bazel - #2110
Mikola Lysenko (mikolalysenko) wants to merge 7 commits into
Conversation
The real-Bazel suite held one flow, hosted mode for a Go module. This adds the same flow for a PyPI package that comes through rules_python's `pip.parse` under Bzlmod, from a root `requirements.txt` in pip's hash-checking mode, on Bazel 9.3.2, 8.8.2 and 7.7.2 (rules_python 2.5.0-rc4 on all three). No file under crates/*/src changes. `pypi::hosted_pip_parse`, in a fresh output base: - builds the fixture workspace and reads "OUT: PRISTINE" from `bazel run //:consumer`; - runs `scan --mode hosted --yes`, which rewrites requirements.txt and nothing else: the pin of the package becomes a direct reference to the patched wheel with its sha256, and the pin of its dependency stays as it was; - runs the same two Bazel commands and reads the patched marker, which holds an id made for the run; the wheel came from the stand-in patch host. Where Bazel's lock records what the hub learned from the index (from Bazel 8.5 on, so 9.3.2 and 8.8.2), that build drops the entry of the package from MODULE.bazel.lock and changes nothing else; on 7.7.2 it changes no file. The release decides which of the two the test expects; - runs `rollback --yes`: requirements.txt is the one of before at once. The lock keeps Bazel's change until the next build, which prints "OUT: PRISTINE" again and writes the lock of before; from then on every file of the checkout is the one of before, byte for byte; - on a second machine (a new output base), takes a patch whose host serves other bytes than the grant pins: after the same scan the build fails on the sha256. No step builds with `--lockfile_mode=error`: where the patched build has to rewrite the lock, a build that may not rewrite it cannot pass. The Socket API, the patch host, the two project pages of the index, the two upstream wheels and the PyPI JSON document `rollback` reads are one loopback wiremock server. Bazel's downloader takes a plain-http URL only with a checksum, and an index page has none, so the hub keeps its default index, and a downloader config that a `.bazelrc` written by the test names sends the URLs of the two packages, and no other, to the stand-in. The second machine shares the repository cache, which keeps every download that has a checksum, so it is given a patch of its own, whose wheel no run has fetched. The same `.bazelrc` selects rules_python's script bootstrap, so `bazel run` starts the binary with the interpreter of the toolchain and the build and the run need no Python from the machine. The default bootstrap is a Python program behind `/usr/bin/env python3`. The ruleset archives, the Python toolchain and the index pages of the packages the rulesets pin for their own tools come from the network (github.com, pypi.org). No cache holds an index page: a run sends 31 requests to pypi.org on 9.3.2, 65 on 8.8.2 and 205 on 7.7.2, whose lock records nothing of the index. - crates/socket-patch-cli/tests/e2e_bazel_build/pypi.rs: the wheels (built in the test, stored and under one fixed time, so the fixture pins their sha256 and no binary is committed), the stand-in and the flow. A hermetic test holds the fixture's requirements.txt to the wheels the test serves. - crates/socket-patch-cli/tests/fixtures/bazel/rules_python/bzlmod: the workspace, as `common` and one era, `e9`. - .github/workflows/bazel-compatibility.yml: for `pypi::`, one Linux leg per Bazel release and one macOS leg (nightly and on dispatch), and the PyPI source files in the path filters. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…i-rules-python The harness branch made .github/workflows/bazel-compatibility.yml a reusable workflow with one caller per ecosystem, and moved the stand-in server, the downloader config and the steps of a flow into harness.rs. The one conflict is in bazel-compatibility.yml, and it takes the harness branch's file whole: the PyPI legs this branch had added to its matrix and its path lists go, and come back in a caller of their own in the next commit, which also ports pypi.rs to the shared harness. Until then pypi.rs does not build against harness.rs. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
pypi.rs is written against harness.rs as the merge brought it. The
stand-in server, the downloader config, the release as three numbers
and steps (a), (b) and (c) of a hosted flow are the harness's, and the
module's own copies go. The flow keeps its own undo step: the harness's
expects every file back before the next build, and here that build
brings back Bazel's lock.
What the flow asserts that it did not:
- The upstream wheel reaches the build through the stand-in in every
run. The wheels of the index are built with an id of the run, so no
repository cache holds them, and step (a) asserts the fetch. Before,
the wheel had fixed bytes and sat in the cache after the first run,
so the check that the patched build "did not fetch the upstream
wheel" could not fail. That check is gone.
- The patched build does not ask the index about the package, and the
build after `rollback` asks it again. Both are counted at the
stand-in.
- After the patched build has rewritten Bazel's lock, the next build
changes no file and a `--lockfile_mode=error` build passes.
Not asserted, and measured once with a temporary edit: between the scan
and that build, a `--lockfile_mode=error` build fails on Bazel 9.3.2
and 8.8.2 ("the extension ... pip has changed its facts") and passes on
7.7.2. The harness runs its lock-frozen build only after an updating
one.
No leg asks pypi.org any more. rules_python resolves the hubs of every
module of the build, its own hub of publishing tools among them, and an
index page comes from no cache: the previous commit counted 31 requests
to pypi.org in a run on Bazel 9.3.2, 65 on 8.8.2 and 205 on 7.7.2. The
downloader config now answers the page of every other project from the
stand-in, with a page that lists no file, and blocks the host: a build
that would need the real index fails instead of asking it. A hub that
finds no file for a pin leaves the package to pip, and nothing starts
that pip, because the build depends on no such package. On an empty
repository cache a run downloads from bcr.bazel.build and github.com
only.
The fixture no longer commits requirements.txt. Its sha256 pins tied a
hermetic test to the exact bytes the `zip` crate writes. The test now
writes the file for the wheels of its run, and builds those wheels with
the builder of the in-process Python suites
(tests/common/patched_wheel.rs) in place of a second one here. The
script-bootstrap line moves from a `.bazelrc` the test wrote into the
fixture.
.github/workflows/bazel-pypi.yml calls the reusable
bazel-compatibility.yml for `pypi::`, as bazel-go.yml does for Go. The
merge took this branch's legs and path patterns out of
bazel-compatibility.yml. The pull request list of the new file names
the product files the flow runs through one by one, read from a run of
the flow against a coverage build of the binary. The four patterns it
replaces (`*pypi*`, `*python*`, `*python*/**`, `*requirements*`) also
matched the benchmark fixtures, the rest of vendored mode and the files
of the lock-file package managers. A hermetic test holds every path the
workflow names without a pattern to a file that exists.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The pattern matched two files, a test-only oracle and a macOS-only module, neither of which is in the binary the Linux legs run, so a change to them started three legs for nothing. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…more PyPI files The PyPI flow on real Bazel ran its `--lockfile_mode=error` build only after a build that may update MODULE.bazel.lock had repaired it. From Bazel 8.5 on that lock records what the pip hub learned from the index, so `scan --mode hosted` and `rollback`, which rewrite requirements.txt and leave Bazel's lock alone, both leave it stale: a CI that builds with a frozen lock fails until the lock is updated and committed. The flow now runs that build right after the scan and right after the rollback, before any build that may update the lock, and holds each release to its case: Bazel must refuse it over the hub's changed facts on 9.3.2 and 8.8.2, and it must pass on 7.7.2, whose lock keeps nothing of the index. The module doc and the evidence line say so, and the failure message names what a user has to do. Comments in the same file: - The era table says that the fixture ran on Bazel 9.3.2, 8.8.2 and 7.7.2 and on no other release of its range, and the 8.5.0 boundary names its source: the facts of a module extension, which the lock keeps from that release on (bazelbuild/bazel#27296). - Step (d) says at the step why it is written out and not `Cell::undone`, in place of the sentence in the module doc. - `grant` names the other builder of the same granted reference, `patched_wheel::mount_grant`. bazel-pypi.yml: - Both path lists gain three PyPI helpers that listed files call: utils/python_lock.rs (the finder of a project's Python lock files, which the crawler and the inventory ask), vendor/pypi_wheel.rs (the spelling of a version in a wheel's file name) and hosted/wheel_record.rs (the hosted scan's check of an installed wheel). The files of other project managers that the same files call (Pipenv's, Poetry's, Hatch's) stay out, as before: there the flow does no more than find that the project is not theirs. - The nightly run gets a concurrency group of its own, as in bazel-go.yml: in the group of main's pushes, a second push would replace a nightly that waits behind the first, and the nightly is the only run with the macOS leg. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
PR SummaryLow Risk Overview A new Fixtures under
Reviewed by Cursor Bugbot for commit 3f44eba. Configure here. |
|
[agent] PR shepherd: review brief for What it does: It runs a hosted Risk: low. It adds only tests and an optional, non-required workflow. Watch one thing: Look here
Verified: merged onto its base in a scratch tree, the non-ignored tests pass and clippy is clean. The selection picks Changes I made: none. Open questions (minor)
Stacking and merge order: this PR's base is Shared notes (#2109–#2111): none of these PRs touches Generated by Claude Code |
No user-visible change.
What this is
Stacked on #2107, which holds the real-Bazel test harness and the Go flow. This pull request adds the PyPI flow: hosted mode for a package that rules_python's
pip.parseresolves from a hashedrequirements.txtunder Bzlmod, built by a real Bazel. Its base is #2107's branch, and it moves onto main when #2107 has landed.Tests only. No file under
crates/*/srcchanges. #2107'sharness.rs,go.rs,bazel-compatibility.ymlandbazel-go.ymlare byte-identical to that pull request's head. The binary under test is the one cargo builds from this branch, whose product code is main's at c09cb0e.Contents
Against the harness branch (f1aa02b): 7 files changed, 771 insertions, no deletions. No implementation line, 730 test and workflow lines, 41 fixture lines, no binary file.
crates/socket-patch-cli/tests/e2e_bazel_build/pypi.rspypimodule: the real-Bazel testpypi::hosted_pip_parse(#[ignore]d) and two tests that need no Bazel.crates/socket-patch-cli/tests/e2e_bazel_build/main.rsmod pypi;.github/workflows/bazel-pypi.ymlbazel-compatibility.yml.tests/fixtures/bazel/rules_python/bzlmod/e9/MODULE.bazelpip.parse(requirements_lock = "//:requirements.txt"), Python 3.11 from the ruleset's toolchain..../common/BUILD.bazelpy_binarythat depends on the package..../common/consumer.pyOUT: ...line from the package..../common/.bazelrcbazel runneeds no Python from the machine.The fixture commits no
requirements.txtand no wheel. The test builds the wheels for each run withtests/common/patched_wheel.rs(unchanged) and writes therequirements.txtthat pins them. Every wheel holds an id of the run, so no cache of an earlier run can supply it.One loopback server plays the Socket API, the patch host, the index (
pypi.org) and its file host (files.pythonhosted.org). A Bazel downloader config sends Bazel there, the way a mirror is configured, and blockspypi.org.What it proves, and where
pypi::hosted_pip_parse, in a fresh output base, on the Bazel it is given. Each line is asserted.bazel build //:consumerandbazel run //:consumerprintOUT: PRISTINE. The pip hub asked the index about the package, and Bazel fetched this run's upstream wheel from it.MODULE.bazel.lockholds an index entry for the package on Bazel 8.5 and later and none before; a release on the wrong side fails the test.socket-patch scan --mode hosted --yes. It applies the one patch and rewritesrequirements.txtand no other file. The file is compared whole: the dependency's pin as it was, andspvictim @ <patch URL> --hash=sha256:<patched wheel>. The scan does not download the patched wheel.bazel build --lockfile_mode=error //:consumerfails on Bazel 9.3.2 and 8.8.2 ("the extension ... pip has changed its facts") and passes on 7.7.2. A failure for any other reason does not satisfy the check.OUT: PATCHED <id of the run>. The wheel came from the patch host, and the hub did not ask the index about the package. On Bazel 8.5 and later the build rewritesMODULE.bazel.lockand nothing else, and the only change is that the package's index entry is gone. On 7.7.2 the build changes no file. From then on the next build changes no file, and a--lockfile_mode=errorbuild passes.socket-patch rollback --yes.requirements.txtis byte for byte the one of before, at once. Bazel's change to its own lock is still there.OUT: PRISTINE, and then every file of the checkout is the one of before.... but wanted sha256-<granted>, and the patch host was asked.MODULE.bazel.lockMODULE.bazel.lockThe times are local runs on one Mac (macOS arm64), with the pinned Bazel binaries checked against
scripts/bazel-pins.txt, run the way the workflow's run step runs them. The warm runs were made at this head with a clean tree. The empty-cache runs were made just before the commit, on the samepypi.rs. Each time is the whole step with the test binary already built.Linux x86_64 (the CI runner): the flow passed on the first run there, on all three releases (run 44062755383; checks
pypi / bazel (9.3.2, e9),(8.8.2, e9),(7.7.2, e9)). Test time on the runner, with no cache saved yet: 48 s on 9.3.2, 26 s on 8.8.2, 32 s on 7.7.2.Checks that the new assertions can fail, each a temporary edit on 9.3.2:
Earlier checks, run at an earlier commit of this branch and not repeated at this head: without the scan, and with the upstream bytes served under the patched name, step (c) fails on the marker. Without the tampered bytes, step (e) fails because the build passed. With upstream wheels of fixed bytes, a second run fails the "fetched the upstream wheel" check. With the downloader rule for other projects removed, the build fails instead of asking pypi.org. The flow also passed on the three releases with no
python3onPATH.What starts on main
.github/workflows/bazel-pypi.yml, "Bazel patch compatibility (PyPI)". It calls #2107's reusable workflow for thepypimodule: three Linux jobs (Bazel 9.3.2, 8.8.2, 7.7.2) and one macOS job (9.3.2).When it runs:
pypi.rs,harness.rs, the fixture directory,tests/common/patched_wheel.rs, and 14 PyPI product files named one by one. Those are the Python crawler, the requirements parser and rewriter, the version comparison, the requirements inventory, the index lookup ofrollback, the Python parts of the hosted scan, and three helpers those files call (utils/python_lock.rs,vendor/pypi_wheel.rs,hosted/wheel_record.rs). Shared engine code is not in this list.main.rsof the test target,patch/redirect/**,hosted/**,commands/**,tests/common/**,tests/hosted_memory_common/**,Cargo.lock, theCargo.tomlfiles,rust-toolchain.tomland.cargo/**.The nightly run has a concurrency group of its own, so a push to main cannot replace a nightly that is waiting. Pull request runs cancel the run before them.
When this pull request lands, the push starts the PyPI jobs and, because
main.rsis in the Go workflow's push list, the Go jobs too.What a job asks of the outside world, measured on Bazel 9.3.2 on macOS arm64 through a logging proxy and Bazel's build event files:
bcr.bazel.build(29),github.com(6) andrelease-assets.githubusercontent.com(3). Bazel downloads the ruleset archives and the Python toolchain, 58 MB in all.bcr.bazel.buildonly (29 connections). The fixture commits noMODULE.bazel.lock, so Bazel reads the registry in every run.pypi.orgorfiles.pythonhosted.orgin either case. The loopback server answered every index page: 189 per run on 9.3.2, 285 on 8.8.2 and 380 on 7.7.2. A build that would need the real index fails instead of asking it.socket-patchitself reaches loopback only.8.8.2 and 7.7.2 were not measured through the proxy at this head. Before the test, a job also does what every job of the reusable workflow does: checkout, the Rust build cache, and the pinned Bazel release from github.com.
Caches: the repository cache is one per ecosystem and fixture era, saved from main only. The Rust build cache uses the shared key
dev-<os>and is saved on main only, the same key and rule as the compatibility workflows already on main (go-compatibility.yml). So aCargo.lockchange on main starts three more jobs that may save that shared cache, as each of those workflows already does.Not required for merge: no
merge_grouptrigger, and no job is aneedsofci-ok. Setting the repository variableBAZEL_COMPAT_DISABLEDtotrueturns every job off.Not claimed
uv.lock,requirements_by_platform, a lock below the root,compile_pip_requirements, WORKSPACEpip_parse, aspect_rules_py, rules_pycross, other rules_python releases, a platform wheel or an sdist. The fixture is one pure-Python wheel with one dependency./usr/bin/env python3.Known behaviour the flow records
These are things a user would meet. The test asserts the first four.
--lockfile_mode=errorfails afterscan --mode hosteduntilMODULE.bazel.lockis updated and committed. A normal build orbazel mod deps --lockfile_mode=updateupdates it. The same holds afterrollback. On Bazel 7.7.2 both builds pass.socket-patchdoes not touch that lock and does not mention it. The same happens after the same edit of a requirement made by hand.MODULE.bazel.lockon Bazel 8.5 and later. The package's index entry goes. A user commits that file together withrequirements.txt.rollback,git statusstill shows Bazel's change to its own lock. The next normal build writes the lock of before.rollbackasks the index. It restores the pin with its hashes from the index's JSON document of the release (pypi.org for a user). Read from the product code; the test serves that document from the loopback server.rollbackneeds to know the patch host of another deployment. The loopback server is notpatch.socket.dev, so the test setsSOCKET_PATCH_SERVER_URL. Without itrollbackdoes not take a URL on another host for a pin to undo. Observed when the flow was first written; not asserted.scanalso reads the machine's system-wide Python packages and sends their names to the API. It asks the firstpython3onPATHfor its site-packages when there is one. Measured when the flow was first written (15 to 22 packages on one Mac); not asserted and not measured again.Code review
First round:
rollbackasked it again.tests/common/patched_wheel.rs.zipcrate. The fixture no longer commitsrequirements.txt... The rules come from the harness, which quotes the whole URL.Second round:
utils/python_lock.rs,vendor/pypi_wheel.rsandhosted/wheel_record.rs. Left out on purpose: five files that belong to another project manager and that the listed files also call (vendor/pypi_pipenv.rs,formats/pipenv.rs,utils/poetry_lock.rs,crawlers/hatch_env.rs,utils/hatch.rs). The workflow already leaves other managers' files out, because this flow only finds that the project is not theirs. A change to them gets the nightly run.the_workflow_names_files_that_existis a generic check in this module, and it skips patterns. Left open: it belongs in the harness with a real pattern evaluation, and the harness is Run the hosted Go flow on real Bazel in CI #2107's. The test stays until then.grantbuilds the same JSON aspatched_wheel::mount_grant. Left open: the shared file belongs to other test targets. A comment names the other builder.Cargo.lockchange starts three more jobs that may save the shared Rust cache. Not changed: the reusable workflow uses the same key and save rule as every compatibility workflow on main.Decisions for the owner
scan --mode hostedandrollbackshould name the remedy when aMODULE.bazel.locksits beside the file they rewrite, or update that lock, is a product change for a later pull request. Until then a user whose CI builds with a frozen lock has to update and commit the lock after a hosted scan and after a rollback.🤖 Generated with Claude Code