Skip to content

prelude/rust: from_any_dir.py needs Python ≥ 3.12 (Path.relative_to(walk_up=True)) but the bootstrap toolchain runs whatever host python3 is #1490

Description

@yesme

Summary

prelude/rust/tools/from_any_dir.py (the CC/CXX/LD/AR shim used by cargo_buildscript) calls Path.relative_to(..., walk_up=True), which only exists on Python ≥ 3.12. Nothing in the prelude declares or checks that minimum: system_python_bootstrap_toolchain() runs the shim with whatever bare python3 the host provides. On macOS the Command Line Tools still ship /usr/bin/python3 = 3.9.6 (verified on macOS 26.6.2), so every Rust build script that compiles C through the shim fails on a stock Mac.

Where

  • prelude/rust/tools/from_any_dir.py line 40 (unchanged between the 2026-08-22 tag and main as of 2026-09-04):
    if original_cwd.is_relative_to(interim_cwd):
        interim_cwd = interim_cwd.relative_to(original_cwd, walk_up=True)
    walk_up was added in Python 3.12 (https://docs.python.org/3/library/pathlib.html#pathlib.PurePath.relative_to).
  • prelude/toolchains/python.bzl: system_python_bootstrap_toolchain defaults the interpreter to the bare name python3 (_interpreter_for_os), so the shim runs under the host's default Python.
  • prelude/rust/cargo_buildscript.bzl _make_cc_shim: wraps CC/CXX/LD/AR in the shim for every buildscript_run.

Reproduce

  1. A host whose default python3 is < 3.12 (e.g. macOS with only the Command Line Tools: /usr/bin/python3 --versionPython 3.9.6).
  2. Default toolchains: system_python_bootstrap_toolchain() and system_cxx_toolchain().
  3. Any Rust crate whose build.rs compiles C via the cc crate, run as a cargo_buildscript (e.g. Reindeer-generated libsqlite3-sys 0.38.2 with buildscript.run = true, bundled feature).

Result: the build script's first cc invocation goes through __cc_shim.shpython3 from_any_dir.py … and dies with

TypeError: relative_to() got an unexpected keyword argument 'walk_up'

Real-world case: yesme/hctl2#157 (fixed downstream in yesme/hctl2#165 by pinning a python-build-standalone 3.12 interpreter and injecting its absolute path into the bootstrap toolchain).

Suggested fix

Any of these would do; the first is a one-line change and drops the 3.12 requirement entirely:

  1. Use os.path.relpath, which has walked up since forever:
    interim_cwd = Path(os.path.relpath(interim_cwd, original_cwd))
  2. Or keep walk_up but fail early with a clear message (sys.version_info < (3, 12)) instead of a TypeError from inside a build-script shim.
  3. And/or document the minimum interpreter version for system_python_bootstrap_toolchain in prelude/toolchains/python.bzl.

Happy to send a PR for (1) if that is the preferred direction.

Environment

  • buck2 prelude 2026-08-22 (file identical on main at 2026-09-04)
  • macOS 26.6.2 arm64, /usr/bin/python3 3.9.6 (Apple Command Line Tools)
  • Reindeer v2026.08.24.00, libsqlite3-sys 0.38.2

Companion issue about the same shim: #1491

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions