Repository navigation
Conversation
github-actions
Bot
added
Component: C++
Component: Python
Component: Documentation
labels
…al types Squash of apache#55264 (97a1f74) on top of main, to stack the stub docstring changes on it. Not for merging.
rok
force-pushed
the
pyarrow-stubs-lib-docstrings
branch
2 times, most recently
from
November 13, 2026 16:37
8c663f0 to
59936e6
Compare
Member
Author
|
@github-actions crossbow submit -g wheel |
|
Revision: 59936e6 Submitted crossbow builds: ursacomputing/crossbow @ actions-087799c9ee
|
rok
force-pushed
the
pyarrow-stubs-lib-docstrings
branch
from
November 13, 2026 16:58
59936e6 to
b44e1d7
Compare
Member
Author
|
@github-actions crossbow submit -g wheel |
|
Revision: b44e1d7 Submitted crossbow builds: ursacomputing/crossbow @ actions-24014aafd7
|
…ype stubs with docstring-adder Squash the stub-only pyarrow.__lib package into lib.pyi, with a section per .pxi file, since pyarrow.lib is a single extension module built from the .pxi files it includes. The per-file __all__ lists go away, and so does the aliasing of pyarrow.__lib modules to pyarrow.lib in run_stubtest.py. Replace the libcst codemod in update_stub_docstrings.py with docstring-adder (https://github.com/astral-sh/docstring-adder). The script still assembles an importable pyarrow from the source and install trees, then runs docstring-adder on the installed stubs and checks that lib.pyi got docstrings. Build the Cython modules with embedsignature.format=clinic, so that the embedded signatures go to __text_signature__ instead of the first lines of __doc__ and don't end up in the docstrings of the stubs. Install the stubs into wheels again and add the docstrings in the wheel builds (PYARROW_REQUIRE_STUB_DOCSTRINGS=ON). py.typed stays excluded from wheels until the stubs are complete (apacheGH-49831). Check the stubs in each built wheel against the modules in the same wheel with stubtest (run_stubtest.py --wheel) in the wheel build scripts. The wheel builds don't install the pandas and scipy stubs, so mypy treats those modules as Any when their stubs are missing. The wheel content validation checks again that the wheels contain the stubs of pyarrow-stubs/ with docstrings, and still that they don't contain py.typed. libcst is no longer a build requirement. docstring-adder isn't on PyPI, so requirements-wheel-build.txt pins it to a commit. Bump the Windows wheel image revision, so that the image installs the new requirements.
rok
force-pushed
the
pyarrow-stubs-lib-docstrings
branch
from
November 13, 2026 19:24
b44e1d7 to
f5af72b
Compare
Member
Author
|
@github-actions crossbow submit wheel-windows-* wheel-macos-monterey-cp315-cp315-amd64 |
|
Revision: f5af72b Submitted crossbow builds: ursacomputing/crossbow @ actions-48add857b9
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Closes #55596.
The runtime docstrings are inserted into the type stubs by a custom libcst codemod (
python/scripts/update_stub_docstrings.py), which needs a hand-written mapping from stub files to runtime modules and makes libcst a build requirement of every source build. docstring-adder does the same job generically.Stacked on #55264; only the last two commits belong to this PR.
What changes are included in this PR?
pyarrow.__libpackage intolib.pyi, one section per.pxifile, matching the singlepyarrow.libextension module. This drops the per-file__all__lists and the module aliasing inrun_stubtest.py, and lets docstring-adder maplib.pyitopyarrow.libby name.update_stub_docstrings.pynow runs docstring-adder on the installed stubs (it still assembles an importable pyarrow from the source and install trees) and fails iflib.pyigot no docstrings.embedsignature.format=clinic, so the embedded signature goes to__text_signature__instead of the first lines of__doc__and out of the stub docstrings.PYARROW_REQUIRE_STUB_DOCSTRINGS=ON).py.typedstays excluded from wheels until the stubs are complete ([Python] type checkers broken for pyarrow.compute and other sub-modules since PyArrow 24.0+ #49831).run_stubtest.py --wheel, no install needed). The wheel builds don't have the pandas and scipy stubs, so mypy treats those modules asAnywhen their stubs are missing.requirements-wheel-build.txtpins it to a commit. The macOS Python 3.15 workaround for pyyaml (pulled in by libcst) is removed.Are these changes tested?
Locally with a minimal Arrow C++ build and
python -m build --wheel --no-isolationwithPYARROW_REQUIRE_STUB_DOCSTRINGS=ON:lib.pyihave docstrings (the rest have none at runtime, e.g. exception classes and Cython properties), and none starts with a signature line.run_stubtest.py --wheelpasses on the wheel in an environment with only the wheel build requirements, and fails on a wheel with an extra stub function and a renamed parameter.inspect.signatureof thepyarrow.libfunctions and methods is unchanged by the clinic format.Are there any user-facing changes?
Wheels ship the type stubs with docstrings again (without
py.typed, so mypy keeps ignoring them). The__doc__of Cython functions no longer starts with the signature;help()andinspect.signature()still show it.Was AI used for this PR?
In accordance to the AI generation guidelines, please disclose below whether and how AI was used in this PR.
PR code and description written by:
Reviewed before submission by: