Skip to content

GH-46872: [C++][Python] Move Arange utility function to an Arrow C++ - #48710

Open
EslamAhmed171 wants to merge 5 commits into
apache:mainfrom
EslamAhmed171:GH-46872/cpp-arange-utility
Open

EslamAhmed171 wants to merge 5 commits into
apache:mainfrom
EslamAhmed171:GH-46872/cpp-arange-utility

Conversation

@EslamAhmed171

@EslamAhmed171 EslamAhmed171 commented Jan 2, 2026 •

Copy link
Copy Markdown
Contributor

Rationale for this change

Move the Arange utility function from Python-specific code to Arrow C++ core utilities.

What changes are included in this PR?

Relocate the Arange implementation from arrow/python/util.{h,cc} to Arrow C++ utility files.

Are these changes tested?

Yes, existing tests for pa.arange continues to pass.

Are there any user-facing changes?

No user-facing changes. This is an internal code refactoring that maintains the existing pyarrow.arange API.

@alippai

alippai commented Jan 3, 2026

Copy link
Copy Markdown
Contributor

Looks good, is there a need for handling pathological cases like skip == INT64_MIN and overflowing sizes here?

@rok

rok commented Jan 4, 2026

Copy link
Copy Markdown
Member

Thanks for working on this @EslamAhmed171 !
It would be nice to add some C++ tests for this and cover edge cases as @alippai suggests.

@EslamAhmed171

Copy link
Copy Markdown
Contributor Author

It would be nice to add some C++ tests for this and cover edge cases as @alippai suggests.

Sorry for the late response, and thank you both for your reviews!
I have addressed both comments

@EslamAhmed171

Copy link
Copy Markdown
Contributor Author

@rok Friendly ping!

Comment thread cpp/src/arrow/array/util.cc Resolved
@Reranko05

Copy link
Copy Markdown
Collaborator

@EslamAhmed171 Sorry for the delayed review! Are you still interested in working on this PR? If so, could you please address the feedback above and rebase your branch against the latest main? Thanks!

@EslamAhmed171
EslamAhmed171 force-pushed the GH-46872/cpp-arange-utility branch from 4d911f8 to 6dbd59b Compare November 14, 2026 21:46
@EslamAhmed171

EslamAhmed171 commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor Author

@EslamAhmed171 Sorry for the delayed review! Are you still interested in working on this PR? If so, could you please address the feedback above and rebase your branch against the latest main? Thanks!

Thanks for the review!

I revisited the implementation and found it had several false-positive rejections, ranges that are perfectly valid but were being rejected because intermediate computations overflowed int64_t. I've reworked the computation to be done entirely in uint64_t, and the byte-size computation now uses MultiplyWithOverflow() as you suggested.

In both branches the range is guaranteed to be in [1, 2^64 - 1], so casting the endpoints to uint64_t and subtracting is exact (unsigned subtraction wraps mod 2^64). |step| is 0 - static_cast<uint64_t>(step), and size = (range - 1) / |step| + 1 can't overflow because range >= 1 and the result is <= range.

Rejection now happens only for step == 0 or when allocation is impossible (byte size overflows uint64_t, or exceeds INT64_MAX). The first rejects size >= 2^61, the second size >= 2^60, giving a real cap of size <= 2^60 - 1.

Cases that were previously rejected but are now accepted

The old code computed the range and size in int64_t (stop - start and (range + step - 1) / step for positive steps, the mirrored form for negative steps), which overflows for wide ranges even when the resulting array is small:

Input Old behavior New behavior
Arange(INT64_MIN, INT64_MAX, 1LL << 62) Invalid (overflow in stop - start) [INT64_MIN, -2^62, 0, 2^62]
Arange(INT64_MAX, INT64_MIN, -(1LL << 62)) Invalid [INT64_MAX, 2^62 - 1, -1, -2^62 - 1]
Arange(10, 0, INT64_MIN) Invalid (-step overflows) [10]
Arange(INT64_MAX, INT64_MIN, INT64_MIN) Invalid [INT64_MAX, -1]

These are all covered by the new TestArray.ArangeBoundaryValues test; the rejection paths are covered by TestArray.ArangeInvalidInputs.

Happy to get another round of review if anything else needs tightening.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants