Repository navigation
feat(swift): support standard row format - #4224
Open
yash-agarwa-l wants to merge 20 commits into
Open
yash-agarwa-l wants to merge 20 commits into
yash-agarwa-l wants to merge 20 commits into
Conversation
A standard row is written in two moves: reserve a zeroed fixed region for the null bitmap and the 8-byte slots, then come back and patch each slot once its value, or the offset and size of its variable-width data, is known. ByteBuffer owns the storage, so it gains the two primitives that make those moves possible. appendZeros reserves the region and keeps null slots and padding deterministic. storeLittleEndian patches an integer at any absolute offset, aligned or not, because a row may begin anywhere in the buffer.
Every row and array opens with a null bitmap: one bit per field or element, packed eight to a byte and padded to whole 8-byte words. Bit i sits in byte i / 8 of the bitmap, counted from the least significant bit, and the bitmap itself starts wherever its row or array starts.
A variable-width value lives after the fixed slots, and its slot holds a single little-endian UInt64 ticket: the offset from the start of its row or array in the upper 32 bits and the byte size in the lower 32 bits. Decoding keeps both halves as UInt32 and checks offset plus size in UInt64, so a corrupt ticket throws instead of trapping, even where Int is 32 bits.
A row stores only values in fixed seats, so the writer and the reader must agree on a schema that says how many fields there are, what type each one has, and whether it can be null. RowDataType, RowField, and RowSchema describe that, using the cross-language ids from TypeId and the list and map child names Java uses, so a Swift schema matches Java's exactly once it is written as bytes and hashed.
A Row is a view over the bytes of one row. It checks once, when it is created, that the bytes are long enough to hold the bitmap and every slot, so the fixed-width getters can read straight from their slots. A string, binary, or nested struct getter first checks the null bit, then checks its offset and size against the row before reading. Strings go through the same UTF-8 check ByteBuffer already used, now shared.
An array starts with its element count, and a forged count could make the size arithmetic overflow, so the count is first compared with the bytes that are actually there and only then multiplied by the element width. Like Row, RowArray and RowMap check everything once when they are created, including that a map has as many values as keys, so no getter can read past the end.
RowWriter reserves a zeroed bitmap and slot region, then fills each slot in place. Strings and bytes go after the slots, padded to 8 bytes, and their slot records the offset from the row's own start, so the same code can later write a nested row inside its parent. A Date is rounded to the nearest microsecond, so a timestamp read from a row and written back gives the same bytes.
RowArrayWriter writes the element count and bitmap, then packs each element at its natural width, with strings and nested values after the elements. RowMapWriter only reserves the keys-size word: the keys and values are two arrays written one after the other, and finishKeys fills in the size between them. A nested struct, list, or map is a child writer on the parent's buffer whose start and size go in the parent's slot, so no extra API is needed for nesting.
A row holds only values, so a reader in another language needs the schema too. toBytes writes each field as a header byte with the name encoding, name size, and nullable flag, then the packed name and the type, with lists, maps, and structs followed by their child fields. RowSchema(bytes:) reads that back, and hash folds the type ids into one number a reader can check before trusting a row.
RowCodable lets each Swift type say how it sits in a row: its row type, whether its field is nullable in the schema, its width, and how it writes and reads itself. Rows and arrays share one write surface and one read surface, so a single conformance serves a struct field or an array element. An optional reuses its wrapped type and stores nil as the null bit, and reading a null into a type that cannot be nil throws.
Arrays and sets are written as a child array after the slots, and a dictionary as its keys array followed by its values array, each element using its own RowCodable rule, so collections nest freely. Data takes the same list shape but copies its bytes in one go, string enums opt in with an empty conformance, and a RowStruct is a child row. RowEncoder puts the 8-byte schema hash in front of the row and checks it before decoding.
@ForyRow reads a struct's stored properties and writes its RowStruct members: a schema with the fields sorted by name and renamed to lower underscore form, so every runtime puts them in the same slots under the same names, a writer that stores each field in its slot, and a reader that calls the memberwise initializer. Types a row cannot hold, such as unsigned integers, nested optionals, tuples, and Date map keys, are reported on the property itself.
The fixtures are rows another runtime wrote for the shared cross-language models, so these tests catch layout drift before the interop runs.
SwiftLint wants identifiers of at least two characters, and the point helper in the array writer tests used x.
Each case checks the decoded value and replays the row through the writers in read order, so maps must match the Java bytes exactly even though Swift dictionary order is unspecified.
It runs the row format cases of GoCrossLanguageTest against the Swift peer, and the byte array case also checks that Swift wrote Java's bytes exactly.
The Swift xlang job now also builds the row format peer and runs SwiftCrossLanguageTest after SwiftXlangTest, as the Go job does.
Covers the @ForyRow macro, RowEncoder, the read views, the type mapping, and hand-written rows, and lists Swift wherever the docs name the Standard Row languages.
yash-agarwa-l
requested review from
PragmaTwice and
theweipeng
as code owners
November 12, 2026 14:55
Apply swift-format to the row reader and writer tests so the Swift format check in CI passes.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why?
Swift has no row format, so it can't exchange row data with Java, Go and the other runtimes
that already support it.
The part worth checking is map handling. Swift dictionaries have no fixed order, so the
entries in a row with a map can come out in a different order than Java's. The interop test
checks the layout by replaying Java's bytes through the Swift writers, not by comparing
encoder output.
What does this PR do?
Adds standard row format reading and writing to Swift, with a
@ForyRowmacro that makes astruct encodable through
RowEncoder, plus a Java test that runs Swift against Java's rowsin the existing Swift CI job.
Known limits: classes and generic structs aren't supported.
Related issues
Closes #3378
AI Contribution Checklist
yes/noyes, I included a completed AI Contribution Checklist in this PR description and the requiredAI Usage Disclosure.yes, my PR description includes the requiredai_reviewsummary and screenshot evidence or equivalent persisted links of the final clean AI review results from both fresh reviewers described inAI_POLICY.md, the Fory-guided reviewer and the independent general reviewer, on the current PR diff or current HEAD after the latest code changes.Does this PR introduce any user-facing change?