Skip to content

Re-escape strings concatenated across quote styles - #653

Open
xiehuanyi wants to merge 2 commits into
python-poetry:masterfrom
xiehuanyi:fix-mixed-string-quote-concat
Open

xiehuanyi wants to merge 2 commits into
python-poetry:masterfrom
xiehuanyi:fix-mixed-string-quote-concat

Conversation

@xiehuanyi

Copy link
Copy Markdown

Summary

Appending a literal String to a basic String reuses raw TOML text with incompatible escaping: a literal backslash path can turn into a tab when the result is parsed. The reverse direction preserves escape spellings as literal text instead of their decoded value. Re-escape cross-style operands for the left string type, retaining the raw spelling for matching types and escaping bare CRs for multiline basic output.

Validation: 9 regression failures and 9 passing controls before the change; all 1202 tests pass afterward, including the TOML conformance corpus. All pre-commit hooks pass. Mypy 1.19.1 reports the same six existing diagnostics on pristine master and the patched tree, with no added diagnostics.

Agent Drafting Metadata

  • Agent: Codex
  • Model: GPT-6.1 Sol
  • Notes: Investigated the raw-spelling mismatch, independently reproduced the round-trip failures, added the quote-style matrix and a CR control, and ran the full suite and checks.

Copilot AI balanced review requested due to automatic review settings October 12, 2026 20:06

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Wkhanex Wkhanex left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-style concatenation still emits invalid TOML when the left operand is literal and the right value cannot fit that literal style. Both operands can be valid individually:

a = tomlkit.string("left", literal=True)
b = tomlkit.string("newline" + chr(10))
c = a + b
tomllib.loads("x=" + c.as_string())  # TOMLDecodeError

The resulting Python string ends in a newline, but as_string emits a single-quoted literal containing that newline. I also reproduced invalid literal output for a basic-string value containing apostrophes, and invalid multiline-literal output for bare carriage returns/triple apostrophes. I used Python 3.12.14 against commit 715989b.

The branch uses no escaping when the left style has no escaped_sequences, and _new bypasses String.from_raw validation. Could it reject unrepresentable literal content consistently with from_raw, or deliberately switch to an escapable style? My 108-case style/content matrix used stdlib tomllib as the serialization oracle. This is a remaining edge case in the change’s cross-style scope; I did not run the full pytest suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants