Skip to content

fix: honour escape=False for a leading newline in multiline strings - #656

Open
shivampwm2020 wants to merge 1 commit into
python-poetry:masterfrom
shivampwm2020:fix/655-multiline-leading-newline-no-escape
Open

shivampwm2020 wants to merge 1 commit into
python-poetry:masterfrom
shivampwm2020:fix/655-multiline-leading-newline-no-escape

Conversation

@shivampwm2020

Copy link
Copy Markdown

Problem

tomlkit.string(..., escape=False) can no longer render a multiline string whose value starts with a newline. The value is correct in memory but the serialized document gains a spurious leading newline:

import tomlkit

s = tomlkit.string("\n  aligned\n  strings\n", multiline=True, literal=True, escape=False)
print(repr(tomlkit.dumps(s)))

Clean master:

"\'\'\'\n\n  aligned\n  strings\n\'\'\'"

The extra blank line after the opening \'\'\' means the intended output \'\'\'\n aligned\n strings\n\'\'\' cannot be produced at all - matching the report in #655.

Root cause

tomlkit/items.py, String.from_raw (~line 2314):

if type_.is_multiline() and string_value[:1] in ("\n", "\r"):
    string_value = "\n" + string_value

This re-adds the newline that the parser trims immediately after a multiline opening delimiter, so a leading newline survives a round-trip. It was introduced unconditionally by PR #551 ("fix: preserve leading newline of multiline string built with string()", commit 8cd44f5).

The compensation only makes sense when escape=True. With escape=False the caller explicitly owns the raw body (see tomlkit/api.py:137-153), so prepending a newline corrupts it. Before #551 the escape=False case rendered verbatim.

Fix

Gate the compensation on escape:

if escape and type_.is_multiline() and string_value[:1] in ("\n", "\r"):
    ...

escape=True keeps #551's round-trip preservation (the existing test_create_multiline_string_with_leading_newline_round_trips still passes); escape=False now emits the body exactly as given.

Testing

Regression tests added to tests/test_api.py:

  • test_create_multiline_literal_string_with_leading_newline_no_escape
  • test_create_multiline_basic_string_with_leading_newline_no_escape

RED (test only, fix reverted):

$ python -m pytest tests/test_api.py -k no_escape -q
assert "\'\'\'\n\n  ali... strings\n\'\'\'" == "\'\'\'\n  align... strings\n\'\'\'"
2 failed

GREEN (with fix):

$ python -m pytest tests/test_api.py -k "no_escape or leading_newline" -q
7 passed

Full suite on the fixed tree: 1204 passed in 6.00s. ruff check tomlkit/items.py tests/test_api.py -> All checks passed!

Fixes #655

PR python-poetry#551 re-added the newline trimmed by the parser for any multiline
string whose value starts with one, but did so regardless of the escape
flag. With escape=False the caller owns the raw body, so the extra
newline made it impossible to render such a value as written.

Gate the compensation on escape so the raw body is emitted verbatim when
escaping is disabled, restoring the pre-python-poetry#551 behaviour for that case.

Fixes python-poetry#655

@Wkhanex Wkhanex left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With escape=False, multiline basic and literal strings retain the exact supplied raw body, including LF and CRLF at the start. Default escaping still preserves the original parsed value, so the raw-body fix does not remove the protective newline on the ordinary path.

Checked 739d8bc8481d with standalone Python 3.12.14 checks (12 passing assertions). I haven't run the full upstream suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Regression: impossible to dump multiline strings with leading newline

2 participants