Skip to content

fix: carry over the fallback hook factories in Converter.copy() - #807

Open
simpleqt wants to merge 1 commit into
python-attrs:mainfrom
simpleqt:fix/copy-fallback-factories
Open

simpleqt wants to merge 1 commit into
python-attrs:mainfrom
simpleqt:fix/copy-fallback-factories

Conversation

@simpleqt

Copy link
Copy Markdown

Do not merge before reading the Generative AI / LLM Policy note at the bottom.

Problem

BaseConverter.copy() and Converter.copy() (and therefore copy.deepcopy(), which delegates to copy()) silently drop the converter's unstructure_fallback_factory and structure_fallback_factory.

A copy of a converter that was configured with custom fallbacks falls back to the built-in defaults instead:

  • unstructuring a type with no hooks returns the object unchanged (the default lambda _: identity) instead of calling the custom factory, and
  • structuring such a type raises StructureHandlerNotFoundError instead of calling the custom factory.

Every other constructor option is carried over — dict_factory, unstruct_strat, omit_if_default, forbid_extra_keys, type_overrides, unstruct_collection_overrides, prefer_attrib_converters, detailed_validation and use_alias — so the two fallback factories look like an oversight rather than a decision. copy() is also the documented way of cloning a converter ("Converters may be cloned using the Converter.copy() method"), and deepcopy() is just copy() underneath.

The same bug class was reported and fixed twice before: #398 (copy lost unstructure hooks, fixed in #399), #410 (copy lost forbid_extra_keys, hence the forbid_extra_keys parameter of copy()).

Reproduction

Save as scratch_repro.py next to a checkout and run it with an interpreter that has cattrs from main installed:

from copy import deepcopy

from cattrs import BaseConverter, Converter


class Unhandled:
    """A type with no registered, generated or default hooks."""


def unstructure_fallback(_):
    return lambda v: "unstructured-by-fallback"


def structure_fallback(_):
    return lambda v, _type: "structured-by-fallback"


def probe(label, converter):
    try:
        u = converter.unstructure(Unhandled())
    except Exception as e:
        u = f"{type(e).__name__}: {e}"
    try:
        s = converter.structure("x", Unhandled)
    except Exception as e:
        s = f"{type(e).__name__}: {e}"
    print(f"{label:24} unstructure -> {u!r}\n{'':24} structure   -> {s!r}")


converter = Converter(
    unstructure_fallback_factory=unstructure_fallback,
    structure_fallback_factory=structure_fallback,
)

print("== cattrs.Converter ==")
probe("original", converter)
probe("copy()", converter.copy())
probe("deepcopy()", deepcopy(converter))

Commands:

python -m venv .venv
uv pip install --python .venv/Scripts/python.exe -e . pytest
.venv/Scripts/python.exe scratch_repro.py

Observed on main (57333a2):

== cattrs.Converter ==
original                 unstructure -> 'unstructured-by-fallback'
                         structure   -> 'structured-by-fallback'
copy()                   unstructure -> <__main__.Unhandled object at 0x000001EE1F772250>
                         structure   -> "StructureHandlerNotFoundError: Unsupported type: <class '__main__.Unhandled'>. Register a structure hook for it."
deepcopy()               unstructure -> <__main__.Unhandled object at 0x000001EE1F772250>
                         structure   -> "StructureHandlerNotFoundError: Unsupported type: <class '__main__.Unhandled'>. Register a structure hook for it."

Expected:

copy()                   unstructure -> 'unstructured-by-fallback'
                         structure   -> 'structured-by-fallback'
deepcopy()               unstructure -> 'unstructured-by-fallback'
                         structure   -> 'structured-by-fallback'

The same happens for BaseConverter.

Root cause

BaseConverter.__init__ hands unstructure_fallback_factory and structure_fallback_factory to MultiStrategyDispatch, where they are stored, but never keeps a reference on the converter itself. copy() builds a fresh converter through self.__class__(...) without passing the factories, so the new instance picks up the defaults from the __init__ signature:

unstructure_fallback_factory: HookFactory[UnstructureHook] = lambda _: identity,
structure_fallback_factory: HookFactory[StructureHook] = lambda t: raise_error(None, t),

No public attribute exposes them either, which is why they cannot be recovered after the fact.

Fix

  • Store both factories on the converter in BaseConverter.__init__ (self._unstructure_fallback_factory, self._structure_fallback_factory; two new __slots__ entries).
  • Add unstructure_fallback_factory and structure_fallback_factory parameters to BaseConverter.copy() and Converter.copy(). They default to None, meaning "keep the current value", matching how every other parameter of copy() behaves, and they can be used to override the factories on the copy.
  • The new parameters are appended to both signatures, so existing positional calls keep working.
  • A HISTORY.md entry under NEXT.

git diff --stat:

 HISTORY.md                 |  3 +++
 src/cattrs/converters.py   | 44 ++++++++++++++++++++++++++++++++++++++++++--
 tests/test_copy.py         | 70 ++++++++++++++++++++++++++++++++++++++++++++++++++++++

Tests

Two new tests in tests/test_copy.py:

  • test_copy_fallback_factories — parametrized over BaseConverter/Converter and over unstructuring strategy, prefer_attrib_converters, detailed_validation and dict_factory; asserts the custom fallbacks still apply after both copy() and deepcopy().
  • test_copy_fallback_factories_overrides — asserts the factories can be replaced on the copy, and that the original is untouched.

Both fail on main and pass with the fix.

RED (git checkout main -- src/cattrs/converters.py, new tests present):

FAILED tests/test_copy.py::test_copy_fallback_factories[BaseConverter] - AssertionError: assert <tests.test_copy.Unhandled object at 0x...> == 'unstructured-by-fallback'
FAILED tests/test_copy.py::test_copy_fallback_factories[Converter] - AssertionError: assert <tests.test_copy.Unhandled object at 0x...> == 'unstructured-by-fallback'
FAILED tests/test_copy.py::test_copy_fallback_factories_overrides - TypeError: Converter.copy() got an unexpected keyword argument 'unstructure_fallback_factory'
3 failed, 11 passed, 2 warnings in 1.22s

GREEN (with the fix):

14 passed, 2 warnings in 0.49s

Full suite (python -m pytest tests -q --benchmark-disable), with the fix: 2 failed, 1020 passed, 2 skipped, 15 xfailed. The two failures are pre-existing on this machine and unrelated:

FAILED tests/strategies/test_include_subclasses.py::test_dataclasses_slots[True]
FAILED tests/strategies/test_include_subclasses.py::test_dataclasses_slots[False]

They are a Python 3.11.0 artifact: @dataclass(slots=True) creates a second class and Python 3.11.0 keeps the discarded one alive, so ParentDC.__subclasses__() returns it twice and cattrs.subclasses.subclasses() (which filters those duplicates only on 3.13+) sees two identical children. gc.collect() frees it on 3.11.14/3.12.12/3.14.2, where these tests pass; the project's CI is green on 3.11. They fail identically before this change.

Lint (the project's just lint):

uv run --python 3.14 --group lint ruff check src/ tests bench          # All checks passed!
uv run --python 3.14 --group lint ruff format --check src tests docs/conf.py  # 116 files already formatted

Notes / limitations

  • mypy --strict on src/cattrs/converters.py reports 109 errors both before and after the change (the file is not mypy-clean in that ad-hoc configuration; the project only runs Mypy/Pyrefly through pytest-typing on CPython 3.14). No new diagnostics are introduced; the pre-existing Signature of "copy" incompatible with supertype "BaseConverter" note is unchanged.
  • Not run: the pytest-typing suites (tests/test_typing_structure.md), which require CPython 3.14 plus pytest-typing; and the PyPy 3.10 job.

AI assistance

This change was developed with the help of an LLM-based coding assistant (DeepSeek Harness), used for repository exploration, reproduction and drafting. Per the project's generative AI policy there are no LLM co-author tags anywhere in the commit, the commit is authored and signed off by the human submitter, and the submitter has verified the reproduction, the fix and the test results and takes full responsibility for every line of it.

Problem
-------
`BaseConverter.copy()` and `Converter.copy()` (and therefore `copy.deepcopy()`,
which delegates to `copy()`) silently drop the converter's
`unstructure_fallback_factory` and `structure_fallback_factory`. A copy of a
converter configured with custom fallbacks falls back to the built-in defaults
instead: unstructuring returns the object as is and structuring raises
`StructureHandlerNotFoundError`.

Every other constructor option - `dict_factory`, `unstruct_strat`,
`omit_if_default`, `forbid_extra_keys`, `type_overrides`,
`unstruct_collection_overrides`, `prefer_attrib_converters`,
`detailed_validation` and `use_alias` - is carried over, and the fallback
factories were simply never threaded through, so this is an oversight rather
than a design decision. `Converter.copy` and `BaseConverter.copy` are also the
documented way of cloning a converter ("Converters may be cloned using the
`Converter.copy()` method", `docs/indepth.md`).

Cause
-----
`MultiStrategyDispatch` receives the fallback factory in its constructor, but
`BaseConverter.__init__` never stored it on the converter, and `copy()` does not
pass the factories to the new instance. The new converter therefore falls back
to the defaults of the `__init__` signature.

Fix
---
Store both factories on the converter in `BaseConverter.__init__`, and add
`unstructure_fallback_factory` and `structure_fallback_factory` parameters to
`BaseConverter.copy` and `Converter.copy` that default to the current values, so
they are both carried over and individually overridable on the copy. The new
parameters are appended to the signatures, keeping existing positional calls
valid.

Tests
-----
`tests/test_copy.py`:

* `test_copy_fallback_factories` - custom fallbacks still apply after `copy()`
  and `deepcopy()`, for both `BaseConverter` and `Converter`.
* `test_copy_fallback_factories_overrides` - the factories can be overridden on
  the copy without affecting the original.

Both fail on `main` and pass with the fix; the rest of the suite is unaffected.

Signed-off-by: simpleqt <2015900818@qq.com>
Copilot AI balanced review requested due to automatic review settings October 20, 2026 10:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@codspeed

codspeed Bot commented Oct 21, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

1 note

ℹ️ 64 benchmarks have no flamegraph

The flamegraph wasn't generated: the symbols of the profiling trace could not be resolved, because no information about the language frames was collected.

See affected benchmarks

✅ 64 untouched benchmarks


Comparing simpleqt:fix/copy-fallback-factories (d5ace96) with main (bde101a)1

Open in CodSpeed

Footnotes

  1. No successful run was found on main (57333a2) during the generation of this report, so bde101a was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@Tinche

Tinche commented Oct 21, 2026

Copy link
Copy Markdown
Member

Do we need the extra attributes? We should already have them in self._un/structure_func._fallback_factory, we could read them from there?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants