Repository navigation
Conversation
Problem
-------
`BaseConverter.copy()` and `Converter.copy()` (and therefore `copy.deepcopy()`,
which delegates to `copy()`) silently drop the converter's
`unstructure_fallback_factory` and `structure_fallback_factory`. A copy of a
converter configured with custom fallbacks falls back to the built-in defaults
instead: unstructuring returns the object as is and structuring raises
`StructureHandlerNotFoundError`.
Every other constructor option - `dict_factory`, `unstruct_strat`,
`omit_if_default`, `forbid_extra_keys`, `type_overrides`,
`unstruct_collection_overrides`, `prefer_attrib_converters`,
`detailed_validation` and `use_alias` - is carried over, and the fallback
factories were simply never threaded through, so this is an oversight rather
than a design decision. `Converter.copy` and `BaseConverter.copy` are also the
documented way of cloning a converter ("Converters may be cloned using the
`Converter.copy()` method", `docs/indepth.md`).
Cause
-----
`MultiStrategyDispatch` receives the fallback factory in its constructor, but
`BaseConverter.__init__` never stored it on the converter, and `copy()` does not
pass the factories to the new instance. The new converter therefore falls back
to the defaults of the `__init__` signature.
Fix
---
Store both factories on the converter in `BaseConverter.__init__`, and add
`unstructure_fallback_factory` and `structure_fallback_factory` parameters to
`BaseConverter.copy` and `Converter.copy` that default to the current values, so
they are both carried over and individually overridable on the copy. The new
parameters are appended to the signatures, keeping existing positional calls
valid.
Tests
-----
`tests/test_copy.py`:
* `test_copy_fallback_factories` - custom fallbacks still apply after `copy()`
and `deepcopy()`, for both `BaseConverter` and `Converter`.
* `test_copy_fallback_factories_overrides` - the factories can be overridden on
the copy without affecting the original.
Both fail on `main` and pass with the fix; the rest of the suite is unaffected.
Signed-off-by: simpleqt <2015900818@qq.com>
Merging this PR will not alter performance1 note
Comparing Footnotes |
Member
|
Do we need the extra attributes? We should already have them in |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Do not merge before reading the Generative AI / LLM Policy note at the bottom.
Problem
BaseConverter.copy()andConverter.copy()(and thereforecopy.deepcopy(), which delegates tocopy()) silently drop the converter'sunstructure_fallback_factoryandstructure_fallback_factory.A copy of a converter that was configured with custom fallbacks falls back to the built-in defaults instead:
lambda _: identity) instead of calling the custom factory, andStructureHandlerNotFoundErrorinstead of calling the custom factory.Every other constructor option is carried over —
dict_factory,unstruct_strat,omit_if_default,forbid_extra_keys,type_overrides,unstruct_collection_overrides,prefer_attrib_converters,detailed_validationanduse_alias— so the two fallback factories look like an oversight rather than a decision.copy()is also the documented way of cloning a converter ("Converters may be cloned using theConverter.copy()method"), anddeepcopy()is justcopy()underneath.The same bug class was reported and fixed twice before: #398 (copy lost unstructure hooks, fixed in #399), #410 (copy lost
forbid_extra_keys, hence theforbid_extra_keysparameter ofcopy()).Reproduction
Save as
scratch_repro.pynext to a checkout and run it with an interpreter that hascattrsfrommaininstalled:Commands:
python -m venv .venv uv pip install --python .venv/Scripts/python.exe -e . pytest .venv/Scripts/python.exe scratch_repro.pyObserved on
main(57333a2):Expected:
The same happens for
BaseConverter.Root cause
BaseConverter.__init__handsunstructure_fallback_factoryandstructure_fallback_factorytoMultiStrategyDispatch, where they are stored, but never keeps a reference on the converter itself.copy()builds a fresh converter throughself.__class__(...)without passing the factories, so the new instance picks up the defaults from the__init__signature:No public attribute exposes them either, which is why they cannot be recovered after the fact.
Fix
BaseConverter.__init__(self._unstructure_fallback_factory,self._structure_fallback_factory; two new__slots__entries).unstructure_fallback_factoryandstructure_fallback_factoryparameters toBaseConverter.copy()andConverter.copy(). They default toNone, meaning "keep the current value", matching how every other parameter ofcopy()behaves, and they can be used to override the factories on the copy.HISTORY.mdentry underNEXT.git diff --stat:Tests
Two new tests in
tests/test_copy.py:test_copy_fallback_factories— parametrized overBaseConverter/Converterand over unstructuring strategy,prefer_attrib_converters,detailed_validationanddict_factory; asserts the custom fallbacks still apply after bothcopy()anddeepcopy().test_copy_fallback_factories_overrides— asserts the factories can be replaced on the copy, and that the original is untouched.Both fail on
mainand pass with the fix.RED (
git checkout main -- src/cattrs/converters.py, new tests present):GREEN (with the fix):
Full suite (
python -m pytest tests -q --benchmark-disable), with the fix:2 failed, 1020 passed, 2 skipped, 15 xfailed. The two failures are pre-existing on this machine and unrelated:They are a
Python 3.11.0artifact:@dataclass(slots=True)creates a second class andPython 3.11.0keeps the discarded one alive, soParentDC.__subclasses__()returns it twice andcattrs.subclasses.subclasses()(which filters those duplicates only on 3.13+) sees two identical children.gc.collect()frees it on 3.11.14/3.12.12/3.14.2, where these tests pass; the project's CI is green on 3.11. They fail identically before this change.Lint (the project's
just lint):Notes / limitations
mypy --strictonsrc/cattrs/converters.pyreports 109 errors both before and after the change (the file is not mypy-clean in that ad-hoc configuration; the project only runs Mypy/Pyrefly throughpytest-typingon CPython 3.14). No new diagnostics are introduced; the pre-existingSignature of "copy" incompatible with supertype "BaseConverter"note is unchanged.pytest-typingsuites (tests/test_typing_structure.md), which require CPython 3.14 pluspytest-typing; and the PyPy 3.10 job.AI assistance
This change was developed with the help of an LLM-based coding assistant (DeepSeek Harness), used for repository exploration, reproduction and drafting. Per the project's generative AI policy there are no LLM co-author tags anywhere in the commit, the commit is authored and signed off by the human submitter, and the submitter has verified the reproduction, the fix and the test results and takes full responsibility for every line of it.