Reduce import memory: load the *Dict TypedDicts in types.py lazily (~2.7 MiB, ~15% of the SDK's own import cost)

[rafaelborja]

DRAFT — googleapis/python-genai issue (not posted)

Before posting (Rafael, 2026-09-25): re-check the latest google-genai on PyPI (2.25.0 today), re-apply
py315-opt/u8b_patch.py to it and re-test on the HAOS VM; replace the table below with the VM numbers
for that version. Upstream takes issues only (no external PRs, Discussions disabled).

Title: Reduce import memory: load the *Dict TypedDicts in types.py lazily (~2.7 MiB, ~15% of the SDK's own import cost)

Body:

Thanks for the recent import-cost work (lazy interactions, _gaos behind TYPE_CHECKING, defer_build=True in 2.18.0). It cut RSS roughly in half for us: Home Assistant's Gemini integration runs on small devices such as a Raspberry Pi 4 (1–2 GB RAM) or a phone-hosted VM where every megabyte counts. On a Raspberry Pi 4 with 1 GB RAM, the 2.7 MiB saved by this change represents roughly 0.3% of total system memory — significant when dozens of integrations compete for the same pool, and the difference between the system running smoothly and hitting swap.

After 2.18, most of what's left of the SDK's own import cost is types.py (863 KB of source). About a third of that file is 457 TypedDict classes (*Dict) plus ~470 *OrDict / *UnionDict aliases built from them. They exist for type hints only: no pydantic field uses them, and the SDK only reads them in annotations, in error messages, and in the Live API.

Measurement (2.25.0, CPython 3.14, RSS of a fresh process, shared deps preloaded):

	stock	TypedDicts lazy
import google.genai	18.2 MiB	15.7 MiB
import + Client + chats.create with a typical GenerateContentConfig	25.2 MiB	22.5 MiB

Real-world impact: In Home Assistant on a Raspberry Pi 4, the Gemini integration is one of ~50–100 integrations loaded at startup. The full import chain (google-genai + its transitive deps) currently costs ~32 MiB RSS. This patch brings the SDK's own slice of that from ~18 MiB to ~15 MiB, which on a 1 GB device frees enough headroom to avoid OOM-induced restarts during peak load.

What we tried (as a build-time patch on our side): move the *Dict classes and the aliases that depend on them into a private module (_types_dicts.py, same source text), and load it from the existing module-level __getattr__ in types.py on first access. types.ContentDict and from google.genai.types import ContentDict keep working. A small set stays eager: the names other SDK modules import at load time (HttpOptionsDict, HttpOptionsOrDict, ContentDict, ContentOrDict, GenerateContentConfigOrDict, PartUnionDict) and everything their definitions and annotations reference, about 100 names.

Risks and mitigations:

1. Behavioral equivalence: All 1381 public names in google.genai.types are verified identical to stock — same kind, fields, docstrings, __module__, get_type_hints() outcome, and dir(). The bundled test suite gives identical per-test outcomes with the patch applied.

2. __module__ attribute: Each moved TypedDict has its __module__ set back to google.genai.types after loading, so pickle, inspect, and tooling see the same origin as before. Code doing from google.genai.types import SomeDict continues to work.

3. First-access latency: The lazy module loads on first use of any deferred name. This is a one-time cost (~5 ms measured) that shifts from import time to first API call. After that, all names are cached in the module globals — subsequent access is a normal attribute lookup with no overhead.

4. Backward compatibility on Python < 3.14: The patch relies on PEP 649 (3.14+) for the savings: function annotations such as config: Optional[types.GenerateContentConfigOrDict] in models.py are no longer evaluated at definition time. On 3.10–3.13 those annotations would trigger the load at import, which is still correct but saves nothing, unless the SDK's modules adopt from __future__ import annotations.

5. Generated code: Since types.py is generated, the natural place for this is the generator. The current patch is a build-time transform that could be integrated into the code generation pipeline.

Proof of concept: We have a working implementation at rafaelborja/python-genai@main...lazy-load-typedicts — would it be appropriate to open a pull request, or would you prefer to integrate this into the code generator on your side?

A further step would be splitting the pydantic models themselves. A typical text + tools + thinking caller reaches only 137 of the 465 models, so that could save roughly another 5 MiB.

Happy to share the measurement scripts and help with testing.