Skip to content

[python] add typeddict models-mode for Python HTTP client emitter - #10439

Merged
msyyc merged 82 commits into
microsoft:mainfrom
iscai-msft:python/addTypedDict
Jun 30, 2026
Merged

[python] add typeddict models-mode for Python HTTP client emitter#10439
msyyc merged 82 commits into
microsoft:mainfrom
iscai-msft:python/addTypedDict

Conversation

@iscai-msft

@iscai-msft iscai-msft commented Apr 21, 2026

Copy link
Copy Markdown
Member

fixes #8800

Add a new 'typeddict' value for the models-mode option that generates Python TypedDict classes instead of DPG model classes. Key features:

  • TypedDict classes with Required[T]/NotRequired[T] annotations
  • TypedDict inheritance for non-discriminated models
  • Discriminated models: Union of leaf TypedDicts, no abstract base class
  • Input-only: operations accept TypedDict input, return dict output
  • Wire names used as TypedDict keys
  • _model_base.py still generated for serialization utilities

Add a new 'typeddict' value for the models-mode option that generates
Python TypedDict classes instead of DPG model classes. Key features:

- TypedDict classes with Required[T]/NotRequired[T] annotations
- TypedDict inheritance for non-discriminated models
- Discriminated models: Union of leaf TypedDicts, no abstract base class
- Input-only: operations accept TypedDict input, return dict output
- Wire names used as TypedDict keys
- _model_base.py still generated for serialization utilities

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@microsoft-github-policy-service microsoft-github-policy-service Bot added the emitter:client:python Issue for the Python client emitter: @typespec/http-client-python label Apr 21, 2026
@pkg-pr-new

pkg-pr-new Bot commented Apr 21, 2026

Copy link
Copy Markdown

Open in StackBlitz

npm i https://pkg.pr.new/@typespec/http-client-python@10439

commit: 5777d0a

@github-actions

github-actions Bot commented Apr 21, 2026

Copy link
Copy Markdown
Contributor

All changed packages have been documented.

  • @typespec/http-client-python
Show changes

@typespec/http-client-python - fix ✏️

Fix invalid lone @overload generated for body parameters in models-mode: typeddict. When the binary and JSON overloads are omitted, the single remaining body variant is now emitted as a plain parameter instead of a single @overload, which mypy rejects with "Single overload definition, multiple required".

@typespec/http-client-python - feature ✏️

[python] Always generate TypedDict typing hints for input models in the types.py file, and named union aliases in the _unions.py file

@azure-sdk

azure-sdk commented Apr 21, 2026

Copy link
Copy Markdown
Collaborator

You can try these changes here

🛝 Playground 🌐 Website 🛝 VSCode Extension

iscai-msft and others added 12 commits April 21, 2026 14:37
- TypedDictModelType returns 'JSON' for response type annotations
- Response.type_annotation/docstring passes is_response=True
- Typeddict deserialization uses response.json() directly
- Removed NotRequired from TypedDictModelSerializer (total=False handles it)
- Updated mock API tests to verify JSON dict responses

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add client/naming typeddict variant to regenerate-common.ts
- Create test_client_naming_typeddict.py with 11 tests verifying TypedDict
  uses wire names (defaultName, wireName) not client names
- Tests cover: ClientNameModel, LanguageClientNameModel,
  ClientNameAndJsonEncodedNameModel, ClientModel, PythonModel

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDict is already JSON, so the MutableMapping[str, Any] overload
is unnecessary. Only keep TypedDict model + IO[bytes] overloads.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Typeddict mode uses response.json() directly, so _deserialize is never
called. Skip importing it to avoid W0611 unused-import lint warning.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Remove unused MutableMapping/Any imports from TypedDictModelType.imports()
- Skip _deserialize import in paging_operation.py for typeddict mode

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
TypedDictModelType returns 'JSON' for response type annotations but
never defined the JSON = MutableMapping[str, Any] type alias, causing
NameError at runtime.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@iscai-msft
iscai-msft force-pushed the python/addTypedDict branch from 34e0cda to 95db199 Compare April 29, 2026 15:39
iscai-msft and others added 3 commits June 24, 2026 16:31
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Replace is_operation_file with serialize_namespace_type to correctly
determine when to add the typing.List import. Operation/client files
only need it when has_operation_named_list is True, while model/types
files need it when has_property_named_list is True.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e conflicts

Remove the 'from typing import List' import entirely from ListType.
Types/model files now always use lowercase 'list' since a TypedDict
field named 'list' doesn't shadow the builtin. Operation files still
use the 'List = list' alias from define_mypy_type when there's an
operation named 'list'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

@msyyc msyyc left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will take another review soon.

@msyyc
msyyc self-requested a review June 25, 2026 00:18
The response type rendering passes is_operation_file=True but not
serialize_namespace_type, so we need to check both to correctly
determine operation context.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@msyyc
msyyc self-requested a review June 25, 2026 00:19
iscai-msft and others added 5 commits June 25, 2026 12:56
For models that are typeddict-only (either via models-mode=typeddict or
individual models marked as typedDictOnly), skip the isinstance check
against _models.X (which doesn't exist) and skip json.dumps
serialization. TypedDict models are plain dicts and should be passed
directly to the request as JSON.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Add tests passing TypedDicts to regular dpg operations (both sync and
  async) to verify model + typeddict combined overloads work
- Add async test for typeddictonly variant

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Comment thread packages/http-client-python/eng/scripts/ci/regenerate.ts
Comment thread packages/http-client-python/eng/scripts/ci/regenerate.ts Outdated
Comment thread packages/http-client-python/tests/mock_api/shared/unittests/test_readme.py Outdated
l0lawrence and others added 2 commits June 26, 2026 10:54
…rate-common.ts

Keep regenerate-common.ts as the synced, shared module instead of duplicating its helpers inline in regenerate.ts. Add the typeddict-specific emitter options and baseline changes there, and remove the now-supported response-as-bool skip entries.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
In models-mode: typeddict the binary/JSON overloads are omitted, leaving a single body variant that was still wrapped in a lone @overload (mypy: Single overload definition, multiple required). Collapse the one-member combined type back to a plain body type so no overload is emitted.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@l0lawrence

l0lawrence commented Jun 26, 2026

Copy link
Copy Markdown
Member

running into a mypy error because of: https://github.com/Azure/azure-sdk-for-python/pull/47346/changes#r3483777805

Comment thread packages/http-client-python/eng/scripts/ci/regenerate-common.ts Outdated
Comment thread packages/http-client-python/eng/scripts/ci/regenerate-common.ts
- Remove service/multiple-services from SKIP_SPECS (supported in main)
- Restore by-design legacyCodePathNeededForTests baseline copy instead of whole-folder copy

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@msyyc
msyyc added this pull request to the merge queue Jun 30, 2026
Merged via the queue into microsoft:main with commit 30d60f8 Jun 30, 2026
41 checks passed
live1206 pushed a commit to live1206/typespec that referenced this pull request Jul 7, 2026
…crosoft#11161)

fix microsoft#11149

## Why

PR microsoft#10439 enabled TypedDict body-overload generation in the default
`dpg` mode. In real Azure mgmt libraries (e.g. `azure-mgmt-netapp`),
this produced dangling `_types.CacheUpdate` references that were absent
from `types.py`, causing an `AttributeError` at import time.

## Root cause

Template-instantiated models all share the *template's*
`crossLanguageDefinitionId`. For example, every
`Azure.ResourceManager.Foundations.ResourceUpdateModel<T, P>`
instantiation (`CacheUpdate`, `VolumeUpdate`,
`ActiveDirectoryConfigUpdate`, ...) carries `clid =
Azure.ResourceManager.Foundations.ResourceUpdateModel`.

The `typeddict_models` dedup keyed the dpg-vs-copy pairing on that
`clid`, so it treated genuinely distinct models as duplicates and
skipped all but the first one. In netapp, `ActiveDirectoryConfigUpdate`
was seen first, so `CacheUpdate` was dropped from `types.py` while still
being referenced via `_types.CacheUpdate`.

## Fix

- Key the dpg-vs-copy pairing on the model `name` instead of
`crossLanguageDefinitionId`. The typeddict copy is a shallow copy of the
source, so it shares the source's name; distinct template instantiations
have distinct names, and model names are unique within a `types.py`
module.
- Harden `_find_existing_typeddict` in preprocess to also match on
`name` (same root cause, tightly coupled).

This keeps TypedDict generation on in `dpg` mode (intentional per
microsoft#10439) and just corrects the rendering so referenced TypedDicts
actually land in `types.py`.

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
tadelesh pushed a commit that referenced this pull request Jul 9, 2026
…1161)

fix #11149

## Why

PR #10439 enabled TypedDict body-overload generation in the default
`dpg` mode. In real Azure mgmt libraries (e.g. `azure-mgmt-netapp`),
this produced dangling `_types.CacheUpdate` references that were absent
from `types.py`, causing an `AttributeError` at import time.

## Root cause

Template-instantiated models all share the *template's*
`crossLanguageDefinitionId`. For example, every
`Azure.ResourceManager.Foundations.ResourceUpdateModel<T, P>`
instantiation (`CacheUpdate`, `VolumeUpdate`,
`ActiveDirectoryConfigUpdate`, ...) carries `clid =
Azure.ResourceManager.Foundations.ResourceUpdateModel`.

The `typeddict_models` dedup keyed the dpg-vs-copy pairing on that
`clid`, so it treated genuinely distinct models as duplicates and
skipped all but the first one. In netapp, `ActiveDirectoryConfigUpdate`
was seen first, so `CacheUpdate` was dropped from `types.py` while still
being referenced via `_types.CacheUpdate`.

## Fix

- Key the dpg-vs-copy pairing on the model `name` instead of
`crossLanguageDefinitionId`. The typeddict copy is a shallow copy of the
source, so it shares the source's name; distinct template instantiations
have distinct names, and model names are unique within a `types.py`
module.
- Harden `_find_existing_typeddict` in preprocess to also match on
`name` (same root cause, tightly coupled).

This keeps TypedDict generation on in `dpg` mode (intentional per
#10439) and just corrects the rendering so referenced TypedDicts
actually land in `types.py`.

---------

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Yuchao Yan <yuchaoyan@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

emitter:client:python Issue for the Python client emitter: @typespec/http-client-python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Python] Add alpha TypedDict support

6 participants