Epic: ModelOutputThunk structural cleanup

[ajbozarth]

Status note: description rewritten on 2026-06-05 to reflect the post-#1197 design. The original April 2026 description is preserved in this issue's edit history. Discussion and design rationale: #1197.

Goal

Reshape ModelOutputThunk's internal layout to make the public surface explicit, the private surface coherent, and copy semantics structural rather than per-field.

Areas

#	Area	Issue	Status
1	Raw provider responses → typed mot.raw namespace	#1215	not started
2	telemetry_span / OTel ownership	#1181	approved, awaiting merge — closes Area 2 on merge
3	Computation logistics → _call / _gen sub-objects	#1216	not started
4	_thinking → public mot.thinking	#1217	not started
5	Public error / generate_log surface	#1191	folded in; design captured in #1191 comment

TL;DR

• Area 1 (raw responses) — move backend _meta[...] keys (chat_response, oai_chat_response, etc.) onto a typed mot.raw: RawProviderResponse namespace. Migrate chat.py:_parse to switch on mot.raw.provider.
• Area 2 (telemetry_span) — resolved by refactor(telemetry)!: move backend tracing onto plugin/hook pattern #1181 on merge; drops out of scope.
• Area 3 (computation logistics) — split the seventeen private attrs into two internal sub-objects: mot._call (originating call, preserved across copies) and mot._gen (in-flight machinery, reset on copy). Add a guard that raises on copy-of-uncomputed-MOT.
• Area 4 (_thinking) — promote to public mot.thinking with a property alias for _thinking through one deprecation cycle.
• Area 5 (Soft-fail errors on ModelOutputThunk are only reachable via a private attribute #1191) — give MOT a public surface for errors and the generate log: expose mot.generate_log as a public property, and add a public mot.error channel so callers can detect a soft-failed thunk without reading private state.

Sequencing

1. refactor(telemetry)!: move backend tracing onto plugin/hook pattern #1181 lands first. Independent, already approved. Closes Area 2 and adds _generation_id to Area 3's field set.
2. Area 5 (Soft-fail errors on ModelOutputThunk are only reachable via a private attribute #1191). Smallest. Closes Soft-fail errors on ModelOutputThunk are only reachable via a private attribute #1191.
3. Area 1 (refactor(core): consolidate backend raw responses onto mot.raw namespace #1215). No dependency on Areas 3/4/5.
4. Area 4 (refactor(core): promote mot._thinking to public mot.thinking #1217). Independent. Same release as Area 1 is fine.
5. Area 3 (refactor(core): split MOT internal state into _call / _gen sub-objects #1216). Largest internal rename. Lands last.

Coordinated, not in scope

• feat: Phase 2 — validated streaming #1013 (Phase 2 streaming chunking) — Area 3's _gen shape will accommodate parsed-repr stream buffers; do not block on feat: Phase 2 — validated streaming #1013, which could remain a placeholder for months.
• Standardize logits in MoT #123 (logits) — when implemented, logits go in mot.generation (e.g. generation.logprobs), not mot.raw. Per ModelOutputThunk field refactor: partition fields into sub-structures #793 precedent.
• ModelOutputThunk should not be a CBlock #269 (MOT-not-CBlock) — orthogonal. If acted on later, Area 1's mot.raw is unaffected.

Related bugs (discovered during audit, filed separately)

• fix(backends): unify raw-path token usage on mot.generation.usage; guard eval_count=None #1218 — fix(backends): unify raw-path token usage on mot.generation.usage; guard eval_count=None
• fix(streaming): use public mot.cancelled property in streaming.py:437 #1219 — fix(streaming): use public mot.cancelled property in streaming.py:437
• Better handling of reasoning output from thinking models #1201 — Message has no thinking field; reasoning silently dropped in multi-turn chat (coordinated with Area 4)

Cross-cutting constraints

• _meta is CBlock surface, not just MOT surface. This epic does not change CBlock._meta; it only removes MOT's writes into the dict.
• Public API is wider than field names suggest. mot._thinking, mot._meta[<keys>], and mot._generate_log are read by code outside base.py today; renames need property aliases for one minor release.
• Streaming validation locks in current shape: cancel_generation(), mot._async_queue single-consumer semantics, and mot.cancelled are load-bearing for stream_with_chunking() (feat(stdlib): add stream_with_chunking() with per-chunk validation (#901) #942).
• Mellea is pre-1.0 — DeprecationWarning aliases for one minor cycle, then remove.

[ajbozarth]

@jakelorocco I'm unsure if this fits into the deliverable for 0.5 or not, if you think I should tackle it this sprint I'll assign it to myself and add it to the iteration. Otherwise this will just sit until other work is done

[psschwei]

I'm not sure if the areas that would be changed under this (or #793 for that matter) would overlap with changes coming out of #891 , but in the case that there were overlaps, I would think that #891 is a higher priority to land first.

[ajbozarth]

Flagging design tension with #891 before this gets assigned.

Three specific collision points:

1. _async_queue / computation logistics fields
Sub-issue #901 (stream_with_chunking()) is explicitly built around the single-consumer constraint on _async_queue, including cancellation behavior. If #909 restructures those internal fields first, the stream_with_chunking() implementation will need to adapt to whatever new shape they land in — doing them in the wrong order creates rework.

2. telemetry_span in ._meta
If #909 relocates telemetry_span, it intersects with an open concern from the #891 review: when does GENERATION_POST_CALL fire in the streaming path, and how does TokenMetricsPlugin see it? The stream_with_chunking() path bypasses BaseSamplingStrategy entirely, so where the span lives affects whether that concern is resolved cleanly or left dangling.

3. Phase 2 of #891 vs. logistics grouping
Epic #891 scopes a second phase where the MOT owns chunking via a new stream_parsed_repr method, driven by the need to relax the single-consumer constraint. Issue #909's "computation logistics" grouping touches the same internals. If #909 lands first without considering that shape, it may make the Phase 2 MOT API harder to design.

Recommendation: hold #909 until #891's core sub-issues (#898–#901) are landed. Once stream_with_chunking() is stable, the scope of #909's logistics grouping will be much clearer — and some of it may naturally align with or be superseded by the Phase 2 design.

[psschwei]

Moving this out of ready based on above.
Backlog probably isn't the right status (maybe we should have something like "Hold" or "Blocked", but that's a different discussion)

[ajbozarth]

Moved this into next sprint as it is still blocked by #942 which even if it lands soon I probably won't have bandwidth to get to this by end of sprint. Should still get to it this release cycle though