Skip to content

Fix output type loss in AgentOperator's human-in-the-loop review#70132

Open
ColtenOuO wants to merge 1 commit into
apache:mainfrom
ColtenOuO:fix-hitl-review-output-type-loss
Open

Fix output type loss in AgentOperator's human-in-the-loop review#70132
ColtenOuO wants to merge 1 commit into
apache:mainfrom
ColtenOuO:fix-hitl-review-output-type-loss

Conversation

@ColtenOuO

Copy link
Copy Markdown
Contributor

Summary

AgentOperator's human-in-the-loop review path (enable_hitl_review=True) loses the original output type for any output_type that is neither str nor a Pydantic BaseModel (e.g. list[str], bool, dict).

The output is round-tripped through a string while it's shown to a human reviewer: HITLReviewMixin._to_string serializes it before review, and AgentOperator.execute() deserializes it back into output_type after approval. The serialization side used str(output), which produces a Python repr (e.g. "['tag-a', 'tag-b']"), not valid JSON.

The deserialization side then tried json.loads() on that repr, which always raised, and silently fell back to returning the raw repr string instead of the original list/bool/dict — a type change downstream tasks don't expect.

regenerate_with_feedback() (used when a reviewer requests changes) had the identical str(output) bug.

This is the same defect class already fixed for the require_approval=True path in #70075, which switched to TypeAdapter(...).dump_json(...). That fix wasn't applied to the separate enable_hitl_review=True path, so it regressed here.

Changes

  • HITLReviewMixin._to_string and AgentOperator.regenerate_with_feedback
    now serialize non-str/non-BaseModel output with
    TypeAdapter(type(output)).dump_json(output), matching the pattern
    already used by LLMApprovalMixin.defer_for_approval.
  • AgentOperator.execute()'s HITL-review branch now reuses the existing
    rehydrate_pydantic_output helper (already used by the approval path)
    instead of a bespoke json.loads/except fallback, so both review paths
    behave consistently.

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Code (Sonnet 5)

Generated-by: Claude Code (Sonnet 5) following the guidelines

When enable_hitl_review=True and output_type is a non-str, non-BaseModel
type (e.g. list[str], bool), the reviewed output was serialized with
str(output), producing a Python repr rather than valid JSON. Rehydrating
it back into output_type then silently failed and returned the raw repr
string instead of the original value. Serialize with TypeAdapter instead,
and reuse the existing rehydrate_pydantic_output helper (already used by
the require_approval path) instead of a separate, broken json.loads
fallback.
@ColtenOuO
ColtenOuO force-pushed the fix-hitl-review-output-type-loss branch from dec8555 to 5c61485 Compare July 20, 2026 14:35
return json.loads(result_str)
except (ValueError, TypeError):
return result_str
return rehydrate_pydantic_output(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropping the json.loads fallback here regresses the exact case this PR targets. rehydrate_pydantic_output returns raw unchanged when output_type isn't a BaseModel subclass (its docstring notes the caller is expected to apply its own json.loads), so with output_type=list[str] this returns the JSON string '["tag-a","tag-b"]' instead of the list. The new test_execute_with_hitl_rehydrates_non_base_model_output fails on exactly this (assert '["tag-a", "tag-b"]' == ['tag-a', 'tag-b']). Keeping the previous json.loads/except fallback after the rehydrate call fixes it while preserving the BaseModel handling.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In PR #70075 (merged), rehydrate_pydantic_output was updated to handle any non-str output_type, not just BaseModel subclasses:

if output_type is str:
    return raw
try:
    rehydrated = TypeAdapter(output_type).validate_json(raw)
except (ValidationError, ValueError, TypeError):
    return raw
uv run --project providers/common/ai python -c "tic_output
from airflow.providers.common.ai.utils.output_type import rehydrate_pydantic_outputtput=False)
result = rehydrate_pydantic_output(list[str], '[\"tag-a\",\"tag-b\"]', serialize_output=False)
print(repr(result), type(result))
"

Output:

['tag-a', 'tag-b'] <class 'list'>

So I don't think the json.loads fallback needs to come back (?

Let me know what you think, or if I might have missed anything!

Thanks for your review!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants