Skip to content

feat(core): add stream_validate() to Requirement #900

Description

@planetf1

What

Add stream_validate() to the Requirement class in mellea/core/requirement.py (class at line 104):

async def stream_validate(
    self, backend, ctx, chunk: str
) -> PartialValidationResult:
    return PartialValidationResult(success="unknown")

Why

validate() (line 143) operates on complete output. Streaming validation needs per-chunk processing with different semantics: incremental input, tri-state result, and stateful accumulation across calls within a single attempt.

The method is named stream_validate, not avalidate. In mellea, the a-prefix means "async version of the same operation" — this is a different operation and should be clearly distinct.

In Phase 1, requirements receive chunks as plain strings passed by the streaming orchestrator. They never touch the MOT stream directly. Stateful requirements maintain their own incremental state across stream_validate calls (e.g. a running counter, an incremental parser). The streaming orchestrator clones each requirement (copy(req)) before every attempt, so streaming state is isolated to the clone and the original is never mutated.

The default is safe for all existing requirements. ALoraRequirement and LLMaJRequirement need no changes — they return "unknown" on every chunk and receive the full output via validate() at stream end.

Returning "pass" from stream_validate() does not short-circuit final validation in Phase 1. The orchestrator always calls validate() at stream end for any requirement that did not return "fail""pass" mid-stream is informational only.

Acceptance criteria

  • stream_validate(self, backend, ctx, chunk: str) -> PartialValidationResult exists on Requirement with the default returning PartialValidationResult(success="unknown")
  • ALoraRequirement and LLMaJRequirement work correctly with the default — no override required
  • Unit tests in test/core/:
    • Default stream_validate returns "unknown" for any chunk
    • A stateful subclass (e.g. counts bullet points) accumulates state correctly across multiple stream_validate calls
    • The same subclass starts from a clean state on a fresh clone — confirms orchestrator clone pattern works
  • Google-style docstring on stream_validate() stating: (a) default returns "unknown"; (b) returning "pass" is informational in Phase 1 and does not skip final validate(); (c) implementations may accumulate state on self — the orchestrator clones the requirement before each attempt so state does not bleed across retries; (d) mutable container fields should be reassigned not mutated in place to be safe under shallow copy

Blocked by #898
Part of #891

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions