fix: Update MultiTurnStrategy to include validation failure reasons in repair messages#633
Merged
jakelorocco merged 2 commits intoMar 13, 2026
Conversation
append validation failure reasons to the repair message for MultiTurnStrategy Fixes generative-computing#631 Signed-off-by: va <va@us.ibm.com>
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
7 tasks
jakelorocco
approved these changes
Mar 13, 2026
jakelorocco
left a comment
Contributor
There was a problem hiding this comment.
@vabarbosa, thank you for the PR. I pushed one change to correct the docstring for the sampling file since the rejection strat doesn't rely on validation_result.reason. Everything else looks good so I'll approve and it'll merge after the workflows pass.
Thank you!
jakelorocco
enabled auto-merge
March 13, 2026 12:56
Merged
via the queue into
generative-computing:main
with commit Mar 13, 2026
ebdd092
4 checks passed
8 tasks
akihikokuroda
pushed a commit
to akihikokuroda/mellea
that referenced
this pull request
May 27, 2026
…n repair messages (generative-computing#633) * fix: update multiturnstrategy repair message append validation failure reasons to the repair message for MultiTurnStrategy Fixes generative-computing#631 Signed-off-by: va <va@us.ibm.com> * docs: update docstring to be correct --------- Signed-off-by: va <va@us.ibm.com> Co-authored-by: jakelorocco <59755218+jakelorocco@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR fixes
MultiTurnStrategyto include detailed validation failure reasons fromValidationResult.reasonin repair messages. The strategy now provides the model with actionable feedback about what went wrong (similar toRepairTemplateStrategy), significantly improving convergence rates. In addition, documentation updates were made and an example added.Changes
Core Fix (
mellea/stdlib/sampling/base.py):MultiTurnStrategy.repair()to includeValidationResult.reasonin user messagesDocumentation (
docs/examples/instruct_validate_repair/README.md):Example (
docs/examples/instruct_validate_repair/multiturn_strategy_example.py):MultiTurnStrategywith validation functionsImpact
MultiTurnStrategynow converges effectively with validation functionsTesting
The new example can be run as a test:
Related Issues
Fixes #631
Checklist
uv run pytest -m "not qualitative")uv run ruff format .)uv run ruff check .)uv run mypy .)