test: react_using_mellea.py example fails intermittently — ReACT loop exceeds 10 iteration budget

[planetf1]

docs/examples/agents/react/react_using_mellea.py fails when the model does not converge on a final_answer within the default loop budget:

RuntimeError: could not complete react loop in 10 iterations

To reproduce:

uv sync --all-extras --all-groups
uv run pytest --durations=0 --ignore=test/formatters/granite/test_intrinsics_formatters.py -v

Environment:

• MacBook M1 (Apple Silicon, macOS)
• Python 3.12.8
• Backend: ollama, model: granite4:micro

Observed behaviour: The model repeatedly calls duckduckgo_results_json but never calls final_answer, exhausting all 10 iterations. This is non-deterministic — it depends on model output.

Current markers: # pytest: ollama, llm

Suggested fix: Add qualitative and slow markers so the example is skipped by default locally and in CI, consistent with other non-deterministic examples:

# pytest: ollama, llm, qualitative, slow

[planetf1]

We may also be able to improve the test code so it's more likely to converge

[planetf1]

Analysis of the root cause

The issue is a combination of test design and a prompt weakness — 10 iterations is plenty for a working ReACT agent; the problem is the model never converging.

Prompt weakness

The "if you catch yourself repeating the same thing, call final_answer" hint is only in ReactInitiator.jinja2 (the setup message). ReactThought.jinja2, which is rendered on every turn, only says:

"If you are done and have a final answer, call the final_answer tool."

There's no per-turn signal that the model is looping or any urgency as iterations tick up. A model may not retain the initiator instruction after several tool-call results fill its context, so reinforcing this each turn would help.

Adding something like the following to ReactThought.jinja2 would help small and large models alike:

"If you already have enough information from previous observations, do not search again — call final_answer now."

Test design problems

1. Live DuckDuckGo — non-deterministic, rate-limited, returns variable result formats. A model may not parse them reliably or recognise when it has "enough".
2. Open-ended goal — "What is the Mellea python library?" has no clear stopping point. The model doesn't know when it has gathered sufficient information. A closed-domain question (e.g. one whose answer is deterministically in the first search result) would converge more reliably.

Suggested path forward

• Short term: add the qualitative and slow markers as suggested — this is the right pragmatic fix for CI/local runs.
• Medium term: improve ReactThought.jinja2 to reinforce the stopping condition on every turn.
• Longer term: consider a version of the example (or a separate unit test) that uses a mock/stub tool so the ReACT loop logic can be tested deterministically without a live network dependency.

[markstur]

I can work on this. I was reproducing it and I think I have a couple possible bug fixes related to it (investigating).

[markstur]

I tried tweaking the jinja and wasn't getting better results

• I tried tweaking the jinja and wasn't getting better results
• I tried make the query more deterministic and got better results but eventually fell into same loop problem
• I fixed the markers as requested in the issue in PR fix: Add qualitative and slow markers so the example is skipped #764
• I found that for me to stop the intermittent looping problem, I had to add an elif condition to find results that were already there (see fix: react loop is missing a check for result without tool calls #762). I think that is the real root problem (and suspect it might be DDGS just hitting a rate limit and changing behavior but I haven't found evidence)

I'd close this after those 2 are merged, but since I punted on jinja and query re-write I'm not sure