refactor(intrinsics): migrate rag.py to new Adapter types (Epic #929 Phase 1 Wave 3) by planetf1 · Pull Request #1321 · generative-computing/mellea

planetf1 · 2026-06-23T16:44:00Z

Summary

This is Epic #929 Phase 1 Wave 3 — whole-file migration of mellea/stdlib/components/intrinsic/rag.py to the new Adapter(identity, io_contract, weights) dataclass system introduced in Phase 1.A (PR #1269, merged 2026-06-23).

Epic #929 is a multi-phase effort to fix Adapter Function Lifecycle & Consistency in Mellea. The phases are:

Phase 0 (feat(intrinsics): introduce Adapter/Identity/IOContract/WeightsBinding scaffolding (Epic #929 Phase 0) #1134) — scaffolding: Adapter dataclass, IOContract ABC, Identity, WeightsBinding types
Phase 1.A (refactor(intrinsics): internal migration with shims — old classes inherit from Adapter; call_intrinsic rewritten (Epic #929 Phase 1) #1136) — call_intrinsic refactor + _ShimIOContract shims; merged ✅
Phase 1 Wave 3 (this PR, refactor(intrinsics): rag.py whole-file migration to new Adapter types (Epic #929 Phase 1) #1137) — full IOContract migration for all 6 RAG adapter functions
Phase 2 (refactor(intrinsics): requirement_check migration — loud schema-mismatch, no more silent False (Epic #929 Phase 1) #1138) — build_prompt implementation (currently stubs raising NotImplementedError)

What this PR changes

rag.py — all six RAG adapter functions fully migrated:

Two new IOContract subclasses: _DictContract (single-dict output) and _ListContract (list-of-dicts output).
_ListContract.parse() wraps the validated list in {"items": [...]} to satisfy the dict[str, object] return type of IOContract.parse(); helper functions unwrap it before returning — public return types are unchanged.
parse() raises AdapterSchemaMismatchError when a required field is missing; raises ValueError when the JSON output is the wrong type entirely (not a dict / not a list). This replaces the previous silent KeyError; callers that catch KeyError should update to AdapterSchemaMismatchError.
Extra fields in the model output do NOT raise (forward-compatible).
parse() propagates ValueError / json.JSONDecodeError from json.loads when the model output is not valid JSON.
Six module-level Adapter constants, one per helper, each with Identity, _*Contract, and LocalFileBinding().
model_options: dict | None = None added as a keyword-only argument to all six helpers (previously absent).
Prose updated to agreed Granite Switch glossary (issue chore(terminology): adopt Granite Switch / Mellea glossary across Epic #929 and repo #1192): "intrinsic" → "adapter function" in backend: parameter descriptions.
All six call sites use cast(str, ...) / cast(list[dict], ...) rather than # type: ignore[return-value].

_util.py — call_intrinsic extended with io_contract: IOContract | None = None (keyword-only, defaulted None). When provided, replaces the raw json.loads call with io_contract.parse() for combined parse+validate. All existing callers (core.py, guardian.py) pass no io_contract and are unaffected. The function now carries an explicit -> dict[str, object] return annotation.

test_rag_contracts.py (new) — 18 pure unit tests (no GPU, no backend):

Two per helper: contract_enforced (missing required field → AdapterSchemaMismatchError) and forward_compat (extra field → no raise)
Two empty-list edge cases for the list-shaped contracts (find_citations, flag_hallucinated_content)
Four type-mismatch cases: ValueError when JSON is not a dict, not a list, or contains a non-dict list element

Output contract verification

Each contract was verified against the current granitelib-rag-r1.0 weights README before writing the tests:

Helper	Required output keys
`check_answerability`	`answerability`
`rewrite_question`	`rewritten_question`
`clarify_query`	`clarification`
`find_citations`	`response_begin`, `response_end`, `response_text`, `citation_doc_id`, `citation_begin`, `citation_end`, `citation_text`
`check_context_relevance`	`context_relevance`
`flag_hallucinated_content`	`response_begin`, `response_end`, `response_text`, `faithfulness`, `explanation`

Examples review

All six docs/examples/intrinsics/ examples that call these helpers (answerability.py, citations.py, context_relevance.py, hallucination_detection.py, query_clarification.py, query_rewrite.py) were reviewed. All existing calls pass the migrated parameters positionally or by name, and none rely on a model_options position — the keyword-only addition is non-breaking. All examples are gated with # pytest: huggingface, e2e (or have no pytest marker and are silently skipped) and therefore do not run in unit CI.

Relationship to other open PRs

refactor(intrinsics): requirement_check_to_bool raises on schema mismatch (Epic #929 Phase 1) #1320 (requirement_check_to_bool, core.py — refactor(intrinsics): requirement_check migration — loud schema-mismatch, no more silent False (Epic #929 Phase 1) #1138): Wave 3 parallel sibling; touches different file, no conflict.
refactor(intrinsics): guardian.py whole-file migration + documents= kw-only discovery (#1139) #1323 (guardian.py — refactor(intrinsics): guardian.py whole-file migration + documents= kw-only + auto-context discovery (Epic #929 Phase 1) #1139): Wave 3 parallel sibling; touches different file, no conflict.
chore(terminology): sweep docs/examples and Python docstrings (#1192 follow-up) #1322 (docs/examples terminology sweep — chore(terminology): complete prose sweep in docs/examples/ and Python docstrings (#1192 follow-up) #1279): touches docs/examples/intrinsics/README.md and Python docstring files, but NOT rag.py or _util.py. No conflict.
feat(thunk+types): add .parsed property and wire format= overloads for cast-free structured output #1284 (.parsed property / format= overloads — bug: structured output via act() silently returns JSON string, not Pydantic instance #1273/chore: thread format= through act()/instruct() return type overloads so structured output is cast-free #1274): unrelated to this file.

Ordering: this PR has no blocking relationship to #1320 or #1323 — all three Wave 3 PRs are independent and can merge in any order.

Related issues

Closes refactor(intrinsics): rag.py whole-file migration to new Adapter types (Epic #929 Phase 1) #1137
Part of Epic Epic: Fix Adapter Function Lifecycle & Consistency in Mellea #929
Prerequisite: refactor(intrinsics): internal migration with shims — old classes inherit from Adapter; call_intrinsic rewritten (Epic #929 Phase 1) #1136 / PR feat(intrinsics): shim old adapters onto Adapter dataclass; call_intrinsic rewritten (Epic #929 Phase 1) #1269 (merged — Adapter, IOContract, LocalFileBinding API)
Next wave: refactor(intrinsics): requirement_check migration — loud schema-mismatch, no more silent False (Epic #929 Phase 1) #1138 (core.py, PR refactor(intrinsics): requirement_check_to_bool raises on schema mismatch (Epic #929 Phase 1) #1320), refactor(intrinsics): guardian.py whole-file migration + documents= kw-only + auto-context discovery (Epic #929 Phase 1) #1139 (guardian.py, PR refactor(intrinsics): guardian.py whole-file migration + documents= kw-only discovery (#1139) #1323)
Related open issues noted but out of scope: context_relevance aLoRA adapter: missing max_completion_tokens causes JSONDecodeError in test; consider deprecating #1301 (context_relevance JSONDecodeError — separate bug), out of range eror when calling intrinsics on empty contexts #937 (empty-context crash — separate bug), fix(rag): groundedness support prompt presents nested citations in input while requesting flat output #1316 (groundedness nested citations — different helper)

Test plan

uv run pytest test/stdlib/components/intrinsic/test_rag_contracts.py -v — 18/18 pass
uv run pytest test/stdlib/components/intrinsic/ -m "not qualitative" — all pass
uv run pytest test/ -m "not qualitative" (full local suite) — 1837 passed, 0 failures (1 pre-existing error: test_richdocument errors on Apple Silicon: docling layout pipeline fails on MPS (float64 unsupported) #1344)
uv run ruff format . && uv run ruff check . — clean
Docstring quality gate: 0 issues
docs/examples/intrinsics/ examples reviewed — backward-compatible, no changes required
GPU integration tests — BlueVela LSF (job 1691399, single A100): 274 passed, 0 failures
CI (code-checks 3.11/3.12/3.13) — all pass (3.13 rerun needed: httpx.ReadTimeout flake, tracked test: backend tracing tests flake with httpx.ReadTimeout on loaded CI runners #1318)

- Add model_options: dict | None = None keyword-only param to all four helpers (policy_guardrails, guardian_check, factuality_detection, factuality_correction); forwarded to call_intrinsic unchanged - Two unit tests verifying model_options forwarding - Fix example comments: ctx_no_doc -> ctx (phantom variable), clarify documents= as an alternative to the context-based approach Addresses AC gap: model_options= was missing from all four helpers despite being listed in the issue and present in sibling rag.py (generative-computing#1321). Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

…ative-computing#929 Phase 1) (generative-computing#1137) All six RAG helpers migrated to the new Adapter dataclass per the per-helper migration template in PR generative-computing#1080 §13: - Add _DictContract and _ListContract IOContract subclasses for all six helpers. parse() raises AdapterSchemaMismatchError on contract-breaking deltas (missing required field); forward-compatible additions (extra optional fields) do not raise. - Create six module-level Adapter constants, one per helper. - Add model_options: dict | None = None keyword argument to every helper. - Extend call_intrinsic in _util.py with an io_contract parameter; when provided, io_contract.parse(result_str) replaces json.loads for combined parse+validate. - find_citations and flag_hallucinated_content extract the validated list from the parse result["items"] wrapper key; public return types are unchanged. Tests: 12 new unit tests in test_rag_contracts.py — two per helper (contract_enforced / forward_compat) — run without GPU or backend. Closes generative-computing#1137 Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

- Add ValueError to Raises: sections in all six RAG helpers and call_intrinsic; json.JSONDecodeError propagates before schema checks. - Document that an empty list parses to {"items": []} in _ListContract. - Remove generative-computing#1138 issue refs from build_prompt NotImplementedError strings (project convention: no #NNN refs in inline code). - Tighten call_intrinsic one-line summary to say "optionally validated". - Add two empty-list edge-case tests (citations, hallucination) to test_rag_contracts.py; 14 tests pass. Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

…unction' in rag.py Five backend: docstring entries in clarify_query, find_citations (×2), check_context_relevance, and flag_hallucinated_content still used the old 'intrinsic' terminology. Aligned with the agreed Granite Switch glossary (issue generative-computing#1192 / PR generative-computing#1256); matches the prose already updated in upstream/main for the other helper files. Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

- Add `-> dict[str, object]` return annotation to `call_intrinsic` so callers have a concrete type to work against - Replace `# type: ignore[return-value]` at all six rag.py call sites with `cast(str, ...)` / `cast(list[dict], ...)` — documents the adapter contract rather than silencing the checker - Raise `ValueError` (not `AdapterSchemaMismatchError`) when parsed JSON is the wrong type entirely (not a dict / not a list); reserve `AdapterSchemaMismatchError` for missing-key violations where its observed/expected-keys vocabulary actually makes sense - Update `Raises:` docstrings in `_DictContract` and `_ListContract` to match Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

Four new unit tests verify the behaviour introduced in the preceding commit: dict-shaped contracts reject non-dicts, list-shaped contracts reject non-lists and non-dict list elements, and the error messages name the adapter. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

Rebasing onto upstream/main introduced a conflict at the import block; resolving it dropped `import warnings`, which is needed for the `check_context_relevance` deprecation warning added in upstream generative-computing#1339. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>

planetf1 · 2026-06-25T11:22:17Z

HPC results (job 1685548, rebased branch fbb206b7 on upstream/main):

Intrinsic tests — test/stdlib/components/intrinsic/, test/backends/test_adapters/, test/formatters/granite/test_intrinsics_formatters.py, docs/examples/intrinsics/:


Passed	273
Skipped	2
xfailed	2
Failed	0

All green on CUDA (A100, BlueVela) and locally (MPS, Apple Silicon).

…call_intrinsic + add mixed-list test Addresses two post-review suggestions from PR generative-computing#1321: - `call_intrinsic` Raises: section now documents that `AdapterSchemaMismatchError` is propagated when an `io_contract` is provided and the model output violates the declared contract. Keeps the docstring accurate ahead of the quality gate. - Add `test_list_contract_rejects_non_dict_element_after_valid_item` to cover the full loop body of `_ListContract.parse` (previously only a single-element list was tested; this adds a valid-then-invalid mixed list to exercise the per-item check on non-first elements). 19 tests pass. Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code

…object] return type Adding -> dict[str, object] to call_intrinsic in _util.py (this PR) made mypy correctly flag return sites in guardian.py and core.py where a subscript value (typed object) was being returned as str or float. Fixes CI quality gate failure. Apply the same cast() pattern already used in rag.py: - guardian.py: cast(str, ...) on policy_guardrails label/score returns; cast(float, cast(dict[str, object], ...)["score"]) on guardian_check; cast(str, ...) on factuality_detection and factuality_correction returns. - core.py: cast(float, ...) on check_certainty and requirement_check returns; cast(list[dict], ...) on find_context_attributions (adapter emits a JSON array). Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code

planetf1 · 2026-06-25T13:54:56Z

Testing

Local (macOS/MPS) — full non-qualitative suite:

uv run pytest test/ -m "not qualitative" -x --timeout=120 -q

Result: 1837 passed, 10 skipped, 1 error (pre-existing #1344 — MPS/docling float64), 0 failures.

BlueVela (LSF, GPU) — intrinsics scope (job 1691399):

uv run pytest test/stdlib/components/intrinsic/ test/backends/test_adapters/ \
    test/formatters/granite/test_intrinsics_formatters.py docs/examples/intrinsics/ -v

Result: 274 passed, 2 skipped, 2 xfailed, 0 failures. Run time: ~11 min on a single A100.

planetf1 · 2026-06-25T13:57:44Z

The code-checks / quality (3.13) failure is a pre-existing flaky test — test_ollama_generate_from_raw_metrics_integration hit an httpx.ReadTimeout on a loaded CI runner, tracked in #1318. Restarting that job.

…w-only discovery (generative-computing#1139) (generative-computing#1323) * refactor(intrinsics): guardian.py whole-file migration + documents= kw-only discovery (generative-computing#1139) Closes generative-computing#1139, closes generative-computing#1094 **`guardian.py` (whole-file migration)** - `policy_guardrails`: migrate `:param`/`:return:` reST docstring to Google-style `Args:`/`Returns:`; tighten `label xor score` guard to raise `ValueError` instead of bare `Exception`. - `factuality_detection` / `factuality_correction`: add `documents=` keyword-only argument (`Iterable[str | Document] | None = None`). When provided, the last assistant turn is extracted from the context and re-added as a `Message` with the supplied documents attached, matching the pattern established by `flag_hallucinated_content` in `rag.py`. Works for both session-generated contexts (last element is a `ModelOutputThunk`) and manually-constructed contexts (last element is a `Message`). Raises `ValueError` when `documents=` is provided but the context does not end with an assistant turn. - Add private helper `_inject_documents` to encapsulate the extraction + reconstruction logic. - Migrate docstrings for both functions to Google style; add `Raises:` sections. - New imports: `collections.abc`, `typing.cast`, `Document`, `_coerce_to_documents`, `Message`. **`creating_a_new_type_of_session.py` (closes generative-computing#1094)** - Remove deprecated `GuardianCheck` / `GuardianRisk` imports. - Replace `requirements=[GuardianCheck(...), ...]` + `self.validate()` with direct `guardian.guardian_check()` calls in `chat()`. - Switch backend from `OllamaModelBackend` to `LocalHFBackend(IBM_GRANITE_4_MICRO_3B)`, which carries the embedded guardian-core adapter; same model used by the existing guardian e2e tests. - Update pytest marker: `ollama` → `huggingface`; add `slow`. **Tests** - `test_guardian.py`: `_read_guardian_input` now returns `(ChatContext, list[Document])`, loading reference documents from `extra_body.documents` in the fixture JSON. Both factuality tests pass documents to the functions under test. - `test_guardian_documents.py` (new): 11 unit tests covering the no-documents path, the documents= injection path, correct intrinsic names, string-to-Document coercion, preservation of preceding turns, and all `ValueError` cases — no model required. Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code * fix(intrinsics): address code review findings on guardian documents= PR - Fix _inject_documents to raise a distinct ValueError when the last assistant turn exists but has not been computed yet (uncomputed thunk), rather than misleadingly saying 'not an assistant response' - Drop redundant _coerce_to_documents pre-call: Message.__init__ already coerces, so pass documents directly; removes private-API import - Remove spurious # type: ignore[assignment] on Message.content access - Clarify docstrings on factuality_detection and factuality_correction: reword 'in precedence order' to 'in any of these ways' and note that documents= replaces the last assistant turn's docs only - Add unit tests covering the ModelOutputThunk (session-generated) path and the uncomputed-thunk error case (4 new tests, 15 total) Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Assisted-by: Claude Code * test(intrinsics): add policy_guardrails XOR tests; narrow example backend type - Add two unit tests for policy_guardrails ValueError paths: both label+score present ("found both") and neither present ("found neither") - Narrow ChatCheckingSession.__init__ to backend: AdapterMixin; retain type: ignore[arg-type] at call site because MelleaSession.backend property is typed Backend Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> * fix(intrinsics): add model_options= to guardian helpers; update examples - Add model_options: dict | None = None keyword-only param to all four helpers (policy_guardrails, guardian_check, factuality_detection, factuality_correction); forwarded to call_intrinsic unchanged - Two unit tests verifying model_options forwarding - Fix example comments: ctx_no_doc -> ctx (phantom variable), clarify documents= as an alternative to the context-based approach Addresses AC gap: model_options= was missing from all four helpers despite being listed in the issue and present in sibling rag.py (generative-computing#1321). Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> * fix(examples): correct phantom ctx_no_doc variable in factuality example comments The documents= kwarg examples referenced ctx_no_doc which doesn't exist in either file; replace with the actual ctx variable and clarify the comment wording. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> * fix(intrinsics): address jakelorocco review comments on guardian PR - Hoist result-type comments above result= assignment in factuality examples - Update policy_guardrails docstring to reflect LoRA or aLoRA adapter Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> * refactor(intrinsics): extract _extract_last_response shared helper _inject_documents and _resolve_response shared the same thunk-extraction and context-rewind logic. Extract _extract_last_response into _util.py and delegate both functions to it. _extract_last_response also handles manually-added assistant Messages (a case _resolve_response did not cover), so find_citations, flag_hallucinated_content, and find_context_attributions now accept manually-constructed contexts where previously they would raise. Addresses code-review feedback from jakelorocco on PR generative-computing#1323. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> * docs(guardian): document image-drop behaviour and enumerate Raises triggers _inject_documents rebuilds the assistant turn from extracted text only; any images on the original turn are silently dropped — add a note in the Returns section. Broaden the Raises: prose in factuality_detection and factuality_correction to enumerate the three triggers (empty context, non-assistant last turn, uncomputed response) rather than vague "cannot be extracted". Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> * refactor(guardian): replace RST markup with Google-style in intrinsic/ docstrings Replace :class:, :func:, :meth:, :data: cross-reference roles and double-backtick ``x`` literals throughout guardian.py and _util.py with plain single-backtick identifiers, matching the style already used in core.py and rag.py in the same package. No logic changes; docstring-only. Closes out the intrinsic/ portion of generative-computing#1336. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> --------- Signed-off-by: Nigel Jones <jonesn@uk.ibm.com> Co-authored-by: AngeloDanducci <angelo.danducci.ii@ibm.com>

github-actions Bot added the enhancement New feature or request label Jun 23, 2026

planetf1 marked this pull request as ready for review June 24, 2026 09:58

planetf1 requested a review from a team as a code owner June 24, 2026 09:58

planetf1 requested review from AngeloDanducci, jakelorocco and markstur June 24, 2026 09:58

planetf1 enabled auto-merge June 24, 2026 10:00

This was referenced Jun 24, 2026

Remove deprecated check_context_relevance and context_relevance adapter #1338

Open

fix(intrinsics): deprecate check_context_relevance — Granite 4.0 only adapter #1339

Merged

jakelorocco reviewed Jun 24, 2026

View reviewed changes

Comment thread mellea/stdlib/components/intrinsic/rag.py Outdated

jakelorocco reviewed Jun 24, 2026

View reviewed changes

Comment thread mellea/stdlib/components/intrinsic/rag.py

planetf1 added 6 commits June 25, 2026 12:03

planetf1 force-pushed the worktree-issue-1137 branch from 5d6f96f to fbb206b Compare June 25, 2026 11:10

planetf1 added 2 commits June 25, 2026 13:28

jakelorocco approved these changes Jun 26, 2026

View reviewed changes

planetf1 added this pull request to the merge queue Jun 26, 2026

Merged via the queue into generative-computing:main with commit d02dc63 Jun 26, 2026
14 of 15 checks passed

planetf1 deleted the worktree-issue-1137 branch June 26, 2026 13:53

psschwei mentioned this pull request Jul 1, 2026

feat: first pass at guardian io contract subclasses #1357

Merged

8 tasks

planetf1 mentioned this pull request Jul 20, 2026

refactor(intrinsics): promote _DictContract to shared adapters module; collapse guardian.py's factuality contracts onto it #1412

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

refactor(intrinsics): migrate rag.py to new Adapter types (Epic #929 Phase 1 Wave 3)#1321

refactor(intrinsics): migrate rag.py to new Adapter types (Epic #929 Phase 1 Wave 3)#1321
planetf1 merged 8 commits into
generative-computing:mainfrom
planetf1:worktree-issue-1137

planetf1 commented Jun 23, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

planetf1 commented Jun 25, 2026

Uh oh!

planetf1 commented Jun 25, 2026

Uh oh!

planetf1 commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

planetf1 commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

What this PR changes

Output contract verification

Examples review

Relationship to other open PRs

Related issues

Test plan

Uh oh!

Uh oh!

Uh oh!

planetf1 commented Jun 25, 2026

Uh oh!

planetf1 commented Jun 25, 2026

Testing

Uh oh!

planetf1 commented Jun 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

planetf1 commented Jun 23, 2026 •

edited

Loading