Skip to content

docs: note policy_guardrails now raises AdapterSchemaMismatchError#1427

Open
planetf1 wants to merge 4 commits into
generative-computing:mainfrom
planetf1:docs/policy-guardrails-exception-1413
Open

docs: note policy_guardrails now raises AdapterSchemaMismatchError#1427
planetf1 wants to merge 4 commits into
generative-computing:mainfrom
planetf1:docs/policy-guardrails-exception-1413

Conversation

@planetf1

Copy link
Copy Markdown
Contributor

Pull Request

Issue

Fixes #1413

Description

PR #1357 changed policy_guardrails() so that malformed adapter output — a
result with neither a label nor a score, or with both — now raises
AdapterSchemaMismatchError rather than the ValueError raised in earlier
releases. Because AdapterSchemaMismatchError extends Exception and does not
subclass ValueError, any caller that wrapped the call in except ValueError
silently stopped catching that failure when v0.7.0 shipped. The function's own
docstring documents the new exception, but there was no user-facing callout of
the behaviour change.

This adds that callout in two places:

  • docs/docs/how-to/safety-guardrails.md — an exception-contract note under
    Policy compliance, where the API is taught and a reader would write their
    try/except.
  • docs/docs/troubleshooting/common-errors.md — a matching entry in the
    Guardian section, with a handler example, for someone debugging why their
    existing except ValueError stopped firing.

The two pages cross-link each other. A ValueError is still raised when the
output is not valid JSON, which both notes state explicitly.

Documentation only — no code changes.

Testing

  • Tests added to the respective file if code was changed
  • New code has 100% coverage if code was added
  • Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

  • AI coding assistants used

Adding a new component, requirement, sampling strategy, or tool?

If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.

  • Component
  • Requirement
  • Sampling Strategy
  • Tool

NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.

Since v0.7.0 (PR generative-computing#1357), policy_guardrails() raises
AdapterSchemaMismatchError — rather than ValueError — when the adapter
returns output with neither or both of `label`/`score`. As
AdapterSchemaMismatchError does not subclass ValueError, callers that
wrapped the call in `except ValueError` silently stopped catching that
failure.

Add a callout to the Safety Guardrails how-to (where the contract is
taught) and a matching troubleshooting entry (where a consumer whose
handler stopped firing would look), each documenting the changed
exception contract and the JSON-parse ValueError that is still raised.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@github-actions github-actions Bot added the documentation Improvements or additions to documentation label Jul 22, 2026
@planetf1
planetf1 marked this pull request as ready for review July 22, 2026 14:30
@planetf1
planetf1 requested a review from a team as a code owner July 22, 2026 14:30
@ajbozarth

Copy link
Copy Markdown
Contributor

IIUC this adds the documentation to the Next tab of docs not the 0.7.0 tab correct? did you intend on editing both given this seems to be a 0.7 specific call out?

@markstur markstur left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the text itself look ok to me.
There's other feedback about how it fits in docs/tabs...

@planetf1

Copy link
Copy Markdown
Contributor Author

When I added the doc build I did add the ability to freeze the docs alongside each release so that they’d be current for that release.

So when workin on main (as we do here) the new content will go into Next until the next release (and before I went on vacation of course we didn’t have anything except Next …)

In terms of the text, it’s accurate, and will be helpful when the next release is shipped (good to know when something was introduced/changed)

The ‘gap’ is that we don’t have an approach to retrospectively update previous release docs - they are just a snapshot in time (rather than being a living reference for that code version) since arguably that could have been even better to do here. We couldn’t do it in the past.

Given the current set up the gap is really in policy, not tooling — the contribution guide doesn’t explain what to do to update ‘old’ docs

Good spot — it’s something we can do better now.

I’ve updated the 0.7.x docs & doc-docs .

planetf1 added 2 commits July 23, 2026 09:06
policy_guardrails() started raising AdapterSchemaMismatchError in the
v0.7.0 release itself, so the callout added for generative-computing#1413 belongs in that
release's published docs snapshot, not only in Next. Mirror the same
two edits into version-0.7.0/how-to/safety-guardrails.md and
version-0.7.0/troubleshooting/common-errors.md.

Also document the backport policy in CONTRIBUTING_DOCS.md so future
retroactive edits to a versioned snapshot follow a consistent process.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Clarify that the versioned-snapshot backport process added in the
previous commit doesn't cover the generated API/CLI reference. Those
pages are produced by tooling/docs-autogen/build.py and are never
regenerated for an already-released version, so hand-editing a frozen
copy would silently drift from its docstring source.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1
planetf1 requested a review from markstur July 23, 2026 08:20
@planetf1

Copy link
Copy Markdown
Contributor Author

@markstur @ajbozarth addressed your points. ready for rereview

@ajbozarth

ajbozarth commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

I’ve updated the 0.7.x docs & doc-docs .

This actually shows a gap in the process not capability. Versioned docs are actually updated via the release workflow, so updating them here and not on the release branch will cause drift and be lost if we ever do a patch release without also opening a PR to patch this onto the release branch (the release branch is the source of truth for it's versioned docs).

The issue from that though is if updating the versioned docs is worth a patch release, I'd strongly argue it's not. So perhaps the best process would be to follow what you did here, but all open a follow up PR to do the same on the reason branch. If so it my be worth documenting the new edge case in the release docs and it would technically be the inverse (main -> release) of what the code process is (release -> main)

Edit: tl;dr
Add a section to your contributing doc update noting that said change also needs to be ported to the release branch to prevent drift upon a patch release, and open a followup PR against the release branch cherry-picking the commit that this PR lands on main

…ssion

The backport-policy section added in fb0aa18/cfe4c118 answers a question
that's still open on the PR: whether docs-versioning process should be
tied to the release-branch workflow at all. Pull it out of this PR and
settle it in a dedicated issue instead of deciding it inline here. The
actual content fix (the AdapterSchemaMismatchError callout, mirrored into
the version-0.7.0 snapshot) is unaffected.

Assisted-by: Claude Code
Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
@planetf1

planetf1 commented Jul 24, 2026

Copy link
Copy Markdown
Contributor Author

@ajbozarth , there are clearly outstanding questions about how we handle doc updates that relate to previous release content — not specific to this fix. I opened #1437 to work through it properly rather than iterate on tweaking it in this pr.

So I've dropped the CONTRIBUTING_DOCS.md backport-policy addition from this PR; the actual fix (exception-type callout, mirrored into version-0.7.0/) stays as is - imo it;s ‘good-enough’ pending process definition. We can add the policy note once #1437 has the process resolved.

@ajbozarth ajbozarth left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I'll leave my remaining notes on the followup issue

@planetf1
planetf1 added this pull request to the merge queue Jul 24, 2026
@github-merge-queue
github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Jul 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: callout that policy_guardrails now raises AdapterSchemaMismatchError instead of ValueError

3 participants