docs: note policy_guardrails now raises AdapterSchemaMismatchError#1427
docs: note policy_guardrails now raises AdapterSchemaMismatchError#1427planetf1 wants to merge 4 commits into
Conversation
Since v0.7.0 (PR generative-computing#1357), policy_guardrails() raises AdapterSchemaMismatchError — rather than ValueError — when the adapter returns output with neither or both of `label`/`score`. As AdapterSchemaMismatchError does not subclass ValueError, callers that wrapped the call in `except ValueError` silently stopped catching that failure. Add a callout to the Safety Guardrails how-to (where the contract is taught) and a matching troubleshooting entry (where a consumer whose handler stopped firing would look), each documenting the changed exception contract and the JSON-parse ValueError that is still raised. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
|
IIUC this adds the documentation to the |
markstur
left a comment
There was a problem hiding this comment.
the text itself look ok to me.
There's other feedback about how it fits in docs/tabs...
|
When I added the doc build I did add the ability to freeze the docs alongside each release so that they’d be current for that release. So when workin on main (as we do here) the new content will go into Next until the next release (and before I went on vacation of course we didn’t have anything except Next …) In terms of the text, it’s accurate, and will be helpful when the next release is shipped (good to know when something was introduced/changed) The ‘gap’ is that we don’t have an approach to retrospectively update previous release docs - they are just a snapshot in time (rather than being a living reference for that code version) since arguably that could have been even better to do here. We couldn’t do it in the past. Given the current set up the gap is really in policy, not tooling — the contribution guide doesn’t explain what to do to update ‘old’ docs Good spot — it’s something we can do better now. I’ve updated the 0.7.x docs & doc-docs . |
policy_guardrails() started raising AdapterSchemaMismatchError in the v0.7.0 release itself, so the callout added for generative-computing#1413 belongs in that release's published docs snapshot, not only in Next. Mirror the same two edits into version-0.7.0/how-to/safety-guardrails.md and version-0.7.0/troubleshooting/common-errors.md. Also document the backport policy in CONTRIBUTING_DOCS.md so future retroactive edits to a versioned snapshot follow a consistent process. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
Clarify that the versioned-snapshot backport process added in the previous commit doesn't cover the generated API/CLI reference. Those pages are produced by tooling/docs-autogen/build.py and are never regenerated for an already-released version, so hand-editing a frozen copy would silently drift from its docstring source. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
|
@markstur @ajbozarth addressed your points. ready for rereview |
This actually shows a gap in the process not capability. Versioned docs are actually updated via the release workflow, so updating them here and not on the release branch will cause drift and be lost if we ever do a patch release without also opening a PR to patch this onto the release branch (the release branch is the source of truth for it's versioned docs). The issue from that though is if updating the versioned docs is worth a patch release, I'd strongly argue it's not. So perhaps the best process would be to follow what you did here, but all open a follow up PR to do the same on the reason branch. If so it my be worth documenting the new edge case in the release docs and it would technically be the inverse (main -> release) of what the code process is (release -> main) Edit: tl;dr |
…ssion The backport-policy section added in fb0aa18/cfe4c118 answers a question that's still open on the PR: whether docs-versioning process should be tied to the release-branch workflow at all. Pull it out of this PR and settle it in a dedicated issue instead of deciding it inline here. The actual content fix (the AdapterSchemaMismatchError callout, mirrored into the version-0.7.0 snapshot) is unaffected. Assisted-by: Claude Code Signed-off-by: Nigel Jones <jonesn@uk.ibm.com>
|
@ajbozarth , there are clearly outstanding questions about how we handle doc updates that relate to previous release content — not specific to this fix. I opened #1437 to work through it properly rather than iterate on tweaking it in this pr. So I've dropped the |
ajbozarth
left a comment
There was a problem hiding this comment.
LGTM, I'll leave my remaining notes on the followup issue
Pull Request
Issue
Fixes #1413
Description
PR #1357 changed
policy_guardrails()so that malformed adapter output — aresult with neither a
labelnor ascore, or with both — now raisesAdapterSchemaMismatchErrorrather than theValueErrorraised in earlierreleases. Because
AdapterSchemaMismatchErrorextendsExceptionand does notsubclass
ValueError, any caller that wrapped the call inexcept ValueErrorsilently stopped catching that failure when v0.7.0 shipped. The function's own
docstring documents the new exception, but there was no user-facing callout of
the behaviour change.
This adds that callout in two places:
docs/docs/how-to/safety-guardrails.md— an exception-contract note underPolicy compliance, where the API is taught and a reader would write their
try/except.docs/docs/troubleshooting/common-errors.md— a matching entry in theGuardian section, with a handler example, for someone debugging why their
existing
except ValueErrorstopped firing.The two pages cross-link each other. A
ValueErroris still raised when theoutput is not valid JSON, which both notes state explicitly.
Documentation only — no code changes.
Testing
Attribution
Adding a new component, requirement, sampling strategy, or tool?
If your PR adds or modifies one of the types below, check the matching box. A checklist of type-specific review items will be posted as a comment.
NOTE: Please ensure you have an issue that has been acknowledged by a core contributor and routed you to open a pull request against this repository. Otherwise, please open an issue before continuing with this pull request.