Add optional 'ATR Threat Rules' deterministic text check - #77
Conversation
Adds an 'ATR Threat Rules' guardrail that runs the open-source Agent Threat Rules ruleset, via the optional pyatr engine, over text and trips when a rule matches at or above a configurable severity. Mirrors the keywords check (a deterministic, non-LLM check). - src/guardrails/checks/text/atr.py: ATRCfg + scan_atr + atr_threat_rules, registered as 'ATR Threat Rules' (engine RegEx). pyatr is imported lazily; when absent the check reports execution_failed with a clear ImportError. - checks/__init__.py: import the check so it registers. - tests/unit/checks/test_atr.py: config, injection->trip, benign->pass, async wrapper, and the no-pyatr execution-failure path (pytest.importorskip). - pyproject: optional extra 'atr = [pyatr>=0.2.6]' (not a core dependency). Match details are serialized to plain dicts; rule counts are left to the live ATR repository to avoid staleness. Signed-off-by: Adam Lin <adam@agentthreatrule.org>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d83d41b28b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| [project.optional-dependencies] | ||
| atr = [ | ||
| "pyatr>=0.2.6", |
There was a problem hiding this comment.
With this requirement, pip install openai-guardrails[atr] cannot resolve because PyPI's pyatr release history currently tops out at 0.2.5. That makes the newly documented ATR extra unusable for anyone trying the advertised install path until 0.2.6 exists; pin to an available release or publish the required pyatr version first.
Useful? React with 👍 / 👎.
|
|
||
| threshold = _SEVERITY_RANK.get(config.min_severity, 1) | ||
| matches = _atr_scan(data, rules_dir=config.rules_dir) | ||
| flagged = [m for m in matches if _SEVERITY_RANK.get(m.severity, 0) >= threshold] |
There was a problem hiding this comment.
When min_severity="low", ATR matches whose severity is informational will trip the guardrail because unknown severities default to rank 0, the same rank as low. ATR rules allow informational severity, so custom or future bundled rules at that level would produce false positives despite being below the configured threshold; add an explicit informational rank below low or default unknown severities below the threshold.
Useful? React with 👍 / 👎.
|
Flagging this one, as it has been open a while with no review yet — I don't know who owns triage here, so apologies for not tagging directly. It adds an optional deterministic text check backed by ATR's ruleset, off by default, so it is opt-in and does not change existing behaviour. Mergeable against main. Two honest questions: is an optional third-party ruleset check something openai-guardrails wants in-tree at all, and if so is there a contribution path I have missed? A clear "not a direction we want" is more useful to me than an open PR, and I would close it and ship it separately. |
1 participant
This adds a new deterministic (non-LLM) text guardrail check, ATR Threat Rules, mirroring the existing keywords check.
What it does: scans text with the open-source pyatr engine and trips when a match at or above a configurable min_severity is found. It returns the matched rule ids and severities in GuardrailResult.info, so it is usable as a fast, in-process pre-filter alongside the LLM-based checks.
Design notes
Files
Local checks: the 5 new tests pass; ruff and format applied. The check is opt-in and changes no default behavior.
Disclosure: I maintain the ATR project referenced here. This is offered as an optional, opt-in check; it is not wired into any default configuration.