detect Pattern 4 injection-guard callout; fix four pr-management skills#220
Merged
Conversation
potiuk
approved these changes
May 19, 2026
This was referenced May 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds a
validate_injection_guardcheck totools/skill-validator(Pattern 4 from
write-skill/security-checklist.md) and retrofits thefour
pr-management-*skills that were missing the required callout.Why
security-checklist.mdstates the callout is required on every skillthat reads external content, and
init_skill.pyalready scaffolds aplaceholder for it — but nothing in the automated toolchain caught skills
that dropped the placeholder without adding the real block. The four
PR-management skills each read attacker-controlled content (PR titles,
bodies, diffs, review comments, issue threads) and relied on an inline
"golden rule" paraphrase instead of the standard callout. That paraphrase
was invisible to any tooling and easy to miss in review.
Changes
tools/skill-validator/src/skill_validator/__init__.pyINJECTION_GUARD_CATEGORY(hard),INJECTION_GUARD_TODO_CATEGORY(soft, added toSOFT_CATEGORIES),INJECTION_GUARD_CALLOUT_SENTINEL,INJECTION_GUARD_TODO_SENTINEL,EXTERNAL_SURFACE_SIGNALS._strip_html_commentsand_skill_body— strip thescaffolded
<!-- TODO … -->comment before scanning for signals, sothe placeholder's own example list ("Gmail, public PRs, scanner
findings") doesn't self-trigger.
validate_injection_guardwired intorun_validationfor every
SKILL.md. Two violation classes:injection_guard) — skill body signals external-surfacereads but the callout phrase is absent and the scaffold TODO has
been deleted.
injection_guard_todo) — theinit_skill.py<!-- TODO — INJECTION-GUARD CALLOUTplaceholder is still present;suppresses the hard check (skill is mid-development).
_SOFT_RULE_PREFIXES.Signal design — seven patterns, kept specific to avoid false positives
on skills that document what to do with external content rather than
reading it themselves (
write-skillmentions Gmail in prose;setup-shared-config-syncdrafts commit messages — neither fires):gh pr view/diff/listgh issue viewponymail,mboxgmail.googleapis | Gmail MCP | Gmail APIgmailrejectedscanner-finding… external content … data/never an instructiontools/skill-validator/tests/test_validator.pyFourteen new tests in
TestValidateInjectionGuardcovering clean skills,each individual signal, callout-inside-HTML-comment not counted,
TODO-suppresses-HARD, signal-inside-HTML-comment ignored, and category
exposure.
TestSoftCategoriesupdated to assert the new soft category.Four
pr-management-*SKILL.md filesPattern 4 callout inserted just before the
---/## Adopter overridespreamble in each, with a surface list specific to what that skill reads:
pr-management-triagepr-management-code-reviewpr-management-mentorpr-management-statsVerification
run_validation()returns zero hard violations and zero soft warningsagainst the full repo.
--quietsweep of the entire
tools/skill-evals/evals/tree).SKILL.mdfiles confirmed to contain the calloutsentinel.
write-skillandsetup-shared-config-syncconfirmed to produce zerofalse-positive hard violations.