add Privacy-LLM gate-check validator

[justinmclean]

What

Adds a new validate_privacy_patterns check to the skill validator that
enforces Pattern 6 from write-skill/security-checklist.md: skills that
read full issue bodies from the private <tracker> repo must run the
Privacy-LLM gate-check before making any outbound LLM call.

Also fixes the two skills the new check identified as missing the gate.

Why

Skills like security-issue-deduplicate and security-issue-fix pass
embargoed CVE detail, reporter PII, and other private content from
<tracker> issue bodies directly to the model. The privacy-LLM
gate-check is the required pre-flight for that — it was documented in
the security checklist but not enforced.

Changes

tools/skill-validator

• New PRIVACY_CATEGORY constant (soft violation category)
• New validate_privacy_patterns — fires when a skill has an
  external-content mode + references <tracker> + contains
  gh issue view (the read discriminator). Skills that only write to
  or query metadata from the tracker are exempt.
• 11 new tests in TestPrivacyPatternP6

.claude/skills

• security-issue-deduplicate: added privacy-LLM gate to Step 0
• security-issue-fix: added privacy-LLM gate to Step 0

Notes

The gh issue view discriminator is what separates genuine gaps from
false positives — security-issue-import-from-pr references <tracker>
as a write destination and uses only gh search/gh api for reads,
so it is correctly exempt.

[potiuk]

Nice!

[potiuk]

Hi @justinmclean — heads-up: this PR currently shows as conflicting against main.

The conflicts are in tools/skill-validator/src/skill_validator/__init__.py and tools/skill-validator/tests/test_validator.py and overlap with merged #220 ("detect Pattern 4 injection-guard callout"), which reorganised the validator registry and the test suite in the same area.

I had a look at rebasing it on your behalf, but resolving how the new validator from this PR should coexist with the Pattern 4 infrastructure feels like a judgement call you should make rather than me guessing. Could you rebase against latest main and resolve when you get a moment? Happy to re-review once it's clean.

Thanks!

[andreahlert]

@justinmclean pulled this locally. 86 tests pass and skill-validate --strict is clean.

audited the discriminator: 7 skills with mode + <tracker> + gh issue view are covered, security-issue-import-from-pr is correctly exempt (only gh search / gh api for metadata + writes). design works for the current codebase.

three things flagged inline. the substring-match one is the real concern, the other two are follow-up scope.

also branch is mergeable=CONFLICTING so a rebase is needed anyway. would fold the substring fix in while you're rebasing.

[andreahlert]

Just a few adjustments left. Put all in the comments.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a new “Privacy-LLM gate-check” validation (Pattern 6) and makes the mode taxonomy canonical by reading it from docs/modes.md, with accompanying tests and skill updates.

Changes:

• Read MISSION mode taxonomy from docs/modes.md and derive ALLOWED_MODES/privacy-related mode sets from it.
• Introduce validate_privacy_patterns() to enforce the Privacy-LLM gate-check for qualifying skills.
• Add extensive tests for Pattern 6 and update select skills to include the required gate-check snippet.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tools/skill-validator/tests/test_validator.py	Adds taxonomy consistency test and a comprehensive test suite for Privacy Pattern 6.
tools/skill-validator/src/skill_validator/init.py	Implements doc-driven mode taxonomy and adds Privacy-LLM gate-check validation + helpers.
.claude/skills/security-tracker-stats-dashboard/SKILL.md	Simplifies description (frontmatter).
.claude/skills/security-issue-fix/SKILL.md	Adds Privacy-LLM gate-check section to satisfy Pattern 6.
.claude/skills/security-issue-deduplicate/SKILL.md	Adds Privacy-LLM gate-check section to satisfy Pattern 6.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[justinmclean]

Just working on the conflicts

[justinmclean]

@andreahlert I believe everything has been done

[justinmclean]

Crap more conflicts - working on it

[andreahlert]

LGTM. Thanks @justinmclean.

@choo121600 could you take an extra look and help us here?

[choo121600]

overall lgtm :)

[potiuk]

Cool :)