feat(analyzer): detect skills snooping on the agent ecosystem (other skills, MCP config, agent memory)#77
Conversation
rng1995
left a comment
rng1995
left a comment
There was a problem hiding this comment.
Nice addition — this closes a real gap (skills snooping on the agent's own ecosystem) that's distinct from the generic credential-path rules, and the implementation follows the existing static_patterns_* convention well: per-(rule, line) dedup, confidence-scored patterns, category/explanation/remediation wired into pattern_defaults, and good positive/negative test coverage including the node entrypoint. The regexes are simple and ReDoS-safe.
Non-blocking, but worth a follow-up given this is a detection tool:
Separator-dependent patterns are easy to evade. AS1 (\.claude/, \.codex/, …) and AS3 (skills/[^/\s'\"]+/SKILL\.md) require a literal forward slash, so they match "~/.claude/settings.json" written as a string literal but miss the equally common constructions that don't embed that separator — e.g. Path.home() / ".claude" / "settings.json", os.path.join(home, ".claude", ...), or Windows backslash paths. A malicious skill can avoid the slash trivially. Consider matching the directory token itself with an optional separator (e.g. \.claude\b and allowing [/\\]) so join-style and pathlib-style access are also caught. AS2's .mcp.json / mcp_servers patterns are already separator-independent, which is good.
Not blocking, since this is purely additive and the most common literal-path case is covered — but tightening AS1/AS3 would close an obvious bypass.
|
@CharmingGroot - Please resolve the conflicts and merge the PR. |
A skill that reads other parts of the agent ecosystem — agent config/home dirs, MCP server configuration, or another installed skill's files — was not flagged by any analyzer. E3 matches generic credential paths (~/.ssh, ~/.aws) but agent-specific paths slipped through entirely. Add a `static_patterns_agent_snooping` analyzer: AS1 (agent config/home dirs like ~/.claude, ~/.codex), AS2 (MCP config such as .mcp.json), AS3 (reading another skill's SKILL.md). Adds an "Agent Ecosystem Snooping" category. The analyzer de-duplicates matches per (rule, line) to avoid duplicate findings. Add tests for detection, same-line de-duplication, a false-positive guard (ordinary project file access), and the node entrypoint. Signed-off-by: CharmingGroot <ohyes9711@gmail.com>
ebf3bd8 to
356d39c
Compare
|
Conflicts resolved and force-pushed. Kept the upstream pattern set from #75 during resolution. Ready to merge. |
2 participants
Fixes #75.
What this adds
A skill that reaches into the agent's own environment — agent config/home dirs, MCP server configuration, or another installed skill's files — was not flagged by any analyzer. Adds a
static_patterns_agent_snoopinganalyzer under a new "Agent Ecosystem Snooping" category:~/.claude/,~/.codex/,~/.gemini/,~/.cursor/) where API keys and settings live — HIGH.mcp.json,mcp_servers) which holds server tokens/endpoints — HIGHskills/*/SKILL.md) — MEDIUMWhy this fits the skill threat model
Unlike generic file access, this threat exists only because skills run inside an agent: a malicious skill performs lateral movement within the agent's environment — stealing another skill's logic/secrets, harvesting MCP server tokens, or exfiltrating prior conversation memory. This is distinct from
E3, which targets generic OS credential files (~/.ssh,~/.aws).The reproduction from the issue (a skill reading
.mcp.json, agent memory, and another skill) went from score 0 / LOW to score 100 / CRITICAL.Scope / non-overlap
Verified the reproduction triggers none of
E1–E4, taint tracking, orbehavioral_ast. The analyzer de-duplicates matches per(rule, line)so a single line is not reported twice by the same rule.Testing
ruff check src/ tests/andruff format --check src/ tests/pass.pytest -m 'not integration'passes (606 passed, 11 skipped).static_patterns_agent_snoopingat 100% coverage. Tests cover: agent-config / MCP-config / cross-skill detection, same-line de-duplication, a false-positive guard (ordinary project file access is not flagged), and the node entrypoint.