[Bug] Duplicate findings from parallel analyzers are never deduplicated

[mimran-khan]

Summary

If a doctor writes "patient has a cough" in the morning notes, afternoon notes, and evening notes, that's still one cough — not three separate diagnoses. A good medical system would show "persistent cough (noted 3 times)" rather than three independent entries that make the patient look three times sicker.

SkillSpector has the opposite behavior. When the same security pattern (say, "uses subprocess") appears in multiple files of a skill, each occurrence is treated as a completely independent finding. A skill that calls subprocess.run() in 5 different modules gets 5 separate findings, each contributing full points to the score — even though it's fundamentally the same design choice (this skill shells out to external commands) observed in different places.

There's no consolidation step. The parallel analyzers each produce their lists, and the results are simply concatenated with operator.add. The same conceptual issue, observed N times, inflates the score N-fold. Combined with the additive scoring (#134), this makes it nearly impossible for any non-trivial skill to score below CRITICAL.

Reproduction

Create a skill with repetitive patterns:

repetitive-skill/
├── SKILL.md
├── step1.py
├── step2.py
├── step3.py
└── step4.py

Each stepN.py contains similar automation logic:

import subprocess
result = subprocess.run(["curl", "-k", f"https://api.example.com/step{N}"])

skillspector scan ./repetitive-skill/ --no-llm --format json -o report.json
python -c "
import json
data = json.load(open('report.json'))
from collections import Counter
rules = Counter(f['rule_id'] for f in data['findings'])
print(rules)
# Counter({'TM1': 4, ...}) — same rule, 4 files, each adds to score independently
"

The rule TM1 fires once per file for the same subprocess.run(["curl", "-k", ...]) pattern. Each occurrence adds 10–25 points to the score independently. The same conceptual issue (this skill uses curl -k) is counted 4 times.

Root Cause

In src/skillspector/state.py:

class SkillspectorState(TypedDict):
    findings: Annotated[list[Finding], operator.add]

The operator.add reducer simply concatenates lists from all parallel analyzer outputs. There is no downstream deduplication step that checks:

• Same rule_id + same file → merge into one finding
• Same rule_id across multiple files with identical matched patterns → consolidate with a count
• Overlapping line ranges in the same file for the same rule → keep highest confidence only

The meta_analyzer can filter individual findings via LLM, but it evaluates each independently without awareness of duplicates.

Impact

• Score inflation (compounds the scoring saturation issue): Same conceptual vulnerability counted N times
• Report noise: Users must manually identify which findings are truly distinct vs. the same pattern repeated
• Misleading severity: A skill that uses subprocess in 10 files appears 10× riskier than one that uses it in 1 file, even if the actual risk surface is identical

Suggested Fix

1. Post-analysis deduplication step: After all analyzers complete but before scoring, merge findings that share the same rule_id + same file → keep only the highest-confidence instance, annotate with "occurrences": N.

2. Cross-file consolidation: Same rule_id + same matched_text pattern across different files → report once with a list of affected files, score once.

3. Consolidation node in the graph: Add a deduplicate_findings node between the parallel analyzers and the report/meta-analyzer nodes.

4. Diminishing returns: If scoring must count each instance, apply diminishing weight: 1st instance = full points, 2nd = 50%, 3rd+ = 25%.

Affected Version

SkillSpector v2.2.3