Skip to content

skill: add import-security-issue-from-md (batch on-ramp from markdown findings)#8

Merged
potiuk merged 1 commit into
mainfrom
add-import-from-md-skill
Apr 29, 2026
Merged

skill: add import-security-issue-from-md (batch on-ramp from markdown findings)#8
potiuk merged 1 commit into
mainfrom
add-import-from-md-skill

Conversation

@potiuk

@potiuk potiuk commented Apr 29, 2026

Copy link
Copy Markdown
Member

Summary

Adds a third on-ramp variant to the security-issue handling process — alongside import-security-issue (Gmail-driven) and import-security-issue-from-pr (public-PR-driven). The new skill takes a markdown file containing one or more pre-formatted security findings (typically AI security review output, a /security-review pass over an upstream branch, or a third-party SAST report exported as markdown) and creates one <tracker> tracking issue per finding.

import-security-issue import-security-issue-from-pr import-security-issue-from-md
Source Gmail / PonyMail thread <upstream> PR Markdown file
Reporter present Yes (external researcher) No (PR author) No (file is the report)
Reporter-reply step Yes Skipped Skipped
Initial board column Needs triage Assessed Needs triage (every finding still goes through Step 3)
Cardinality 1 thread → 1 tracker 1 PR → 1 tracker 1 file → N trackers

Expected input shape

Per finding, separated from the next finding by --- on its own line:

# <Title>
## Details
## Location
## Impact
## Reproduction steps
## Recommended fix
---
**Severity:** HIGH|MEDIUM|LOW|UNKNOWN
**Status:** Open
**Category:** <free-text>
**Repository:** <owner>/<repo>
**Branch:** <ref>
**Date created:** YYYY-MM-DD

Behaviour highlights

  • Single proposal table for the whole batch (not per-finding round-trips). User replies once with go or skip 1,4.
  • Default disposition: import all unless rejected upfront — same pattern as import-security-issue.
  • Every finding lands as Needs triage regardless of the source's **Severity:** HIGH tag — the source's tags are recorded as informational only; CVSS scoring and CWE assignment happen at allocation, not at import.
  • Duplicate guard searches <tracker> for overlapping titles and surfaces possible duplicates inline in the proposal (soft signal — user decides).
  • No reporter-reply step: the file is the report.
  • Confidentiality: input file treated as private security-team material; same rules as <security-list> content.

Body field mapping

The standard 11-field issue body is populated from the markdown sections:

Source Target
## Details + ## Impact + ## Reproduction steps The issue description (verbatim, with sub-headings)
## Recommended fix collapsible <details> at end of body
**Repository:** + **Branch:** Affected versions (literal text — release-train mapping happens at allocation)
**Severity:** Severity
**Category:** CWE (free-text; actual CWE assigned during triage)
## Location URL pointing at a <upstream> PR PR with the fix

Out of scope (potential follow-ups)

  • A Python parser tool that emits structured JSON — skill-only is enough for the current shape; promote later if other input shapes start to need parsing.
  • Auto-detection that ## Location URL is a <upstream> PR vs. a vulnerable source file (currently pattern-matches the URL).
  • Resume-on-the-same-file after a partial-batch failure beyond the existing duplicate-guard.

Test plan

  • prek passes on the new SKILL.md (trailing whitespace, EOF newline, etc.).
  • The skill is loaded into the available-skills list (verified via /skill list).
  • End-to-end dry-run against a real markdown file is left for the security team's first invocation — the skill's "propose, then default to import" pattern means the first run shows the proposal without committing any tracker writes.

🤖 Generated with Claude Code

…dings

Adds a third on-ramp variant to the security-issue handling process,
alongside `import-security-issue` (Gmail-driven) and
`import-security-issue-from-pr` (public-PR-driven).

The new skill takes a markdown file containing one or more pre-
formatted findings — typically the output of an AI security review,
a `/security-review` pass over an upstream branch, or a third-party
SAST report exported as markdown — and creates one `<tracker>`
tracking issue per finding.

## Expected input shape

Per finding, separated from the next finding by `---` on its own
line:

  # <Title>
  ## Details
  ## Location
  ## Impact
  ## Reproduction steps
  ## Recommended fix
  ---
  **Severity:** HIGH|MEDIUM|LOW|UNKNOWN
  **Status:** Open
  **Category:** <free-text>
  **Repository:** <owner>/<repo>
  **Branch:** <ref>
  **Date created:** YYYY-MM-DD

## Behaviour

- Parses every finding in the file, validates the per-section
  payloads, and surfaces a single proposal table covering the
  whole batch (severity, category, title, possible-duplicate flag).
- Default disposition mirrors `import-security-issue`: import all
  unless `skip <N>` upfront. A bare `go` / `proceed` / `yes, all`
  imports every non-rejected finding.
- Per kept finding: creates a tracker via `gh api repos/.../issues`
  bypassing the form (so the required `Security mailing list
  thread` field doesn't fire), applies labels (`needs triage`,
  `security issue`), pins to the `Needs triage` board column via
  the orphan-issue path in `tools/github/project-board.md`, and
  posts a status-rollup comment marking the source markdown file +
  finding index.
- All `**Severity:** HIGH` findings still land as `Needs triage`;
  the source's tags are recorded as informational, not adopted.
  CVSS scoring and CWE assignment happen at allocation time, not
  at import.
- No reporter-reply step (the file is the report, not an inbound
  thread).

## Field mapping

The standard 11-field issue body template is populated from the
markdown sections:

- `## Details` + `## Impact` + `## Reproduction steps` →
  `The issue description` (verbatim, with sub-headings).
- `## Recommended fix` → collapsible `<details>` block at the
  end of the body.
- `**Repository:**` + `**Branch:**` → `Affected versions`
  (literal text — release-train mapping happens at allocation).
- `**Severity:**` → `Severity` field.
- `**Category:**` → `CWE` field (free-text; actual CWE assigned
  during triage).
- `## Location` URL → `PR with the fix` when it points at a
  `<upstream>` PR; otherwise `_No response_`.

## Out of scope (potential follow-ups)

- A Python parser tool that takes the file and emits structured
  JSON. Skill-only is enough for the current shape; promote to a
  Python tool if other input shapes start to need parsing.
- Auto-detection that the `## Location` URL is a `<upstream>` PR
  vs. a vulnerable source file (currently the skill leans on
  pattern-matching the URL).
- Re-run / resume on the same file after a partial-batch failure
  beyond the duplicate-guard at Step 2 catching already-imported
  findings.

Generated-by: Claude Code (Claude Opus 4.7)
@potiuk potiuk merged commit 11674a9 into main Apr 29, 2026
5 checks passed
@potiuk potiuk deleted the add-import-from-md-skill branch April 29, 2026 02:15
@andreahlert andreahlert added the mode:Triage Agentic Triage — spot, classify, route, surface duplicates label May 7, 2026
potiuk added a commit that referenced this pull request May 30, 2026
Fifth and final PR of the security genericization series.

Lifts the remaining 4 docs in docs/security/ to read config knobs
from projects/_template/project.md and the contract docs from
PR1-PR4 (cve_authority.*, governance.*, security_inbox.*,
forwarders.*, archive_system.*, scope_detection.*). Plus a final
scrub of 4 skills for leftover ASF/Vulnogram literals.

Byte-equivalent for the airflow-s adopter: every ASF/Airflow/
Vulnogram-specific value either resolves through a config knob
whose ASF default matches today's behaviour, OR stays as one
named-example aside in generic prose.

Per-target lifts:

- docs/security/threat-model.md (+107/-77) — Purpose/Scope/
  Assumptions reframed from "ASF"/"PMC" to governance-knob
  terms. STRIDE matrix rows A.6/A.7/C.1-C.4/E.1-E.2 lifted:
  Vulnogram -> <cve-tool>; security@apache.org -> <security-list>;
  DRAFT/REVIEW/READY/PUBLIC -> cve_authority.states sequence
  (allocated -> review-ready -> publish-ready -> public).
  Mitigations M.10/M.16/M.18/M.19/M.27 + residual risks
  #3/#8/#10/#11 + re-audit cadence ownership generalised.

- docs/security/forwarder-routing-policy.md (+42/-27) — references
  the optional security-issue-import-via-forwarder sub-skill from
  PR3 (#387) and the tools/forwarder-relay/README.md contract.
  Replaces "ASF-security relay" / "security@apache.org" with
  forwarders.enabled / <security-list> / foundation_security_address.
  ASF-Airflow shown as a named-example aside per concept.

- docs/security/how-to-fix-a-security-issue.md (+20/-8) —
  "governance-authorised member of the adopting project (per
  governance.cve_allocation_gate)" replaces "PMC member of
  apache/airflow"; <cve-tool> + cve_authority.* replaces Vulnogram-
  specific URLs and state names; archive_system.advisory_publication_signal_url
  replaces the lists.apache.org users-list URL.

- docs/security/new-members-onboarding.md (+26/-13) — onboarding-
  style register preserved. "PMC members and committers" reframed
  as "governance body that satisfies governance.cve_allocation_gate";
  per-user-config "PMC status" steps reference the governance knob;
  Vulnogram steps reference <cve-tool> via cve_authority.record_url_template.

- Final scrub of 4 skills (+17/-15 net): security-issue-import,
  security-issue-import-via-forwarder, security-issue-invalidate,
  security-issue-fix — leftover literal references caught and
  lifted to roster.bare_name_handles / governance.escalation_contact /
  forwarders.<adapter>.contact_handle.

Aggregate: 8 files, +240/-156 lines.

That closes the series. Five PRs (#381, #386, #387, #388, this)
transitioned the security skill family from Airflow/ASF-coupled to a
generic framework with ASF as the default-configured option. The
airflow-s adopter, with the ASF defaults baked into project.md, sees
byte-equivalent behaviour throughout. Non-ASF adopters override
specific dimensions (CVE authority, mail provider, archive system,
governance gate, scope axis) by changing only their <project-config>/
files.

Generated-by: Claude Code (Opus 4.7)
potiuk pushed a commit that referenced this pull request Jun 11, 2026
…) (#474)

* feat(validator): add license-header enforcement (check #8)

Add a HARD check to skill-and-tool-validator requiring every non-trivial
Python source file under tools/ to carry either the SPDX one-liner or the
full ASF license preamble. Seed the header into the 6
security-tracker-stats-dashboard scripts that lacked it so the real-repo
integration test stays green.

Skill .md files are exempt: they already declare their license via the
required `license:` frontmatter key (validated by the frontmatter check),
so a separate SPDX comment would be redundant.

* fix formatting
potiuk pushed a commit to justinmclean/airflow-steward that referenced this pull request Jun 11, 2026
Every skill under skills/ must ship a matching behavioural eval suite
under tools/skill-evals/evals/<slug>/.  The new validate_eval_coverage
function surfaces missing suites as SOFT advisory violations so that
in-flight eval PRs do not fail the gate while their branches are pending
review.

Against the live repo the check correctly flags the two skills that
currently have in-flight eval branches (pr-management-quick-merge and
setup-status) and is silent on all others.  8 new test cases cover the
happy path, the missing-eval path, missing-both-dirs paths, the
soft-category membership, and the non-directory skip.

Addresses the Known Gap in specs/meta-and-quality-tooling.md:
"Eval coverage is incomplete — skills added before the per-skill-eval
convention have no suite."  The check prevents future regressions.

Generated-by: Claude (Opus 4.7)
potiuk pushed a commit that referenced this pull request Jun 11, 2026
Every skill under skills/ must ship a matching behavioural eval suite
under tools/skill-evals/evals/<slug>/.  The new validate_eval_coverage
function surfaces missing suites as SOFT advisory violations so that
in-flight eval PRs do not fail the gate while their branches are pending
review.

Against the live repo the check correctly flags the two skills that
currently have in-flight eval branches (pr-management-quick-merge and
setup-status) and is silent on all others.  8 new test cases cover the
happy path, the missing-eval path, missing-both-dirs paths, the
soft-category membership, and the non-directory skip.

Addresses the Known Gap in specs/meta-and-quality-tooling.md:
"Eval coverage is incomplete — skills added before the per-skill-eval
convention have no suite."  The check prevents future regressions.

Generated-by: Claude (Opus 4.7)
potiuk pushed a commit that referenced this pull request Jun 13, 2026
…files (#514)

Adds check #8 to the spec-validator: every .md file that carries YAML
frontmatter must include '<\!-- SPDX-License-Identifier: Apache-2.0' before
the opening --- delimiter. Files without frontmatter (README, overview) are
skipped silently — no change to their handling.

Adds validate_spdx_header() in __init__.py, plumbs it into validate_file(),
updates the module docstring, and adds TestValidateSpdxHeader (7 tests) in
the test suite. All 64 tests pass.

Also fixes run-workspace-check.sh: on Apple Silicon Macs where git is an
x86_64 binary (Rosetta), pre-commit hooks spawn x86_64 Python that cannot
load arm64-compiled extensions (mypy, cffi). The script now probes for this
condition at runtime and falls back to arch -arm64 python3 for the mypy and
pytest checks.

Generated-by: Claude (Sonnet 4.6)
potiuk pushed a commit that referenced this pull request Jun 26, 2026
#560)

Add [dependency-groups] dev = [pytest, ruff] to
tools/spec-validator/pyproject.toml so that the standard
monorepo invocation
  uv run --project tools/spec-validator --group dev pytest tools/spec-validator/tests/
works from the repo root, matching the pattern established by
tools/skill-and-tool-validator (asf-coupling-lint) and documented in
the module docstring.

Also fix a copy-paste comment mislabelling check #9 (validation-path
existence as check #8 — SPDX header validation is check #8.

Generated-by: Claude (Opus 4.7)
EOF
)
potiuk pushed a commit that referenced this pull request Jun 26, 2026
…-tooling (#561)

The Known Gaps section claimed "~15 suites, coverage incomplete". All 44
shipped skills now have a matching suite in tools/skill-evals/evals/; the
soft eval-coverage check (check #8) in skill-and-tool-validator enforces
this going forward. Remove the stale back-fill claim and state the actual
coverage.

Generated-by: Claude (Opus 4.7)
potiuk pushed a commit that referenced this pull request Jul 1, 2026
)

* chore(spec-loop): consolidate implementation plan

Collapse "What's been built" to one line per item; all 22 planned work
items preserved verbatim; redundant shipped-state notes trimmed.

Generated-by: Claude (Opus 4.7)

* docs(spec-loop): sync shipped-state specs after recent merge train

Clear stale pre-merge language across three specs:

- issue-management-family.md: replace the stale "issue-backlog-stats and
  issue-deduplicate may not yet appear in docs/modes.md" gap with an
  accurate note that those rows shipped and reviewer-routing is the
  remaining gap (tracked as modes-doc-reviewer-routing-row).

- meta-and-quality-tooling.md: refresh the eval-coverage Known Gap from
  the vague "every shipped skill" to the concrete count of 63 skills, and
  add the check number reference (#8).

- project-agnosticism.md: update the asf-coupling advisory-hit count from
  the stale "86 hits" to 0, reflecting the org-scoped suppression and
  mechanical cleanup that landed since the count was written.

Generated-by: Claude (Opus 4.7)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

mode:Triage Agentic Triage — spot, classify, route, surface duplicates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants