From 76beea3a4055acc3f6d58d3b78bfd59477c7591b Mon Sep 17 00:00:00 2001 From: Jarek Potiuk Date: Sat, 30 May 2026 17:22:17 +0200 Subject: [PATCH 1/2] feat(security): config schema + adapter contracts (PR1/5) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit First of 5 PRs converting the security skill family from Airflow/ASF-coupled to a generic framework with ASF as the default-configured option. This PR is pure additions — zero behaviour change. Every existing ASF assumption gets a config knob with the current behaviour as the default, so the airflow-s reference adopter is byte-equivalent. Additions: - projects/_template/project.md — new "Security workflow configuration" section with 11 YAML blocks covering every ASF-coupling dimension surfaced by the discovery audit (179 findings across 18 files): cve_authority, governance, security_inbox, forwarders, mail_provider, archive_system, tracker, scope_detection, release_process, roster, product. Every field carries a comment naming what it controls, the ASF default, when a non-ASF adopter would override it, and the 1-3 skills that consume it. - tools/cve-tool/README.md — adapter contract for CNA backends. Defines the interface every CVE-authority adapter must implement (allocate, fetch, push, publish, retract) plus a generic state-verb mapping (allocated -> review-ready -> publish-ready -> public). ASF-default adapter: tools/vulnogram/ (renamed to tools/cve-tool-vulnogram/ in PR4). - tools/mail-archive/README.md — adapter contract for public mail-archive backends. Defines search_thread_url, fetch_thread_by_url, list_recent_threads, publication_signal_url. ASF-default adapter: tools/ponymail/ (renamed in PR3). - tools/forwarder-relay/README.md — adapter contract for forwarder-relay inbound paths. Defines detect, extract_credit, contact_handle, preamble_match, reporter_addressing_block. ASF-default adapter: the ASF Security forwarder shape in tools/gmail/asf-relay.md (renamed in PR3). - docs/labels-and-capabilities.md — 3 new rows for the new tool stubs (all capability:setup, pure interface specs). No skill bodies touched. No tool implementations renamed. No ASF-default adapter changes. Skills will be lifted to read these knobs in PR2-PR5. Generated-by: Claude Code (Opus 4.7) --- docs/labels-and-capabilities.md | 3 + projects/_template/project.md | 612 ++++++++++++++++++++++++++++++++ tools/cve-tool/README.md | 410 +++++++++++++++++++++ tools/forwarder-relay/README.md | 403 +++++++++++++++++++++ tools/mail-archive/README.md | 343 ++++++++++++++++++ 5 files changed, 1771 insertions(+) create mode 100644 tools/cve-tool/README.md create mode 100644 tools/forwarder-relay/README.md create mode 100644 tools/mail-archive/README.md diff --git a/docs/labels-and-capabilities.md b/docs/labels-and-capabilities.md index 1792142a..27462600 100644 --- a/docs/labels-and-capabilities.md +++ b/docs/labels-and-capabilities.md @@ -171,11 +171,14 @@ Tools under [`tools/`](../tools/). Tools with two values (separated by |---|---|---| | [`tools/agent-isolation`](../tools/agent-isolation/) | `capability:setup` | Secure-agent sandbox helpers | | [`tools/cve-org`](../tools/cve-org/) | `capability:resolve` + `capability:intake` | Publishes to CVE.org *(resolve)* and records the resulting CVE state back into the tracker *(intake)* | +| [`tools/cve-tool`](../tools/cve-tool/) | `capability:setup` | Adapter contract for CNA backends (Vulnogram, MITRE form, CVE.org direct, GHSA). Pure interface spec; no executable code — adapters under sibling `tools/cve-tool-*/` directories implement it. | | [`tools/dashboard-generator`](../tools/dashboard-generator/) | `capability:stats` | Self-contained HTML dashboard generator | | [`tools/dev`](../tools/dev/) | `capability:setup` | Framework dev-loop helpers | +| [`tools/forwarder-relay`](../tools/forwarder-relay/) | `capability:setup` | Adapter contract for inbound-relay backends (ASF Security relay, huntr.com, HackerOne triagers). Pure interface spec; adapters declare detection + credit-extraction + reporter-addressing rules. | | [`tools/github`](../tools/github/) | `capability:setup` | GitHub REST / GraphQL substrate (called by every lifecycle phase — pure substrate, no single phase) | | [`tools/gmail`](../tools/gmail/) | `capability:setup` | Gmail API substrate | | [`tools/jira`](../tools/jira/) | `capability:setup` | JIRA REST substrate (read-only today; write subcommands tracked in [#301](https://github.com/apache/airflow-steward/issues/301)) | +| [`tools/mail-archive`](../tools/mail-archive/) | `capability:setup` | Adapter contract for public mail-archive backends (PonyMail, Hyperkitty, Discourse, Google Groups, GitHub Discussions). Pure interface spec. | | [`tools/mail-source`](../tools/mail-source/) | `capability:setup` + `capability:intake` | Mail-source backend abstraction (mbox / IMAP / Mailman 3); the abstraction is setup, every concrete read is part of the intake pipeline | | [`tools/ponymail`](../tools/ponymail/) | `capability:setup` + `capability:intake` | PonyMail archive substrate; same dual role as `mail-source` — substrate plus an intake-pipeline component | | [`tools/pr-management-stats`](../tools/pr-management-stats/) | `capability:stats` | PR-backlog analytics engine | diff --git a/projects/_template/project.md b/projects/_template/project.md index 87cbb3b0..929f1e31 100644 --- a/projects/_template/project.md +++ b/projects/_template/project.md @@ -13,6 +13,18 @@ - [Backend declaration](#backend-declaration) - [Per-backend config](#per-backend-config) - [Issue-template fields](#issue-template-fields) + - [Security workflow configuration](#security-workflow-configuration) + - [CVE authority](#cve-authority) + - [Governance](#governance) + - [Security inbox](#security-inbox) + - [Forwarders](#forwarders) + - [Mail provider](#mail-provider) + - [Archive system](#archive-system) + - [Tracker](#tracker) + - [Scope detection](#scope-detection) + - [Release process](#release-process) + - [Roster](#roster) + - [Product](#product) - [Pointers to sibling files](#pointers-to-sibling-files) @@ -218,6 +230,606 @@ project. | `severity` | TODO | `dropdown` | TODO | | `cve-tool-link` | TODO | `input` | TODO | +## Security workflow configuration + +This block declares the **plug-points** that drive every ASF-coupled +assumption in the skills. The defaults shipped here reproduce the +current Apache Airflow security-team behaviour byte-for-byte: an +adopter who copies `projects/_template/` into a fresh `/` +and changes nothing in this section ends up with the same workflow +that runs in `airflow-s/airflow-s` today. Non-ASF adopters override +individual fields (CNA tool, mail backend, archive system, governance +gate, etc.) without touching skill bodies — skills resolve these knobs +at run time. Each field carries a `#` comment stating *what it +controls*, the *ASF default*, *when a non-ASF adopter would override +it*, and the *consuming skills* (1-3 most relevant names). + +The adapter contracts these blocks reference live under: + +- [`../../tools/cve-tool/contract.md`](../../tools/cve-tool/contract.md) — CNA tool interface (ASF default adapter: `tools/vulnogram/`) +- [`../../tools/mail-archive/contract.md`](../../tools/mail-archive/contract.md) — public-archive interface (ASF default adapter: `tools/ponymail/`) +- [`../../tools/forwarder-relay/contract.md`](../../tools/forwarder-relay/contract.md) — inbound-relay interface (ASF default adapter: the ASF-security forwarder shape in `tools/gmail/asf-relay.md`) + +### CVE authority + +```yaml +cve_authority: + # Which CNA tool the project uses to allocate, edit, and publish CVE + # records. Selects the adapter under tools/cve-tool/. + # ASF default: vulnogram (ASF-hosted Vulnogram instance at + # cveprocess.apache.org). Non-ASF adopters running their own MITRE + # CNA pick `mitre-form` or `cve-org-direct`; GHSA-only projects pick + # `ghsa`; pre-CNA projects pick `none`. + # Consumed by: security-cve-allocate, security-issue-sync, + # generate-cve-json. + tool: vulnogram + + # Front-door allocation URL. Skill prints this and waits for the + # operator to paste the allocated ID back. + # ASF default: Vulnogram's ASF allocation endpoint. + # Override when: pointing at a non-ASF Vulnogram tenant, or any + # other CNA tool's allocation UI. + # Consumed by: security-cve-allocate. + allocate_url: https://cveprocess.apache.org/allocatecve + + # Template for the per-record edit/view URL. `` is the + # placeholder the skill substitutes. + # ASF default: Vulnogram's cve5 record view. + # Override when: any non-Vulnogram CNA tool. + # Consumed by: security-cve-allocate, security-issue-sync. + record_url_template: https://cveprocess.apache.org/cve5/ + + # Template for the "Source" tab inside the CNA tool — used when the + # skill needs to inspect raw CNA_private state. + # ASF default: Vulnogram cve5 source tab. + # Override when: the adapter exposes raw record state via a + # different URL (or leave null if the adapter has no equivalent). + # Consumed by: security-issue-sync, generate-cve-json. + source_tab_url_template: https://cveprocess.apache.org/cve5/?tab=source + + # Template for the "preview the allocation email" tab in the CNA + # tool. The Vulnogram default emits an allocation email visible + # under this URL; null for adapters that don't preview email. + # ASF default: Vulnogram email preview tab. + # Consumed by: security-cve-allocate. + email_preview_url_template: https://cveprocess.apache.org/cve5/?tab=email + + # Generic state machine the adapter exposes. Adapters map their + # tool-native states to this 4-stop sequence; the rest of the + # workflow speaks only in these terms. + # ASF default mapping: Vulnogram's DRAFT -> allocated, + # REVIEW -> review-ready, READY -> publish-ready, PUBLIC -> public. + # Override when: the adapter has a different state machine — the + # adapter declares its own mapping in its contract.md. + # Consumed by: security-issue-sync, security-cve-allocate, + # generate-cve-json. + states: [allocated, review-ready, publish-ready, public] + + # How "record is now PUBLIC" propagates back to the workflow. + # `poll` = skill re-fetches the record on a sweep; `webhook` = + # adapter pushes; `manual` = operator flips a label by hand. + # ASF default: poll (Vulnogram has no webhook). + # Override when: a CNA tool offers a webhook (`webhook`) or only a + # human-driven publication signal (`manual`). + # Consumed by: security-issue-sync. + publication_propagation: poll + + # Whether the CNA tool emits an allocation email of its own that + # the skills should expect to see on the security mailing list. + # ASF default: true (Vulnogram auto-emails the assigner list). + # Override when: the adapter is silent on allocation — skills then + # skip the "wait for Vulnogram email" step. + # Consumed by: security-cve-allocate. + emits_allocation_email: true + + # Where the human review of the draft CVE record happens before + # publication. `mailing-list` = an off-system thread; `github-pr` + # = a PR on the tracker repo; `none` = no formal review gate. + # ASF default: mailing-list (PMC reviews on private@). + # Override when: an adopter wires review into a tracker-repo PR. + # Consumed by: security-cve-allocate, security-issue-sync. + reviewer_channel: mailing-list +``` + +### Governance + +```yaml +governance: + # Who has authority to allocate a CVE on behalf of the project. + # `pmc-member` = an ASF-style governance committee membership gate; + # `security-team-member` = looser, anyone on the security team; + # `maintainer` = any committer; `none` = no formal gate. + # ASF default: pmc-member (ASF PMC membership via OAuth into + # Vulnogram). + # Override when: non-ASF projects with their own authority model. + # Consumed by: security-cve-allocate, security-issue-sync. + cve_allocation_gate: pmc-member + + # Label the tracker applies to mark "this account is governance- + # authorised" — distinct from "security-team member". Skills use + # this to gate the allocation step. + # ASF default: "PMC" (matches the existing airflow-s label). + # Override when: a non-ASF adopter uses a different label name. + # Consumed by: security-cve-allocate, pr-management-triage. + gate_label: "PMC" + + # Whether release votes block on outstanding security work. When + # true, release-manager skills check the security tracker for + # un-fixed-but-public CVEs before greenlighting a vote. + # ASF default: true (ASF release process gates on this). + # Override when: projects with no formal release-vote gate. + # Consumed by: security-issue-sync, generate-cve-json. + release_vote_gating: true + + # Private mailing list the governance body uses for escalation, + # PMC discussions, and "this is bigger than security@" routing. + # ASF default: private@.apache.org. + # Override when: non-ASF — point at the equivalent private list + # or leave null if no such list exists. + # Consumed by: security-issue-sync, security-issue-invalidate. + private_governance_list: private@.apache.org + + # GitHub handle (or external contact) the skills cc / @-mention + # when escalating beyond the security team. + # ASF default: the PMC chair or designated escalation contact — + # filled in per-project. Use the `@` form for GitHub + # surfaces; an email is acceptable for off-GitHub escalation. + # Override when: non-ASF — point at the equivalent role-holder. + # Consumed by: security-issue-sync, pr-management-triage. + escalation_contact: "@" + + # URL of the public committee roster, used for "is this person + # authorised" checks in the allocation flow. + # ASF default: ASF committee URL (whimsy/projects/.../committee). + # Override when: non-ASF — link to the equivalent roster page. + # Consumed by: security-cve-allocate. + roster_url: https://projects.apache.org/committee.html? +``` + +### Security inbox + +```yaml +security_inbox: + # The inbound channel reports land on. `mailing-list` = an SMTP + # address; `ghsa-inbox` = GitHub Security Advisories private + # reports; `hackerone` = a HackerOne program inbox; + # `chat-channel` = e.g. a private Slack; `intake-form` = a + # web form posting into a tracker. + # ASF default: mailing-list. + # Override when: non-ASF projects on GHSA / HackerOne / etc. + # Consumed by: security-issue-import, security-issue-sync. + kind: mailing-list + + # The concrete address / channel ID / form URL the inbound channel + # uses. For `mailing-list`, this is the SMTP address. + # ASF default: security@.apache.org. + # Override when: non-ASF — replace with the adopter's inbox. + # Consumed by: security-issue-import, security-issue-sync, + # canned-responses templating. + address: + + # The foundation-wide security address that gates the + # "don't exclude this sender" rule in the inbound importer. + # Null for non-ASF adopters with no foundation-level inbox. + # ASF default: security@apache.org (the ASF security team + # forwards reports here onto project security@ lists). + # Override when: non-ASF — set to null (or your foundation's + # equivalent address if one exists). + # Consumed by: security-issue-import. + foundation_security_address: security@apache.org + + # Whether reports arrive via a forwarder/relay (an upstream party + # that triages and re-sends, rather than the reporter mailing the + # project list directly). When true, the forwarders block below + # declares the enabled adapters. + # ASF default: true (the ASF security team relays many reports). + # Override when: non-ASF projects with no forwarder layer. + # Consumed by: security-issue-import, gmail/asf-relay (adapter). + has_forwarder_relay: true + + # Optional Gmail/IMAP search filter used by the inbound importer to + # scope which threads count as "security inbox messages". + # ASF default: `list:` (Gmail's list-id + # filter for the project security list). + # Override when: the mail backend uses a different scoping + # mechanism (folder, label, etc.). + # Consumed by: security-issue-import. + list_filter_query: "list:" +``` + +### Forwarders + +```yaml +forwarders: + # Enabled forwarder/relay adapters. Each name must match an + # adapter directory under tools/ that conforms to + # tools/forwarder-relay/contract.md. + # ASF default: [asf-security] — the ASF security team relays + # reports onto project security@ lists with a known preamble and + # credit line. + # Override when: non-ASF — set to [] if no forwarder layer + # exists, or add the adopter's relay adapter name(s) here. + # Consumed by: security-issue-import, gmail/asf-relay (adapter). + enabled: [asf-security] + + # Per-adapter configuration. Keys must match `enabled` entries. + asf-security: + # Handle / address the forwarder sends from. Skills use this to + # detect "this is a relayed message, not a direct report". + # ASF default: security@apache.org. + # Override when: a different upstream relay address. + # Consumed by: gmail/asf-relay, security-issue-import. + contact_handle: security@apache.org + + # Regex matched against the message body to confirm a message + # really is a relay (not just a CC'd address). + # ASF default: the ASF preamble — "Dear PMC, The security + # vulnerability report..." + # Override when: a different relay's preamble shape. + # Consumed by: gmail/asf-relay, security-issue-import. + preamble_match: "^Dear PMC,\\s+The security vulnerability report" + + # Rule the adapter uses to lift the original reporter's credit + # line out of the relayed body. Adapters define their own + # extraction shape; see tools/forwarder-relay/contract.md. + # ASF default: the existing ASF-security credit extraction (the + # "Reported by: <>" line near the top of the + # forwarded body). + # Override when: a different relay shape. + # Consumed by: gmail/asf-relay, security-issue-import. + credit_extraction_rule: "first-line-matching:^Reported by:\\s+(.+)$" +``` + +### Mail provider + +```yaml +mail_provider: + # Primary mail backend the skills read inbound mail from and write + # drafts into. Adapters live under tools//. + # ASF default: gmail-mcp (triager Gmail account via the Gmail MCP + # adapter at tools/gmail/). + # Override when: a non-ASF adopter uses an IMAP triager mailbox + # (`imap`), an Outlook inbox (`outlook`), or a forum-style + # inbound channel (`discourse`). + # Consumed by: security-issue-import, security-issue-sync, + # security-issue-invalidate (draft replies). + primary: gmail-mcp + + # Read-only fallback backend used when the primary can't reach a + # thread (e.g. message older than the Gmail retention window). + # `none` means no fallback — operations that fail on the primary + # surface a hard error instead of trying a secondary. + # ASF default: ponymail (read-only ASF archive backstop). + # Override when: non-ASF adopters typically set this to `none` + # or to their own archive adapter (hyperkitty, mbox, ...). + # Consumed by: security-issue-sync, security-issue-import. + fallback: ponymail +``` + +### Archive system + +```yaml +archive_system: + # Public mailing-list / forum archive the project's advisories + # eventually surface on. Adapter selection — drives URL shapes + # and thread-fetch verbs. + # ASF default: ponymail (lists.apache.org). + # Override when: non-ASF adopters on hyperkitty (Mailman 3), + # discourse, google-groups, github-discussions, or none. + # Consumed by: security-issue-sync, generate-cve-json. + kind: ponymail + + # Domain the public lists live under — used to assemble per-list + # URLs (`@`) when the archive search needs + # qualified addresses. + # ASF default: .apache.org (e.g. airflow.apache.org). + # Override when: non-ASF — the adopter's public-list domain. + # Consumed by: security-issue-sync, generate-cve-json. + list_domain: .apache.org + + # Template the search-thread verb assembles. Placeholders: + # `{list}` (list short name), `{year}`, `{month}`, `{query}`. + # ASF default: ponymail's `list?` search endpoint. + # Override when: a different archive's search URL shape. + # Consumed by: security-issue-sync (search before announce), + # generate-cve-json (references[] assembly). + search_url_template: "https://lists.apache.org/list?{list}:{year}-{month}:{query}" + + # Template for the archive's programmatic thread-fetch endpoint + # (used by the mail-archive adapter's fetch_thread_by_url). + # ASF default: ponymail's thread.lua API. + # Override when: a different archive — hyperkitty has a different + # API shape; discourse exposes JSON on `/t/.json`; etc. + # Consumed by: tools/ponymail (adapter), security-issue-sync. + api_query_url_template: "https://lists.apache.org/api/thread.lua?list={list}&domain={list_domain}&id={thread_id}" + + # The URL the skill polls to detect "advisory has been announced + # publicly" — i.e. the archive page where the announcement thread + # appears once published. + # ASF default: lists.apache.org `users` list page. + # Override when: the announcement surfaces on a different list / + # forum. + # Consumed by: security-issue-sync. + advisory_publication_signal_url: "https://lists.apache.org/list.html?" +``` + +### Tracker + +```yaml +tracker: + # Platform the tracker repo lives on. Selects the API adapter + # (gh CLI today; gitlab CLI / forgejo / jira REST in the future). + # ASF default: github (airflow-s/airflow-s). + # Override when: non-ASF adopters on gitlab, gitea, jira, forgejo. + # Consumed by: every skill that touches the tracker. + platform: github + + # Project-board backend on the tracker platform. Drives the + # board-reconciliation step in the triage skills. + # ASF default: github-projects-v2. + # Override when: gitlab-board for a GitLab tracker, or `none` + # if the adopter doesn't run a board at all. + # Consumed by: pr-management-triage, security-issue-sync, + # security-issue-triage. + board: github-projects-v2 + + # Visibility of the tracker repo. Drives "may this URL leak + # publicly" guards in the canned-responses + CVE-JSON references. + # ASF default: private (tracker existence is itself secret per + # the AGENTS.md rules). + # Override when: a project that runs its security tracker openly. + # Consumed by: every skill that emits URLs to outside surfaces. + visibility: private + + # Whether the reporter can see the tracker issue once opened. + # ASF default: false (private tracker — reporter never gets a + # link). + # Override when: a public tracker where the reporter is added as + # a collaborator. + # Consumed by: security-issue-import, canned-responses templating. + reporter_has_access: false + + # Whether the tracker drives a board / kanban view. When false, + # skills skip column transitions entirely. + # ASF default: true. + # Override when: an adopter using only labels + milestones. + # Consumed by: pr-management-triage, security-issue-sync. + project_board_enabled: true + + # Template the skills use to compose a "link back to the skill + # docs as seen in this repo" URL. `` is the slug under + # .claude/skills/. + # ASF default: tracker default branch on github.com. + # Override when: a non-GitHub tracker — replace with the platform's + # equivalent file-view URL shape. + # Consumed by: pr-management-mentor, canned-responses (footer link). + skill_url_template: "https://github.com//blob/main/.claude/skills//SKILL.md" + + # Tracker body-field heading names — the literal `###` headings + # the skills read and write under in the tracker's issue + # template. The skill code refers to these by *role*; this map + # binds role -> concrete heading. + # ASF default: the existing airflow-s headings. + # Override when: an adopter with a different issue-template shape + # — change the headings here, not the skills. + # Consumed by: every skill that reads/writes the issue body. + body_fields: + cve_link: "CVE tool link" + mailing_thread: "Mailing list thread URL" + affected_versions: "Affected versions" + + # Tracker labels — role -> concrete label name. Skills speak in + # roles; this map binds role -> literal label. + # ASF default: the airflow-s label set. + # Override when: a tracker with a different label vocabulary. + # Consumed by: security-issue-triage, security-issue-sync, + # pr-management-triage. + labels: + security_marker: "security" + needs_triage: "needs triage" + pr_open: "pr created" + pr_merged: "pr merged" + cve_allocated: "cve allocated" + not_cve_worthy: "not cve worthy" +``` + +### Scope detection + +```yaml +scope_detection: + # Whether the project distinguishes scope sub-products (e.g. + # airflow vs providers vs chart). When false, every issue maps + # to the single product declared in the `product` block. + # ASF/Airflow default: true. + # Override when: a single-artifact project — set to false and + # drop the labels map. + # Consumed by: security-issue-triage, generate-cve-json, + # security-issue-sync. + enabled: true + + # Scope label -> sub-product mapping. Each entry binds a tracker + # label to the CVE `product` field value, the package-name shape + # the advisory will use, and the upstream path prefix the skill + # uses to confirm a PR really touches that scope. + # ASF/Airflow default: the three existing scope labels. + # Override when: a project with different scope axes — keep the + # `product`/`packageName`/`path_prefix` triad shape. + # Consumed by: security-issue-triage, generate-cve-json. + labels: + airflow: + product: "Apache Airflow" + packageName: "apache-airflow" + path_prefix: "^(airflow-core/|airflow/(?!providers/)|airflow-ctl/)" + providers: + product: "Apache Airflow" + packageName: "apache-airflow-providers-" + path_prefix: "^providers/" + chart: + product: "Apache Airflow Helm Chart" + packageName: "apache-airflow-helm-chart" + path_prefix: "^chart/" +``` + +### Release process + +```yaml +release_process: + # Cascade of sources skills consult to resolve "who is RM for + # version X". First match wins; later entries are tried only if + # earlier entries fail to surface a handle. + # ASF default: the project's release-trains.md roster file, then + # the wiki release-managers page, then the dev@ mailing-list + # VOTE/RESULT threads. + # Override when: non-ASF — collapse to whatever roster source the + # project keeps. Drop entries that don't apply. + # Consumed by: security-issue-sync, security-issue-fix (PR + # reviewer assignment). + release_manager_lookup_cascade: + - kind: roster_file + path: "release-trains.md" + - kind: wiki_url + url: "https://cwiki.apache.org/confluence/display//Release+Managers" + - kind: mailing_list_vote_thread + list: "" + + # Artifact registries where the project publishes releases — the + # skills cross-check that a fix has shipped here before flipping + # the issue to "fix released". + # ASF default: [pypi, artifacthub] (Python wheels + Helm chart). + # Override when: non-Python projects (`maven`, `npm`, ...) or + # projects that publish elsewhere. + # Consumed by: security-issue-sync, generate-cve-json + # (references[] population). + artifact_registries: [pypi, artifacthub] + + # Milestones the skills treat as "stale" — i.e. anything still + # pinned to one of these is overdue for re-targeting. Listed as + # exact milestone-name matches. + # ASF/Airflow default: the current airflow-s stale-milestone list. + # Override when: replace with the adopter's stale-milestone names. + # Consumed by: security-issue-sync, pr-management-triage. + stale_milestones: + - "Airflow 2.x" + - "Airflow 2.10.x" + - "Airflow 3.0.x" + + # Whether the upstream repo uses a newsfragments / changelog + # fragment tool, and which one. Skills hook this when proposing + # a fix — the fix PR must include a fragment. + # ASF/Airflow default: enabled with towncrier. + # Override when: projects without a fragment tool (`enabled: + # false`) or with a different tool (reno, changie, ...). + # Consumed by: security-issue-fix, issue-fix-workflow. + newsfragments: + enabled: true + tool: towncrier +``` + +### Roster + +```yaml +roster: + # Source of truth for the security team membership and the + # bare-name -> handle mapping. Selects how the skills resolve + # "is X on the security team". + # `tracker-collaborators` = read the tracker repo's collaborator + # list; `roster-file:` = read a checked-in file (path + # relative to /); `inline:` = a literal + # list spelled out below. + # ASF default: roster-file:release-trains.md (the canonical + # source for the Airflow security team). + # Override when: non-ASF — pick whichever shape matches the + # adopter's source of truth. + # Consumed by: security-issue-sync, security-cve-allocate, + # security-issue-triage. + source: roster-file:release-trains.md + + # Bare-name -> @handle mapping. Mailing-list threads frequently + # reference contributors by first name only; this map binds + # those bare names to GitHub handles so the skills can produce + # @-mentions on the tracker. + # ASF/Airflow default: the existing airflow-s bare-name map. + # Override when: adapt to the adopter's roster — add/remove + # entries as needed. + # Consumed by: security-issue-sync, pr-management-mentor. + bare_name_handles: + Jarek: "@potiuk" + Ash: "@ashb" + Kaxil: "@kaxil" + Ephraim: "@ephraimbuddy" + Jed: "@jedcunningham" + + # Release-manager handles, ordered by recency of train ownership. + # First handle is the current default RM; the rest are the + # historical RMs the skill falls back to when assigning legacy + # trains. + # ASF/Airflow default: the current RM order from release-trains.md. + # Override when: keep in sync with `release-trains.md`. + # Consumed by: security-issue-sync, security-issue-fix. + release_managers: + - "@ephraimbuddy" + - "@jedcunningham" + - "@potiuk" + - "@kaxil" +``` + +### Product + +```yaml +product: + # Human-readable product name — what lands in the CVE record's + # `product` field and what canned responses address the product + # as. + # ASF/Airflow default: Airflow. + # Override when: any other project — replace with the canonical + # short name. + # Consumed by: generate-cve-json, canned-responses templating. + name: Airflow + + # Package name shape for the primary artifact — used by the + # advisory templating and the CVE JSON `affected[].packageName`. + # ASF/Airflow default: apache-airflow (PyPI distribution). + # Override when: any other project — use the package-registry + # name (PyPI / npm / Maven / ...). + # Consumed by: generate-cve-json, canned-responses templating. + package_name: apache-airflow + + # Regex matched against changed paths in an upstream PR to + # confirm "this PR really touches the product". Used as a + # backstop sanity check in the fix flow. + # ASF/Airflow default: starts with `airflow` (matches + # airflow/, airflow-core/, airflow-ctl/, etc.). + # Override when: any other repo layout. + # Consumed by: security-issue-fix, pr-management-triage. + code_pointer_path_prefix: "^airflow" + + # Prefixes the title-normalization skill strips when normalising + # an inbound subject line into a CVE title. Matched at the start + # of the subject, case-insensitively, in order; the first match + # wins and is removed. + # ASF/Airflow default: the existing airflow-s strip cascade. + # Override when: any other project — replace with the adopter's + # subject-prefix conventions. + # Consumed by: title-normalization, generate-cve-json, + # canned-responses templating. + subject_prefix_strip: + - "[SECURITY]" + - "[Security Report]" + - "Re:" + - "Fwd:" + - "Airflow:" + - "Apache Airflow:" + + # Prefix the affected-versions extractor looks for in mailing-list + # bodies — reporters typically write "Airflow 2.10.0 is affected". + # Skill strips this prefix to leave the bare version literal. + # ASF/Airflow default: "Airflow". + # Override when: any other product — the literal product token + # reporters use in version expressions. + # Consumed by: security-issue-sync, generate-cve-json. + affected_version_extract_prefix: "Airflow" +``` + ## Pointers to sibling files - [`release-trains.md`](release-trains.md) — fast-moving release state, release-manager attribution, security-team roster. diff --git a/tools/cve-tool/README.md b/tools/cve-tool/README.md new file mode 100644 index 00000000..6c7df9fa --- /dev/null +++ b/tools/cve-tool/README.md @@ -0,0 +1,410 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [`tools/cve-tool/`](#toolscve-tool) + - [What this is](#what-this-is) + - [Today's adapters](#todays-adapters) + - [Interface](#interface) + - [`allocate(scope, fields) to cve_id`](#allocatescope-fields-to-cve_id) + - [`fetch_current_state(cve_id) to {state, fields}`](#fetch_current_statecve_id-to-state-fields) + - [`push_update(cve_id, fields, state_transition=None) to diff`](#push_updatecve_id-fields-state_transitionnone-to-diff) + - [`publish(cve_id) to ok`](#publishcve_id-to-ok) + - [`retract(cve_id, reason) to ok`](#retractcve_id-reason-to-ok) + - [Generic state verbs](#generic-state-verbs) + - [Skills that consume this contract](#skills-that-consume-this-contract) + - [ASF default — Vulnogram](#asf-default--vulnogram) + - [Configuration](#configuration) + + + + + +# `tools/cve-tool/` + +**Capability:** capability:setup + +## What this is + +The framework's CVE-tooling adapter contract. Today every CVE-aware +skill in this repository — `security-cve-allocate`, +`security-issue-sync`, `security-issue-invalidate`, +`security-issue-deduplicate` — speaks Vulnogram. They embed +Vulnogram's URL prefix (`cveprocess.apache.org`), Vulnogram's +`#source` / `#json` / `#email` tab shape, and Vulnogram's `DRAFT` → +`REVIEW` to `READY` to `PUBLIC` state machine directly in the +skill prose. That is fine for the ASF — the foundation runs a single +CNA tool and the skills can afford to be Vulnogram-shaped — but it +forecloses every other plausible adopter. A project running CVE.org's +direct submission portal, the MITRE form for non-CNAs, GHSA's own +CNA flow, or no CVE allocation at all has no seam to plug into. +This contract is that seam. It defines the methods every CVE-tool +backend must implement, the lifecycle moments at which the skills +fire each method, and the generic state verbs the skills use to +talk about a CVE record without baking in any one tool's vocabulary. +Skills consume the contract; the adapter consumes the wire protocol +of whichever CNA tool the adopter actually uses. + +The contract is read by the framework, not by humans during normal +operation. New adopters declare `cve_authority.tool: ` in +`projects//project.md`; the skills resolve that to a +sibling directory under `tools/` and call the methods named here. +Adopters who run Vulnogram inherit the ASF defaults verbatim. + +## Today's adapters + +Only one CVE-tool adapter ships today: + +| Adapter | Directory | Status | Notes | +|---|---|---|---| +| Vulnogram (ASF) | `tools/vulnogram/` | Shipping | The reference implementation. PR4 of the ASF-pluggable security flow renames the directory to `tools/cve-tool-vulnogram/` to match the contract's adapter-naming rule. **Not renamed in this PR.** | +| CVE.org direct submission | `tools/cve-tool-cve-org-direct/` *(planned)* | Placeholder | For adopters who are themselves a CNA and submit records straight to `cve.org` via the CVE Services API rather than through an intermediate CNA tool. | +| MITRE form | `tools/cve-tool-mitre-form/` *(planned)* | Placeholder | For adopters who are not a CNA and request CVE IDs via the [MITRE CVE Request form](https://cveform.mitre.org/). Allocation is asynchronous and operator-mediated; `fetch_current_state` will frequently return `unknown` until the form's email reply arrives. | +| GHSA-as-CNA | `tools/cve-tool-ghsa/` *(planned)* | Placeholder | For adopters who already drive their advisory flow through GitHub Security Advisories (GHSA) and use GitHub as the CNA. `allocate` becomes a GHSA draft create; `publish` becomes a GHSA publish. | +| None | `tools/cve-tool-none/` *(planned)* | Placeholder | The disable-CVE-allocation backend. All methods are no-ops returning a synthetic `not-applicable` result. For adopters who triage security issues but do not allocate CVEs (e.g. an internal product with no public distribution). | + +The four placeholder adapters do not exist on disk yet; they exist +in this table so the contract can describe the surface they will fit +into when an adopter actually needs one. PR4 of the ASF-pluggable +security flow will create `tools/cve-tool-vulnogram/` (the rename +target) but will not pre-create the other three — those land when an +adopter implements them. + +## Interface + +Every CVE-tool adapter exposes five methods. The names here are the +generic verbs the skills use; an adapter is free to name its internal +CLIs whatever fits its tool's vocabulary, as long as the skill-facing +surface uses these names. + +### `allocate(scope, fields) to cve_id` + +Reserve a CVE ID and create the initial record. + +- **Lifecycle moment.** Fires once per tracking issue, inside + `security-cve-allocate` after the security team has agreed the + report is valid and a CVE should be allocated. The pre-allocation + consensus discussion happens on the tracker; `allocate` is the + step that turns that consensus into a reserved ID. +- **Inputs.** + - `scope` — a string identifying the project scope under which + the CVE belongs. Concretely for ASF: the scope label + (`affects: airflow` / `affects: providers` / `affects: chart`) + drives which `vendor` / `product` / `packageName` triad the + adapter writes into the initial record. For other adopters this + is whatever string their CVE tool needs to route the allocation + to the right CNA pool. + - `fields` — a dict of the tracker's body fields at the moment + allocation fires: `title`, `description`, `affected_versions`, + `cwe`, `severity`, `credits`, `references`. The adapter is free + to ignore fields its tool cannot accept at allocation time and + fill them in later via `push_update`. The Vulnogram adapter + accepts the full set at allocation; the MITRE-form adapter + only sends `title` + `affected_versions` + a short description + in the initial form submission. +- **Output.** The allocated CVE ID as a string in canonical + `CVE-YYYY-NNNN+` form (`CVE-2026-12345`). If the adapter cannot + allocate synchronously (the MITRE-form case), it returns a + provisional placeholder string and the skill stores that on the + tracker as `cve allocation pending`; the next `fetch_current_state` + call upgrades it once the real ID arrives. +- **No-op case.** The `none` adapter raises `NotApplicable` and + the skill skips allocation entirely. The `security-cve-allocate` + skill detects this and posts a "this adopter does not allocate + CVEs; closing without an ID" comment on the tracker instead. + +### `fetch_current_state(cve_id) to {state, fields}` + +Read the current record state and the fields the tool stores. + +- **Lifecycle moment.** Fires inside `security-issue-sync` at + Step 1e (the reviewer-comment sync) and Step 5b (the + state-progression gate before the release-manager hand-off). + Also fires from `security-issue-invalidate`'s Step 0 gate, to + refuse closure when a record exists and is past `review-ready`. +- **Inputs.** The CVE ID. Nothing else — the adapter is responsible + for resolving the tool-specific URL or API call from the ID. +- **Output.** A two-field dict: + - `state` — one of the generic state verbs defined below + (`allocated`, `review-ready`, `publish-ready`, `public`, + `retracted`, `unknown`). The adapter is responsible for + mapping its tool's native state vocabulary onto these verbs. + - `fields` — a dict of the fields the tool currently stores + for the record. Shape matches the `allocate` input dict; the + skills diff this against the tracker body to detect drift. +- **No-op case.** The `none` adapter returns `{state: "unknown", + fields: {}}`. Skills treat `unknown` as "do not gate progression + on tool state"; they fall back to the tracker body's view. + +### `push_update(cve_id, fields, state_transition=None) to diff` + +Write field updates to the record, optionally with a state move. + +- **Lifecycle moment.** Fires inside `security-issue-sync` at + Step 5c, whenever the tracker body has drifted from what + `fetch_current_state` returned. Also fires from + `security-issue-deduplicate` when the kept tracker absorbs + credits or references from the merged-in tracker. +- **Inputs.** + - `cve_id` — as for `fetch_current_state`. + - `fields` — the new desired fields. The adapter is responsible + for translating these into whatever wire format its tool + accepts (JSON record for Vulnogram, web-form fields for the + MITRE form, GraphQL mutation for GHSA). + - `state_transition` *(optional)* — one of the generic state + verbs. When supplied, the adapter performs the state move as + part of the same write; when omitted, the adapter only writes + fields and leaves state untouched. The Vulnogram adapter today + embeds the state inside the JSON body and so does both in one + PUT; other adapters may need a separate API call after the + field write. The contract does not constrain how — only that + the call appear atomic from the skill's point of view. +- **Output.** A diff dict — `{added: [...], removed: [...], + changed: [...]}` — that the skill can show the user as part of + the sync confirmation. Empty diff means the record was already + up to date; the skill suppresses the confirmation prompt in + that case. +- **No-op case.** The `none` adapter returns `{added: [], removed: + [], changed: []}` and writes nothing. The `unknown`-state + return from `fetch_current_state` also causes the skill to skip + `push_update` — the adapter has nowhere to push to. + +### `publish(cve_id) to ok` + +Move the record to its terminal public state. + +- **Lifecycle moment.** Fires inside `security-issue-sync` at + the publication step, **after** the public advisory archive URL + has been captured on the tracker. The captured archive URL is + the real-world signal that the advisory has actually shipped to + the users-list; before that point the adapter would be + publishing a record before the world has heard about the issue, + which is the opposite of the desired ordering. +- **Inputs.** The CVE ID. +- **Output.** `ok` (a sentinel; the skill does not inspect the + return body — failure raises). The skill follows up with + `fetch_current_state` to confirm the state landed at `public` + and posts a confirmation comment on the tracker. +- **No-op case.** The `none` adapter raises `NotApplicable`. The + skill treats this as "this adopter does not publish CVEs; the + tracker close is the only terminal action" and proceeds to the + archive-from-board step directly. The `ghsa` adapter implements + this as a GraphQL `publishSecurityAdvisory` mutation. + +### `retract(cve_id, reason) to ok` + +Mark an allocated-but-not-public record as rejected. + +- **Lifecycle moment.** Fires inside `security-issue-invalidate` + for trackers that already carry a CVE ID. CVE retraction is + governance-sensitive; the skill requires explicit confirmation + from a release-vote-gating role (`governance.cve_allocation_gate` + from `project.md`) before invoking `retract`. The retracted + state is terminal — once a record has been retracted, no + subsequent `push_update` or `publish` is valid. +- **Inputs.** + - `cve_id` — as above. + - `reason` — a short string captured from the tracker + discussion explaining the retraction. The adapter is + responsible for writing this into the appropriate field of + its tool's record (Vulnogram's `CNA_private.justification`, + CVE.org's `rejectedReason`, etc.). +- **Output.** `ok` sentinel as for `publish`. +- **No-op case.** The `none` adapter raises `NotApplicable`. The + skill falls back to "close the tracker as invalid; there is no + CVE record to retract." For adapters whose tool does not + support retraction of `public` records (Vulnogram refuses this; + CVE.org has a separate REJECT flow), the adapter must raise a + distinguishable `AlreadyPublic` error and the skill escalates + to the configured governance contact. + +## Generic state verbs + +The skills speak in generic verbs about a CVE record's lifecycle. +The adapter is responsible for mapping its tool's native states +onto these verbs. The verbs are: + +| Verb | Meaning | Vulnogram-native state | +|---|---|---| +| `allocated` | Record exists, ID is reserved, content is being filled in. Not visible publicly. | `DRAFT` | +| `review-ready` | Record content is complete and ready for CNA review. Reviewer comments may arrive at this state. Not visible publicly. | `REVIEW` | +| `publish-ready` | Record content is final, reviewer comments addressed, staged for the advisory-send step. The advisory emails are dispatched from the CVE tool while in this state. Not visible publicly. | `READY` | +| `public` | Record pushed to `cve.org` and world-readable. Terminal in the success path. | `PUBLIC` | +| `retracted` | Record marked rejected post-allocation. Terminal in the failure path. | `REJECTED` (in `CNA_private.state`) | +| `unknown` | The adapter cannot determine the state (network failure, asynchronous tool that hasn't replied yet, `none` backend). Skills treat this as "fall back to the tracker body's view." | n/a | + +The map is **adapter-internal**. Skills never write `DRAFT` or +`REVIEW` — they write `allocated` and `review-ready`. An adapter +that needs a more granular internal model is free to introduce +sub-states inside its `tool/` directory, as long as the +contract-facing methods normalise on the verbs above. + +The `DRAFT` to `REVIEW` transition is sync-driven (the tracker body +fields determine readiness) in the Vulnogram adapter; the +`REVIEW` to `READY` transition is release-manager-driven because the +adapter cannot tell from the tracker body alone whether reviewer +comments are still pending. Other adapters may collapse these two +transitions into one (the GHSA adapter has no separate +review/publish-ready distinction) or split them further (a +hypothetical multi-stage CNA tool with `REVIEW` to `ADDRESSING` → +`READY`). The contract does not constrain how the adapter maps +its internal states onto the four generic verbs — only that the +verbs are what the skills see. + +## Skills that consume this contract + +The CVE-tool contract is consumed by four skills today: + +- **`security-cve-allocate`** — calls `allocate` to reserve the + CVE ID and create the initial record, then writes the ID back + onto the tracker's *CVE tool link* body field. The skill is + PMC-gated (or `governance.cve_allocation_gate`-gated for + non-ASF adopters); the adapter does not enforce that gate + itself — the skill does. +- **`security-issue-sync`** — calls `fetch_current_state` at + Step 1e (to surface reviewer-comment signals) and at Step 5b + (to gate the release-manager hand-off on the post-push state + being `review-ready`). Calls `push_update` at Step 5c whenever + the tracker body has drifted from the tool's view. Calls + `publish` at the publication step after the public advisory + archive URL has been captured. +- **`security-issue-invalidate`** — uses `fetch_current_state` at + its Step 0 gate to refuse closure when a CVE record exists and + is past `review-ready`. Calls `retract` when the tracker + carries a CVE ID and the consensus decision is "invalid after + allocation" (escalates first via `governance.escalation_contact`, + since CVE retraction has public consequences). +- **`security-issue-deduplicate`** — calls `push_update` on the + kept tracker's CVE record after merging in credits and + references from the duplicate tracker. The duplicate tracker's + CVE record (if any) is the subject of a separate `retract` + call once the dedup decision has been confirmed. + +These four skills are the only consumers in the current shape of +the framework. Future skills that need to touch a CVE record +(e.g. a hypothetical `security-cve-rotate-credit` action) will +extend the contract rather than reaching past it into a specific +adapter. + +## ASF default — Vulnogram + +The ASF reference adapter lives at [`tools/vulnogram/`](../vulnogram/). +It is the only adapter shipping today, and the only adapter the +skills are tested against. Key properties of the ASF default: + +- **URL prefix:** `https://cveprocess.apache.org/cve5/`. + The record page, `#source` tab, `#json` tab, and `#email` tab + are all rooted there. See [`tools/vulnogram/record.md`](../vulnogram/record.md#record-urls) + for the canonical URL table. +- **Email preview URL:** `https://cveprocess.apache.org/cve5/#email`. + Renders the advisory exactly as Vulnogram will dispatch it to + the users-list and announce-list — same subject, same body, + same recipient list. The release-manager checklist calls this + out as a load-bearing checkpoint before the advisory-send step. +- **Source tab URL:** `https://cveprocess.apache.org/cve5/#source`. + The copy-paste fallback target for the JSON record. The default + write path is the OAuth-authenticated API (`vulnogram-api-record-update`); + copy-paste is the documented fallback when the OAuth session + has expired or the operator has opted out. +- **State machine:** `DRAFT` to `REVIEW` to `READY` to `PUBLIC`, + carried inside `CNA_private.state` on the CVE 5.x record. The + adapter maps these onto the generic verbs as shown in the + state-verb table above. +- **Reviewer-comment channel:** mailing-list (the ASF CNA + reviewers email the project's `security_list` rather than + surfacing comments on the tracking issue directly). This is + the `cve_authority.reviewer_channel: mailing-list` setting in + `project.md`; an adapter could equally well declare + `reviewer_channel: github-pr` (for a CNA tool that uses pull + requests as its review surface) or `reviewer_channel: none` + (for a CNA tool with no separate review step). +- **Publication propagation:** poll (the skills check the public + advisory archive on every sync rather than waiting for a + webhook). This is the `cve_authority.publication_propagation: + poll` setting in `project.md`. + +**Rename pending.** PR4 of the ASF-pluggable security flow renames +`tools/vulnogram/` to `tools/cve-tool-vulnogram/` to match the +contract's adapter-naming rule. **This PR does not perform the +rename** — the existing skill prose and the `cve_authority.tool: +vulnogram` setting in `project.md` continue to point at +`tools/vulnogram/` until PR4 lands. + +## Configuration + +Every adopter declares its CVE-tool choice in +`projects//project.md` under the `cve_authority` block. +The shape is: + +```yaml +cve_authority: + tool: vulnogram # vulnogram | cve-org-direct | mitre-form | ghsa | none + allocate_url: "https://cveprocess.apache.org/cve5/new" + record_url_template: "https://cveprocess.apache.org/cve5/{cve_id}" + source_tab_url_template: "https://cveprocess.apache.org/cve5/{cve_id}#source" + email_preview_url_template: "https://cveprocess.apache.org/cve5/{cve_id}#email" + states: [allocated, review-ready, publish-ready, public] + publication_propagation: poll # poll | webhook | manual + emits_allocation_email: true + reviewer_channel: mailing-list # mailing-list | github-pr | none +``` + +Field-by-field: + +- **`tool`** — names the adapter directory the skills resolve to. + The ASF default is `vulnogram` (resolves to `tools/vulnogram/`; + becomes `tools/cve-tool-vulnogram/` after PR4). Adopters using + a different CVE-tool backend pick one of the other four + enumerated values, each of which is expected to resolve to a + sibling `tools/cve-tool-/` directory. +- **`allocate_url`** — the URL the human operator opens to begin + the allocation flow (for tool-mediated allocation paths) or + the API endpoint the adapter POSTs to (for fully automated + paths). The Vulnogram default is the `/cve5/new` form; the + MITRE-form default would be `https://cveform.mitre.org/`. +- **`record_url_template`** — the per-record URL pattern. The + `{cve_id}` placeholder is the only token the skills substitute. + Skills use this for the "open the CVE record" links they post + on the tracker. +- **`source_tab_url_template`** — the copy-paste fallback target. + Optional for adapters that have no copy-paste fallback (e.g. + GHSA — there is no JSON form to paste into). When the field is + null, the skills suppress the copy-paste fallback proposal. +- **`email_preview_url_template`** — the advisory-email preview + URL. Optional for adapters whose CVE tool does not dispatch + the advisory itself; when null, the release-manager checklist + omits the preview step. The ASF default points at the `#email` + tab. +- **`states`** — the ordered list of generic state verbs the + adapter exposes. Adapters with fewer states (the GHSA adapter + collapses `review-ready` and `publish-ready` into a single + pre-publish state; the `none` adapter has only `allocated` and + `unknown`) declare a shorter list. Skills branch on the + declared list when deciding which lifecycle steps apply. +- **`publication_propagation`** — how the skills learn that a + record has reached `public`. `poll` (the ASF default) means + the skills check the public archive on every sync; `webhook` + means an external hook updates the tracker directly; `manual` + means a human flips the tracker label and the skills trust it. +- **`emits_allocation_email`** — whether the CVE tool sends an + allocation-confirmation email at `allocate` time. The ASF + default is `true` (Vulnogram emails the project's + `security_list` on every allocation); the MITRE-form default + is also `true` (the form replies with the allocated ID); + GHSA's default is `false` (no email — the allocation result is + the API response). Skills that wait on this email surface a + "do not close this tracker until the allocation email lands" + hint when the flag is `true`. +- **`reviewer_channel`** — where reviewer comments arrive. + `mailing-list` (ASF default) means `security-issue-sync` reads + reviewer comments off the security mailing list; `github-pr` + means it reads them off a backing PR; `none` means there is + no separate review step. + +The contract does not constrain how the adapter implements any of +these settings — only that the settings are present and that the +adapter respects them. Adapters are free to add their own +tool-specific sub-keys under `cve_authority.:` (e.g. +`cve_authority.vulnogram.asf_org_id`, `cve_authority.vulnogram.cna_private_owner`) +for fields the contract does not surface. diff --git a/tools/forwarder-relay/README.md b/tools/forwarder-relay/README.md new file mode 100644 index 00000000..909415b6 --- /dev/null +++ b/tools/forwarder-relay/README.md @@ -0,0 +1,403 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [tools/forwarder-relay/ — adapter contract](#toolsforwarder-relay--adapter-contract) + - [What "a relay message" means](#what-a-relay-message-means) + - [Today's adapters](#todays-adapters) + - [Interface](#interface) + - [`detect(message) -> adapter_name | null`](#detectmessage---adapter_name--null) + - [`extract_credit(body) -> {name, kind, raw_string} | null`](#extract_creditbody---name-kind-raw_string--null) + - [`contact_handle` (attribute)](#contact_handle-attribute) + - [`preamble_match` (attribute)](#preamble_match-attribute) + - [`reporter_addressing_block(...) -> string`](#reporter_addressing_block---string) + - [`via_forwarder_question_mode` (attribute)](#via_forwarder_question_mode-attribute) + - [Skills that consume this contract](#skills-that-consume-this-contract) + - [ASF default — ASF Security forwarder](#asf-default--asf-security-forwarder) + - [Why `@raboof` is the contact handle today](#why-raboof-is-the-contact-handle-today) + - [Configuration](#configuration) + - [Cross-references](#cross-references) + - [What this contract does NOT cover](#what-this-contract-does-not-cover) + + + + + +# tools/forwarder-relay/ — adapter contract + +**Capability:** capability:setup + +A forwarder-relay adapter is a pluggable seam that teaches the +security skills how to recognise an inbound report that arrived +**through a relay** (someone else forwarded the original +reporter's message to the project), how to extract the +original-reporter credit from the relayed body, and how to route +reporter-facing drafts back through the same relay channel. This +file defines what the skills expect from an adapter, what the +single shipping adapter (ASF Security relay) does today, and how +adopters declare which adapters are enabled in +[`/project.md`](../../projects/_template/project.md). + +The framework default is the ASF Security relay adapter, which is +the only one shipping in the tree today. The contract exists so +that adopters whose security inbox sits behind huntr.com, +HackerOne, GitHub Security Advisories, an internal SOC, or any +other forwarding service can plug in an adapter without +patching the skill bodies. + +## What "a relay message" means + +Many security reports never reach the project's security inbox +directly. The original reporter files with a third-party broker — +the ASF Security team at `security@apache.org`, huntr.com, +HackerOne, GHSA — and the broker forwards the report to the +project. On the inbound thread, the broker is the visible +correspondent; the actual reporter is one hop away, reachable +only by asking the broker to relay messages back. + +This matters for three skill behaviours: + +1. **Credit extraction.** The `From:` header of a relay message + names the broker, not the reporter. Per + [`docs/security/reporter-credit-policy.md`](../../docs/security/reporter-credit-policy.md) + the tracker's *Reporter credited as* field must name the + external reporter, so the skill has to pull the name from the + message body (the broker's preamble convention) instead of + from the header. +2. **Reply routing.** Drafts intended for the reporter must go + through the broker — but addressed to the broker, with the + reporter-facing content folded inside as a paste-ready block + the broker can copy verbatim into their reply to the reporter. + See [`tools/gmail/asf-relay.md`](../gmail/asf-relay.md) for + the paste-ready-block convention introduced in PR #375. +3. **Question batching.** The project should not pester the + broker with every workflow chatter event. The + [`docs/security/forwarder-routing-policy.md`](../../docs/security/forwarder-routing-policy.md) + policy doc — which consumes this contract — defines which + milestones get relayed and which stay on the project side. + +A forwarder-relay adapter is the seam that lets each skill ask +the right adapter "is this a relay message? whose credit is in +it? how do I draft back through it?" without hard-coding the +ASF preamble or the `@apache.org` sender pattern. + +## Today's adapters + +| Adapter | Status | Inbound channel | Reference doc | +|---|---|---|---| +| `asf-security` | shipping | Mail from `security@apache.org` or a personal `@apache.org` address with the ASF forwarding preamble | [`tools/gmail/asf-relay.md`](../gmail/asf-relay.md) | +| `huntr-relay` | placeholder | Mail from huntr.com's notification address with the huntr disclosure preamble | not implemented — contract slot only | +| `hackerone-relay` | placeholder | Mail from HackerOne's notification address with the HackerOne handoff preamble | not implemented — contract slot only | +| `ghsa-relay` | placeholder | Mail forwarded from a GHSA private report by the GitHub notification system | not implemented — contract slot only | +| `custom` | escape hatch | Adopter-specific broker (internal SOC, third-party VRP, mailing-list relay) | adopter ships their own adapter dir | + +Only `asf-security` is wired in. The other rows are reserved +contract slots: when an adopter needs huntr.com or HackerOne +support, they implement an adapter directory under +`tools/forwarder-relay//` that satisfies the interface +below, and add `` to the `forwarders.enabled` list in +their `/project.md`. + +## Interface + +A forwarder-relay adapter exposes the following operations. Skills +dispatch through this interface; they do not import adapter +internals directly. + +### `detect(message) -> adapter_name | null` + +Given a fetched mail-source message (`From`, `Subject`, +`Body`, `Date`, `Message-ID`, headers), return the adapter's +own name if the message is a relay message handled by this +adapter, or `null` if not. + +Detection is the OR of two signals: + +* **Sender pattern.** A regex or set-membership check against + the `From:` address. ASF Security: `security@apache.org` OR + any `*@apache.org` address. huntr: huntr.com's outbound + notification address. HackerOne: HackerOne's notification + address. +* **Preamble match.** A regex against the first ~400 characters + of the message body, anchored to the broker's standard + forwarding header. ASF Security: *"Dear PMC, The security + vulnerability report has been received by the Apache Security + Team …"*. + +Both signals are evaluated; either one matching is sufficient, +but the adapter MAY require both for higher-confidence cases. +The skill calls each enabled adapter's `detect()` in the order +listed under `forwarders.enabled`; first non-null wins. + +**Lifecycle:** called by `security-issue-import` Step 3 +(classification), and by `security-issue-sync` Step 2b when +re-reading an existing tracker's inbound thread for routing +decisions. + +### `extract_credit(body) -> {name, kind, raw_string} | null` + +Given the relay-message body, extract the original reporter's +credit. Returns: + +* `name` — the human-readable reporter name as it appears in + the body. Used verbatim in the tracker's *Reporter credited + as* field unless the reporter later requests a different + rendering through a credit-preference exchange. +* `kind` — one of `human` (named individual), `tool` + (automated scanner like `bugbunny.ai`, `protectai/modelscan`), + `service` (a broker / VRP / SOC operating on someone else's + behalf). Drives the bot-credit policy gate in + [`tools/vulnogram/bot-credits-policy.md`](../vulnogram/bot-credits-policy.md). +* `raw_string` — the exact substring lifted from the body + (e.g. *"This vulnerability was discovered and reported by + bugbunny.ai"*). Stored so a later sync can diff against the + current tracker field and detect that the reporter has been + manually overridden. + +Returns `null` when the body does not contain a credit line in +the adapter's expected shape. The skill then surfaces a "credit +unknown — please confirm before drafting the receipt" prompt +rather than guessing. + +**Lifecycle:** called by `security-issue-import` Step 4 (field +population for the new tracker body), and by +`security-issue-sync` Step 2b when reconciling the tracker +field against the latest read of the thread. + +### `contact_handle` (attribute) + +The GitHub-style handle (or back-channel identifier) of the +relay contact the skills should `@mention` when proposing a +draft. For ASF Security this is currently `@raboof` (Arnout +Engelen, the on-duty ASF Security liaison); for huntr the +handle would be huntr's program-issued contact. Lifted into +config now so the skill body never hard-codes a name. + +The handle MAY be a list of fallbacks (`[@raboof, +@securityasf-rota]`) for adapters whose contact rotates; the +skill picks the first available one and surfaces the chosen +handle in the proposal recap. + +### `preamble_match` (attribute) + +The regex used by `detect()`. Exposed as a read-only attribute +so that: + +* `security-issue-import` can print the matched preamble in its + Step 3 classification proposal ("detected as ASF Security + relay because the body starts with ``"), + giving the human reviewer a one-line "yes this looks right" + affordance; +* the test harness in + [`tools/skill-and-tool-validator/`](../skill-and-tool-validator/) + can replay sample bodies through the adapter and assert the + detect outcome. + +### `reporter_addressing_block(...) -> string` + +Render the paste-ready block convention introduced in +[`tools/gmail/asf-relay.md`](../gmail/asf-relay.md) — the +fenced block addressed to the reporter in the project's voice, +signed *" security team"*, inside a wrapper addressed +to the forwarder asking them to forward it verbatim. The +adapter owns the exact wrapper wording; the inner reporter +block is built by the calling skill and passed in. + +Parameters: + +* `forwarder_first_name` — for the *"Hi "* salutation + on the wrapper. +* `reporter_first_name` — for the *"Hello "* salutation + on the inner block, when known. +* `links` — list of `(label, url)` pairs (GHSA URL, CVE + record URL, advisory URL) the wrapper prints near the top + so the forwarder can one-click context-switch on their side. +* `inner_body` — the project's reporter-facing text. The + adapter wraps it in the paste-ready block; it does not + modify the inner content. + +The adapter's responsibility is the wrapper shape: where the +links go, how the inner block is fenced, how the signature +attaches. The reason this is in the adapter rather than in +the skill is that different brokers have different forwarding +conventions — huntr.com expects the inner block to be a +markdown comment that pastes back into their UI; ASF Security +expects a `---`-fenced plaintext block. One skill body, many +adapter renderings. + +**Lifecycle:** called by `security-issue-import` Step 7 +(receipt-of-confirmation draft), by `security-cve-allocate` +Step 4 (CVE-allocation notification draft), by +`security-issue-sync` Step 2b (status-update drafts), and by +`security-issue-invalidate` Step 5d (invalidation notice draft). + +### `via_forwarder_question_mode` (attribute) + +A boolean signalling how the adapter prefers credit-preference +questions to be handled: + +* `true` — fold the credit-preference question into the + **same** receipt-of-confirmation draft, addressed to the + reporter via the paste-ready block. The forwarder makes one + forward-and-paste action total. This is the right default + for adapters where the broker prefers not to be a question + router (ASF Security: yes). +* `false` — send the credit-preference question on a + **separate** draft, framed as a back-channel request to the + forwarder (*"please ask the reporter their credit + preference"*). This is appropriate for adapters where the + broker actively reviews each exchange (some HackerOne + programs). + +The skill body branches on this attribute in +`security-issue-import` Step 7 and `security-cve-allocate` +Step 4 instead of carrying an `if asf_relay:` check inline. + +## Skills that consume this contract + +| Skill | Step | What the skill calls | +|---|---|---| +| [`security-issue-import`](../../.claude/skills/security-issue-import/SKILL.md) | Step 3 — classification | `detect()` on every enabled adapter; the first non-null return classifies the candidate as a relay import | +| `security-issue-import` | Step 4 — field population | `extract_credit()` for the *Reporter credited as* field | +| `security-issue-import` | Step 7 — receipt-of-confirmation draft | `reporter_addressing_block()` + `via_forwarder_question_mode` to fold the credit-preference question | +| [`security-issue-sync`](../../.claude/skills/security-issue-sync/SKILL.md) | Step 2b — draft routing | `contact_handle` + `reporter_addressing_block()` for any reporter-facing draft (CVE-allocated, fix-merged, advisory-shipped) on a relay tracker | +| [`security-issue-invalidate`](../../.claude/skills/security-issue-invalidate/SKILL.md) | Step 5d — ASF-relay branch | `reporter_addressing_block()` for the polite-but-firm invalidation notice routed through the forwarder | +| [`security-cve-allocate`](../../.claude/skills/security-cve-allocate/SKILL.md) | Step 4 — dual-mode draft | `via_forwarder_question_mode` to decide whether the CVE-allocation draft folds in the credit-preference ask or sends it separately | +| [`tools/gmail/asf-relay.md`](../gmail/asf-relay.md) | reference doc for the shipping adapter | this whole file is the formal contract that `asf-relay.md` documents prose-style | + +## ASF default — ASF Security forwarder + +The ASF Security adapter is the one shipping today. Its +parameter values, lifted out of skill bodies and into +`/project.md`, are: + +```yaml +forwarders: + enabled: + - asf-security + asf-security: + sender_pattern: '(security@apache\.org|.+@apache\.org)' + preamble_match: 'Dear PMC,\s+The security vulnerability report has been received by the Apache Security Team' + credit_extraction_rule: + # The ASF forwarding template ends with a credit line in one of these shapes. + patterns: + - 'This vulnerability was discovered and reported by (?P.+?)\.' + - 'Credit:\s+(?P.+?)$' + - 'Reported by:\s+(?P.+?)$' + kind_hints: + # Substring matches on the extracted name to classify kind. + tool: ['\.ai\b', 'bot\b', 'scanner\b'] + service: ['security team\b', 'soc\b'] + # default: human + contact_handle: '@raboof' # ASF Security on-duty liaison; lift to a rota when one exists + via_forwarder_question_mode: true + reporter_addressing_block: + wrapper_salutation: 'Hi ,' + links_section: true # GHSA / CVE / advisory URLs on their own lines near the top + fence: '---' # `---`-fenced plaintext block + inner_salutation: 'Hello ,' + inner_signature: ' security team' + wrapper_signoff: 'Thanks,\n' +``` + +The exact shape of the paste-ready block is defined in +[`tools/gmail/asf-relay.md`](../gmail/asf-relay.md) under the +*"Reporter-facing content goes as a ready-to-paste block, not as +a third-person ask"* rule, with the worked GHSA / CVE example. + +### Why `@raboof` is the contact handle today + +Arnout Engelen (`@raboof`, `engelen@apache.org`) is the ASF +Security team member who currently triages relayed reports for +the projects this framework's reference adopter belongs to. The +handle is lifted into config rather than hard-coded so that: + +* a future on-duty rota can declare a list (`[@raboof, + @next-on-duty, @asf-security-rota]`) without touching skill + bodies; +* adopters whose ASF Security liaison is a different individual + declare their own handle locally; +* the handle is reviewable in one place during a security-team + rotation hand-off instead of being scattered across draft + templates. + +## Configuration + +The adopter declares enabled adapters in +[`/project.md`](../../projects/_template/project.md) +under the `forwarders` block: + +```yaml +forwarders: + enabled: + - asf-security # default; ships with framework + # - huntr-relay # placeholder — uncomment when implemented + # - hackerone-relay # placeholder — uncomment when implemented + asf-security: + contact_handle: '@raboof' + via_forwarder_question_mode: true + # sender_pattern / preamble_match / credit_extraction_rule + # inherit framework defaults unless the adopter overrides +``` + +The framework ships sensible defaults for every key under +`asf-security`. An adopter typically only overrides +`contact_handle` (their liaison) and possibly the +`sender_pattern` (if they accept relays from a wider set of +addresses than just `*@apache.org`). + +## Cross-references + +* **Policy** — + [`docs/security/forwarder-routing-policy.md`](../../docs/security/forwarder-routing-policy.md) + defines *when* via-forwarder mode applies on a tracker and + *which* milestones get relayed. The adapter contract here is + the mechanism; that doc is the policy that drives it. +* **Drafting convention** — + [`tools/gmail/asf-relay.md`](../gmail/asf-relay.md) carries + the prose explanation of the paste-ready block, the clickable + external-reference URL rule, and the threading semantics for + relay drafts. The contract surfaces those rules as an + interface; the prose file remains the human-readable + reference for the shipping adapter. +* **Bot-credit gate** — + [`tools/vulnogram/bot-credits-policy.md`](../vulnogram/bot-credits-policy.md) + reads the `kind` field returned by `extract_credit()` to + decide whether a CVE record should list the credit as a tool + / organisation rather than an individual. +* **Mail-source layer** — this contract sits on top of the + abstract mail operations defined in + [`tools/mail-source/contract.md`](../mail-source/contract.md); + the forwarder-relay adapter consumes a message returned by + the mail-source layer and produces routing metadata. It does + not itself fetch or send mail. + +## What this contract does NOT cover + +* **Detection of GHSA / private-reporting trackers without an + inbound relay message.** Those are handled by the + `` marker + documented in + [`docs/security/forwarder-routing-policy.md`](../../docs/security/forwarder-routing-policy.md). + The contract is for adapters that recognise a message; the + marker is for trackers with no message at all. +* **Send semantics.** Drafts produced via + `reporter_addressing_block()` are handed back to the + mail-source layer's `create_draft` operation; this contract + does not send mail. The framework rule remains *draft, never + send*. +* **Tracker field schema.** The names of the tracker body + fields (*Reporter credited as*, *Security mailing list + thread*, etc.) are declared in + [`/project.md`](../../projects/_template/project.md) + under the `tracker.body_fields` block. The adapter returns + values; the tracker decides where to write them. +* **Multi-thread reconciliation.** When a tracker records both + a direct reporter thread and a separate relay thread, the + primary-vs-relay selection rule lives in + [`tools/gmail/threading.md`](../gmail/threading.md) — + *Selecting the inbound thread when multiple are recorded*. + The adapter contract assumes one inbound message at a time + and lets the threading layer decide which message to ask + about. diff --git a/tools/mail-archive/README.md b/tools/mail-archive/README.md new file mode 100644 index 00000000..0fb61b67 --- /dev/null +++ b/tools/mail-archive/README.md @@ -0,0 +1,343 @@ + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [tools/mail-archive/](#toolsmail-archive) + - [Today's adapters](#todays-adapters) + - [Interface](#interface) + - [`search_thread_url(list, year, month, query) to string`](#search_thread_urllist-year-month-query-to-string) + - [`fetch_thread_by_url(url) to thread_data | null`](#fetch_thread_by_urlurl-to-thread_data--null) + - [`list_recent_threads(list, since) to [thread_summary]`](#list_recent_threadslist-since-to-thread_summary) + - [`resolve_advisory_announcement_url(list, advisory_id) to string | null`](#resolve_advisory_announcement_urllist-advisory_id-to-string--null) + - [`publication_signal_url(list) to string`](#publication_signal_urllist-to-string) + - [Skills that consume this contract](#skills-that-consume-this-contract) + - [ASF default — PonyMail](#asf-default--ponymail) + - [Configuration](#configuration) + + + + + +# tools/mail-archive/ + +**Capability:** capability:setup + +This file defines the adapter contract for **public mail-archive +backends** — the seam that lets adopting projects plug a non-ASF +archive system (Hyperkitty, Discourse, Google Groups, GitHub +Discussions, or none at all) into the same skills that today reach +straight into `lists.apache.org` via the PonyMail MCP. The contract +declares *verbs* that the generic skill bodies call, with input and +output shapes that every backend must satisfy; the concrete URL +construction, authentication model, and search syntax stay inside +each adapter directory. + +The contract exists because the skills currently hard-code two +ASF-specific assumptions in their step bodies: + +1. They call `mcp__ponymail__search_list`, `mcp__ponymail__get_thread`, + `mcp__ponymail__list_recent_threads`, and friends by name — the + `mcp__ponymail__*` prefix is woven into roughly two dozen step + bodies across `security-issue-import`, `security-issue-sync`, and + `security-issue-invalidate`. +2. They construct `https://lists.apache.org/...` URLs inline (the + `list?:YYYY-M:` search form, the + `api/thread.lua?...` JSON form, the + `thread/?` resolved form, and the + `list.html?` advisory-published form) from + `` / `` / `` placeholders that come + from the project manifest. + +Both assumptions are documented today in +[`tools/gmail/ponymail-archive.md`](../gmail/ponymail-archive.md) and +in [`tools/ponymail/operations.md`](../ponymail/operations.md). PR1 +introduces this contract so that PR3 can rename `tools/ponymail/` +to `tools/mail-archive-ponymail/` without disturbing the skills, and +so that later PRs can refactor every `mcp__ponymail__*` call site +and every `lists.apache.org` URL template to flow through the verbs +declared here. + +ASF projects keep the existing PonyMail behaviour by default. A +non-ASF adopter declares a different `archive_system.kind` in +`projects//project.md` and the adapter for that kind is +loaded in place of PonyMail. Skills do not change. + +## Today's adapters + +| Adapter | Status | Source | Notes | +|---|---|---|---| +| `ponymail` | shipping | [`tools/ponymail/`](../ponymail/) (will be renamed to `tools/mail-archive-ponymail/` in PR3) | ASF's `lists.apache.org` deployment; PMC LDAP OAuth for private lists; anonymous read for public lists; load-bearing for the `security-issue-import`, `security-issue-sync`, and `security-issue-invalidate` skills today. | +| `hyperkitty` | placeholder | not implemented | Mailman 3's archive UI. Same conceptual surface as PonyMail (per-list archive, search by month + query, per-thread permalink) but a different URL grammar and a different JSON API. | +| `discourse` | placeholder | not implemented | Topic-based forum, with mailing-list-mode bridging. The verbs map onto Discourse's `/search.json` and `/t//.json` endpoints. | +| `google-groups` | placeholder | not implemented | `groups.google.com` archives. Lacks a stable JSON read API; an adapter likely degrades to user-paste for the per-thread URL, the same way PonyMail does for private lists. | +| `github-discussions` | placeholder | not implemented | GitHub's discussions feature, accessed via the GraphQL API the `gh` CLI already wraps. Useful for projects whose announcement channel is a `discussions` category rather than a mailing list. | +| `none` | placeholder | not implemented | Explicit *"no archive backend"* declaration. Every verb returns `null` / no-op; the consuming skill falls back to user-paste or to a *"not available"* note in the tracker body field. Useful for projects that do not host their security discussions on an archived list at all (private-only intake forms, chat-channel intake). | + +The ASF default adapter is the only one that ships today. Every +placeholder above is named, with a one-paragraph justification, so +that an adopter who needs that backend can author the adapter +without re-inventing the contract. + +## Interface + +Every adapter exposes the verbs below. Each verb declares: + +- **When it fires** — the skill lifecycle point that calls the verb. +- **Inputs** — the typed arguments. +- **Output shape** — the return type, including the `null` / no-op + shape when the backend cannot resolve the request. + +The output shapes are documented in conceptual terms rather than as +a strict JSON schema; an adapter is free to return a language-native +object as long as the consuming skill can read the named fields. + +### `search_thread_url(list, year, month, query) to string` + +**When it fires.** Step 5 of `security-issue-import` — the skill +needs a one-click URL the user can open in their authenticated +browser to land on the inbound report's archive page. Also fires +inside `security-issue-invalidate` when the relay-rewritten +inbound thread needs to be located for the closing reply. + +**Inputs.** + +| Arg | Type | Notes | +|---|---|---| +| `list` | string | The fully-qualified mailing-list address. For ASF: `security@.apache.org`. Adapters that don't use email addresses (Discourse, GitHub Discussions) interpret this as the channel identifier. | +| `year` | integer | Four-digit Gregorian year, e.g. `2026`. | +| `month` | integer | 1–12. The adapter is responsible for formatting (PonyMail wants `YYYY-M`, no leading zero; Hyperkitty wants `YYYY-MM`). | +| `query` | string | The search query in the adapter's native dialect — typically the report subject for `security-issue-import`, the CVE ID for advisory scans. The adapter is free to URL-encode, escape, or transform as needed. | + +**Output.** A complete URL string that a human can open in a +browser to land on the search results. The skill proposes this URL +to the user at Step 5 of `security-issue-import` and waits for the +user to paste back the resolved per-thread URL. + +**No-op case.** Adapters that cannot generate a search URL (the +`none` adapter, or a backend that gates search behind an interactive +session) return an empty string. The skill treats an empty return +as *"no search URL available, fall back to user-paste with no +prompt URL"*. + +### `fetch_thread_by_url(url) to thread_data | null` + +**When it fires.** Step 1c / 1e / 1h / 2b of `security-issue-sync` +— when the tracker already carries an archive thread URL and the +sync skill wants to re-read the discussion for new mailing-list +activity since the last sync. Also fires inside `security-issue-import` +when the user pastes a URL back and the skill wants to verify the +URL resolves before recording it in the tracker body field. + +**Inputs.** + +| Arg | Type | Notes | +|---|---|---| +| `url` | string | A URL the adapter previously produced via `search_thread_url` or via `resolve_advisory_announcement_url`. The adapter is responsible for parsing its own URL grammar. | + +**Output.** A `thread_data` object with at least: + +- `thread_id` — the adapter's opaque per-thread identifier. +- `list` — the list address (echoed from the URL for adapter consistency). +- `subject` — the thread subject. +- `messages[]` — an array of `{message_id, from, date, body, in_reply_to}` records, ordered by date. +- `participant_handles[]` — every distinct sender, formatted as the adapter's native handle (email address for mailing-list adapters; `@user` for Discourse; `@user` for GitHub Discussions). + +**No-op case.** Returns `null` when: + +- The URL is well-formed but the thread no longer exists (deleted / + retracted / archive purged). +- The URL is well-formed but requires authentication the adapter + doesn't have (the `ponymail` adapter against a private list when + the user has not run `mcp__ponymail__login`). +- The URL is malformed (parsing failure). + +Skills handle a `null` return by surfacing the gap to the user at +the next sync — they do not retry automatically. + +### `list_recent_threads(list, since) to [thread_summary]` + +**When it fires.** Periodic-sweep bodies (`security-issue-import` +when it scans `security@` for unimported threads; +`security-issue-sync` when it scans `users@` for `[RESULT][VOTE]` +announcements). Also fires on the *"check for new activity on this +list"* shortcut path used by triage sweeps. + +**Inputs.** + +| Arg | Type | Notes | +|---|---|---| +| `list` | string | The fully-qualified list address. | +| `since` | ISO-8601 date or relative duration string | The lower bound for the scan window. Adapters that don't accept a free-form `since` are expected to translate (PonyMail takes a `d=lte=` / `d=gte=` syntax; Hyperkitty takes a `?date=` param; Discourse takes an `after:` operator). | + +**Output.** An array of `thread_summary` records, each at minimum: + +- `thread_id` +- `subject` +- `first_message_date` +- `last_message_date` +- `message_count` +- `permalink` — the URL `fetch_thread_by_url` would accept. + +Ordering is *newest-first by `last_message_date`*. Adapters that +return an unordered set internally must sort before returning. + +**No-op case.** Returns `[]` (empty array) when the list has no +activity in the requested window, or when authentication is +missing and the list is private. Empty `[]` and *"no access"* are +indistinguishable from the skill's perspective by design — the +skill surfaces the gap without distinguishing reason. + +### `resolve_advisory_announcement_url(list, advisory_id) to string | null` + +**When it fires.** Step 1h / Step 2b of `security-issue-sync` — the +sync skill polls for the *"advisory archived on ``"* +signal that flips the tracker from `fix released` to `announced`. +Today the ASF adapter resolves this by curling +`https://lists.apache.org/list.html?:YYYY:` and +checking for a 200 response with a thread hit; other adapters +implement the equivalent against their own search APIs. + +**Inputs.** + +| Arg | Type | Notes | +|---|---|---| +| `list` | string | The public announcement list address (`` or `` for ASF). | +| `advisory_id` | string | The advisory identifier the skill is scanning for — typically the CVE ID once `cve_authority.publish` has fired, but could equally be a GHSA ID for projects using GHSA as their `cve_authority.tool`. | + +**Output.** The resolved permalink for the advisory thread +(equivalent to the return shape of `search_thread_url` but already +narrowed to the single matched thread), or `null` when no thread +matches. + +**No-op case.** Returns `null` when: + +- No thread mentions the advisory ID on the named list within the + adapter's default scan window. +- The list is private and the adapter has no access (the + announcement list should always be public, so this is an + adapter-misconfiguration signal — the skill flags it but does + not retry). +- The adapter is `none` (no archive backend declared). + +The skill treats `null` as *"not yet archived"* and re-checks on the +next sync run. A non-null return is a load-bearing signal — it +triggers the multi-step `fix released to announced` close-out flow +in `security-issue-sync` Step 4 (label flips, CVE JSON +regeneration, Vulnogram `REVIEW to PUBLIC` push, milestone close, +board archival, RM hand-off comment). + +### `publication_signal_url(list) to string` + +**When it fires.** On every `security-issue-sync` run that has the +*"public-advisory-url not yet populated"* condition — the skill +needs the URL that *flips visible* when a release-announcement is +archived, so it can present it to the user as a one-click verify +URL alongside the `resolve_advisory_announcement_url` programmatic +scan. + +**Inputs.** + +| Arg | Type | Notes | +|---|---|---| +| `list` | string | The announcement list address. | + +**Output.** A URL pointing at the list's *most-recent-activity* +view. For PonyMail this is +`https://lists.apache.org/list.html?` (the unfiltered +list-index page that updates as new messages arrive). For +Hyperkitty this is `https:///archives/list//`. +For Discourse this is the category permalink. The skill embeds the +URL in informational comments and in the *"check for advisory +archive"* sync prompt. + +**No-op case.** Adapters that have no concept of *"most-recent- +activity page"* (the `none` adapter; some Discourse configurations) +return an empty string. The skill omits the verify-URL line from +its sync prompt when this happens. + +## Skills that consume this contract + +| Skill | Where the call lives today | Verb | +|---|---|---| +| `security-issue-import` | Step 5 — *"PonyMail URL construction"* — the skill builds the per-month search URL from the project manifest's `` value plus the inbound message's received-month, proposes it to the user, and waits for the resolved per-thread URL to be pasted back. | `search_thread_url(list=, year, month, query=)` for the prompt URL; `fetch_thread_by_url(url=)` for the verification step. | +| `security-issue-sync` | Step 1c — *"check the mailing-list thread for new activity since last sync"*. | `fetch_thread_by_url(url=)` re-read; the skill diffs participants and message dates against the previous sync. | +| `security-issue-sync` | Step 1e — *"locate the `[RESULT][VOTE]` thread for the release that ships this CVE"*. | `list_recent_threads(list=, since=)` filtered for `[RESULT][VOTE]` subject prefix. | +| `security-issue-sync` | Step 1h — *"has the advisory been archived on `` yet?"*. | `resolve_advisory_announcement_url(list=, advisory_id=)`; non-null return triggers the close-out flow. | +| `security-issue-sync` | Step 2b — *"present the verify URL to the user alongside the programmatic scan result"*. | `publication_signal_url(list=)`. | +| `security-issue-invalidate` | Closing-reply step — *"locate the original relay-rewritten inbound thread so the polite-but-firm rejection lands on the right archive entry"*. | `search_thread_url(list=, year, month, query=)`; the skill cross-checks against the tracker's *security-thread* body field. | + +Every call site listed above currently hard-codes `mcp__ponymail__*` +or constructs a `https://lists.apache.org/...` URL inline. PR3 +refactors the call sites to flow through the verbs declared in this +contract; PR1 (this PR) only declares the contract. + +## ASF default — PonyMail + +The ASF default adapter is documented today at +[`tools/ponymail/`](../ponymail/) (read-side via the +[`apache/comdev` `mcp/ponymail-mcp/`](https://github.com/apache/comdev/tree/main/mcp/ponymail-mcp) +MCP server) and [`tools/gmail/ponymail-archive.md`](../gmail/ponymail-archive.md) +(URL-template form used for in-tracker cross-links). + +URL-construction shape that the ASF adapter satisfies: + +| Verb | URL template | +|---|---| +| `search_thread_url` | `https://lists.apache.org/list?:YYYY-M:` | +| `fetch_thread_by_url` | `https://lists.apache.org/api/thread.lua?list=&domain=&q=` (JSON read), backed by `https://lists.apache.org/thread/?` (the canonical per-thread permalink the skill stores in tracker body fields) | +| `list_recent_threads` | `https://lists.apache.org/api/stats.lua?list=&domain=&d=lte=` | +| `resolve_advisory_announcement_url` | `https://lists.apache.org/list.html?:YYYY:` (text-mode existence check), resolving to `https://lists.apache.org/thread/?` on a hit | +| `publication_signal_url` | `https://lists.apache.org/list.html?` | + +Month-token format note: the PonyMail search URL takes the month +**without a leading zero** (`2026-4`, not `2026-04`). Adapters that +front-end a backend with a different convention (Hyperkitty uses +`2026-04`) must normalise at the boundary. + +Auth note: private-list reads (`security@.apache.org`, +`private@.apache.org`) require an authenticated PonyMail +MCP session (PMC LDAP OAuth). The first-login flow is documented +in [`tools/ponymail/tool.md`](../ponymail/tool.md#setup) and is run +once per workstation; the session cookie is cached at +`~/.ponymail-mcp/session.json`. + +**Rename plan.** PR3 of this refactor renames `tools/ponymail/` to +`tools/mail-archive-ponymail/` and updates every cross-link. The +directory contents stay the same — only the path moves. PR1 (this +PR) keeps the existing path so the diff stays minimal and reviewable. + +## Configuration + +The adapter selection lives in `projects//project.md` +under the `archive_system` block: + +```yaml +# archive_system — public mail-archive backend +# ASF default: ponymail (lists.apache.org) +archive_system: + kind: ponymail # ASF default; override per-adopter for hyperkitty | discourse | google-groups | github-discussions | none + list_domain: .apache.org # ASF default; the list's domain component + search_url_template: https://lists.apache.org/list?{list}:{year}-{month}:{query} + # ASF default; the URL `search_thread_url` returns + api_query_url_template: https://lists.apache.org/api/thread.lua?list={list_local}&domain={list_domain}&q={query} + # ASF default; the URL `fetch_thread_by_url` reads + advisory_publication_signal_url: https://lists.apache.org/list.html? + # ASF default; the URL `publication_signal_url` returns +``` + +Adopters override per-field. A Hyperkitty deployment would set +`kind: hyperkitty`, point `search_url_template` at +`https:///hyperkitty/list/{list}/{year}/{month}/?q={query}`, +and the rest follows. A project that has no archive backend at all +declares `kind: none` and the skills degrade — `search_thread_url` +returns empty, `fetch_thread_by_url` returns `null`, and the +tracker body fields fall back to the *"not available, see Gmail +thread ``"* textual note. + +Adapter selection is *purely declarative*. The skill bodies do not +branch on `kind` — they call the verbs, and the dispatch into the +adapter happens at the contract boundary. This is the property that +makes the contract a stable seam: adding `discourse` later is a +new directory under `tools/mail-archive-/`, not a change to +the skills. From dbc155bf898831d99bb22f3407a713ba0cb65f14 Mon Sep 17 00:00:00 2001 From: Jarek Potiuk Date: Sat, 30 May 2026 18:07:53 +0200 Subject: [PATCH 2/2] =?UTF-8?q?fix(security/pr1):=20broken=20links=20?= =?UTF-8?q?=E2=80=94=20contract.md=20->=20README.md,=20bot-credits-policy?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The drafted files referenced `/contract.md` but were written as `/README.md` (matching the validator's required filename). Update project.md cross-references accordingly. Also fix one hallucinated link in tools/forwarder-relay/README.md that pointed at `docs/security/reporter-credit-policy.md` (file does not exist) — repoint to the actual bot/AI credit policy doc at `tools/vulnogram/bot-credits-policy.md`. Generated-by: Claude Code (Opus 4.7) --- projects/_template/project.md | 12 ++++++------ tools/forwarder-relay/README.md | 5 +++-- 2 files changed, 9 insertions(+), 8 deletions(-) diff --git a/projects/_template/project.md b/projects/_template/project.md index 929f1e31..d29d76d2 100644 --- a/projects/_template/project.md +++ b/projects/_template/project.md @@ -246,9 +246,9 @@ it*, and the *consuming skills* (1-3 most relevant names). The adapter contracts these blocks reference live under: -- [`../../tools/cve-tool/contract.md`](../../tools/cve-tool/contract.md) — CNA tool interface (ASF default adapter: `tools/vulnogram/`) -- [`../../tools/mail-archive/contract.md`](../../tools/mail-archive/contract.md) — public-archive interface (ASF default adapter: `tools/ponymail/`) -- [`../../tools/forwarder-relay/contract.md`](../../tools/forwarder-relay/contract.md) — inbound-relay interface (ASF default adapter: the ASF-security forwarder shape in `tools/gmail/asf-relay.md`) +- [`../../tools/cve-tool/README.md`](../../tools/cve-tool/README.md) — CNA tool interface (ASF default adapter: `tools/vulnogram/`) +- [`../../tools/mail-archive/README.md`](../../tools/mail-archive/README.md) — public-archive interface (ASF default adapter: `tools/ponymail/`) +- [`../../tools/forwarder-relay/README.md`](../../tools/forwarder-relay/README.md) — inbound-relay interface (ASF default adapter: the ASF-security forwarder shape in `tools/gmail/asf-relay.md`) ### CVE authority @@ -300,7 +300,7 @@ cve_authority: # ASF default mapping: Vulnogram's DRAFT -> allocated, # REVIEW -> review-ready, READY -> publish-ready, PUBLIC -> public. # Override when: the adapter has a different state machine — the - # adapter declares its own mapping in its contract.md. + # adapter declares its own mapping in its README.md. # Consumed by: security-issue-sync, security-cve-allocate, # generate-cve-json. states: [allocated, review-ready, publish-ready, public] @@ -443,7 +443,7 @@ security_inbox: forwarders: # Enabled forwarder/relay adapters. Each name must match an # adapter directory under tools/ that conforms to - # tools/forwarder-relay/contract.md. + # tools/forwarder-relay/README.md. # ASF default: [asf-security] — the ASF security team relays # reports onto project security@ lists with a known preamble and # credit line. @@ -471,7 +471,7 @@ forwarders: # Rule the adapter uses to lift the original reporter's credit # line out of the relayed body. Adapters define their own - # extraction shape; see tools/forwarder-relay/contract.md. + # extraction shape; see tools/forwarder-relay/README.md. # ASF default: the existing ASF-security credit extraction (the # "Reported by: <>" line near the top of the # forwarded body). diff --git a/tools/forwarder-relay/README.md b/tools/forwarder-relay/README.md index 909415b6..995156ae 100644 --- a/tools/forwarder-relay/README.md +++ b/tools/forwarder-relay/README.md @@ -59,8 +59,9 @@ only by asking the broker to relay messages back. This matters for three skill behaviours: 1. **Credit extraction.** The `From:` header of a relay message - names the broker, not the reporter. Per - [`docs/security/reporter-credit-policy.md`](../../docs/security/reporter-credit-policy.md) + names the broker, not the reporter. Per the bot/AI credit + policy in + [`tools/vulnogram/bot-credits-policy.md`](../vulnogram/bot-credits-policy.md) the tracker's *Reporter credited as* field must name the external reporter, so the skill has to pull the name from the message body (the broker's preamble convention) instead of