apache · potiuk · Jul 1, 2026 · Jul 1, 2026
diff --git a/docs/labels-and-capabilities.md b/docs/labels-and-capabilities.md
@@ -245,7 +245,7 @@ it implements multiple contracts (e.g. `tools/gmail` provides both
 | [`tools/jira`](../tools/jira/) | `contract:tracker` | JIRA REST substrate (read-only today; write subcommands tracked in [#301](https://github.com/apache/magpie/issues/301)) |
 | [`tools/mail-archive`](../tools/mail-archive/) | `contract:mail-archive` | Adapter contract for public mail-archive backends (PonyMail, Hyperkitty, Discourse, Google Groups, GitHub Discussions). Pure interface spec. |
 | [`tools/mail-source`](../tools/mail-source/) | `contract:mail-source` | Mail-source backend abstraction (mbox / IMAP / Mailman 3) feeding a uniform inbound thread/message view to the intake pipeline |
-| [`tools/ponymail`](../tools/ponymail/) | `contract:mail-archive` | PonyMail public mail-archive substrate (ASF `lists.apache.org`); implements the `tools/mail-archive/` contract |
+| [`tools/ponymail`](../tools/ponymail/) | `contract:mail-archive` + `contract:mail-source` | PonyMail public mail-archive substrate (ASF `lists.apache.org`); implements the `tools/mail-archive/` contract for archive reads and the `tools/mail-source/` contract for inbound list-traffic ingestion |
 | [`tools/scan-format`](../tools/scan-format/) | `contract:scan-format` | Adapter contract for security-scanner report formats (ASVS reference); reads a scan's finding index + per-finding evidence for the `security-issue-import-from-scan` pipeline. |
 | [`tools/permission-audit`](../tools/permission-audit/) | `substrate:sandbox` | Audit + atomically edit Claude Code `permissions.allow[]` entries; backs `/magpie-setup verify --apply-permission-audit` (check 8d) |
 | [`tools/pr-management-stats`](../tools/pr-management-stats/) | `substrate:analytics` | PR-backlog analytics engine |
@@ -258,6 +258,7 @@ it implements multiple contracts (e.g. `tools/gmail` provides both
 | [`tools/skill-evals`](../tools/skill-evals/) | `substrate:framework-dev` | Eval harness for skills; framework-dev infrastructure whose run output is governance evidence |
 | [`tools/skill-and-tool-validator`](../tools/skill-and-tool-validator/) | `substrate:framework-dev` | Skill-frontmatter and convention validator |
 | [`tools/spec-status-index`](../tools/spec-status-index/) | `substrate:framework-dev` + `substrate:analytics` | Index of spec / RFC implementation status — framework-dev substrate that also doubles as a governance/stats view (`analytics`) |
+| [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/) | `substrate:framework-dev` + `substrate:analytics` | Deterministic vendor-neutrality score — reads each contract tool's `**Kind:**` / `**Vendor:**` metadata and scores per-contract + per-skill neutrality (`analytics`); backs the score block in [`docs/vendor-neutrality.md`](vendor-neutrality.md) |
 | [`tools/spec-validator`](../tools/spec-validator/) | `substrate:framework-dev` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` |
 | [`tools/symlink-lint`](../tools/symlink-lint/) | `substrate:framework-dev` | Self-adoption symlink hygiene — rejects cyclic symlinks and misdirected skill relays (canonical/relay target-correctness) |
 | [`tools/pilot-report-validator`](../tools/pilot-report-validator/) | `substrate:framework-dev` | Adopter pilot-report validator — required frontmatter keys, no unfilled placeholders, valid profile, and required body sections; counterpart to `spec-validator` for `docs/pilot-report-template.md` |
@@ -286,7 +287,7 @@ backend the adopter wired in. The framework consumes four:
 |---|---|---|---|---|
 | GitHub MCP | `mcp__github__*` | [`tools/github`](../tools/github/) | `contract:tracker` + `contract:source-control` | — |
 | Gmail MCP (claude.ai) | `mcp__claude_ai_Gmail__*` | [`tools/gmail`](../tools/gmail/) | `contract:mail-source` + `contract:mail-draft` + `contract:mail-archive` | — |
-| PonyMail MCP (`apache/comdev`) | `mcp__ponymail__*` | [`tools/ponymail`](../tools/ponymail/) | `contract:mail-archive` | ASF |
+| PonyMail MCP (`apache/comdev`) | `mcp__ponymail__*` | [`tools/ponymail`](../tools/ponymail/) | `contract:mail-archive` + `contract:mail-source` | ASF |
 | apache-projects MCP (`apache/comdev`) | `mcp__apache-projects__*` | [`tools/apache-projects`](../tools/apache-projects/) | `contract:project-metadata` | ASF |
 
 Non-MCP backends fulfil the same contracts: JIRA is reached over REST

diff --git a/docs/vendor-neutrality.md b/docs/vendor-neutrality.md
@@ -22,6 +22,9 @@
   - [What keeps it neutral over time](#what-keeps-it-neutral-over-time)
   - [The contribution model — neutrality as an invitation](#the-contribution-model--neutrality-as-an-invitation)
   - [Status at a glance](#status-at-a-glance)
+  - [Vendor-neutrality score](#vendor-neutrality-score)
+    - [How the score is computed](#how-the-score-is-computed)
+    - [What the number means](#what-the-number-means)
   - [What "vendor neutral" does and does not claim](#what-vendor-neutral-does-and-does-not-claim)
   - [See also](#see-also)
 
@@ -490,6 +493,89 @@ coverage without pretending one team can implement an open-ended set.
 adding a backend is an adapter against a documented contract, not a
 change to any skill.
 
+## Vendor-neutrality score
+
+The six axes above are the *narrative*. This section is the
+*measurement* — a deterministic score computed straight from repository
+metadata by
+[`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/), so
+the number is reproducible from the source tree and cannot quietly drift
+from the code.
+
+### How the score is computed
+
+Neutrality is measured per **capability contract** — the `contract:*`
+verbs a skill depends on. Substrate tools (Magpie's own machinery:
+sandboxing, analytics, framework-dev) are excluded, because they are not
+a vendor choice.
+
+Every contract tool declares three fields in its README: `**Capability:**`
+(the contract it fulfils), `**Kind:**` (`interface` for a pure spec,
+`implementation` for a concrete backend), and `**Vendor:**` (the backend
+identity). The scorer reads them and applies one rule per contract
+**class**:
+
+- **vendor-backed** → GREEN once **two or more distinct backend vendors**
+  implement it. One backend, however good, is a *default*, not
+  neutrality. Interface specs do not count — only shipping backends do.
+- **agnostic** → GREEN by construction: a single vendor-neutral spec
+  serves every backend, so there is no vendor to be neutral *between*.
+- **single-organisation** → GREEN by exemption: the capability is bound
+  to one organisation's data model (e.g. ASF governance rosters); there
+  is no vendor choice to make.
+
+The overall score is `green contracts / total contracts` — a hard,
+falsifiable number. Add a second outbound-mail backend and `mail-draft`
+flips to green on the next run; remove a backend and its contract flips
+back. The same rule then classifies every **skill**: *capability-pure*
+if it names no backend, *portable* if every backend it invokes has an
+alternative, and *vendor-coupled* only if it reaches for the sole
+implementation of a capability.
+
+<!-- BEGIN vendor-neutrality-score — generated by `uv run --project tools/vendor-neutrality-score vendor-neutrality-score --markdown`; do not edit by hand -->
+**Overall vendor-neutrality score: 8/9 capability contracts (89%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.
+
+| Capability contract | Neutral? | Class | Backends today | Basis |
+|---|---|---|---|---|
+| `contract:tracker` | ✅ | vendor-backed | Atlassian, GitHub | 2 backend vendors: Atlassian, GitHub |
+| `contract:source-control` | ✅ | vendor-backed | Git, GitHub, Subversion | 3 backend vendors: Git, GitHub, Subversion |
+| `contract:mail-archive` | ✅ | vendor-backed | Google, PonyMail | 2 backend vendors: Google, PonyMail |
+| `contract:mail-source` | ✅ | vendor-backed | Google, PonyMail | 2 backend vendors: Google, PonyMail |
+| `contract:mail-draft` | ❌ | vendor-backed | Google | only 1 backend vendor (Google); needs 1 more |
+| `contract:cve-authority` | ✅ | vendor-backed | CVE.org, Vulnogram | 2 backend vendors: CVE.org, Vulnogram |
+| `contract:report-relay` | ✅ | agnostic | — | vendor-neutral by construction — one spec serves every backend |
+| `contract:scan-format` | ✅ | agnostic | — | vendor-neutral by construction — one spec serves every backend |
+| `contract:project-metadata` | ✅ | single-org | ASF | single-organisation capability (ASF); no vendor choice to make |
+
+**Per-skill assessment: 59/63 skills carry no vendor lock-in.** A skill is *capability-pure* when it names no backend at all, *portable* when every backend it names has an alternative (its contract is green), and *vendor-coupled* only when it reaches for a backend that is the sole implementation of a capability.
+
+| Skill neutrality | Count |
+|---|---|
+| capability-pure (names no backend) | 9 |
+| portable (named backends are swappable) | 50 |
+| vendor-coupled (sole-backend dependency) | 4 |
+
+Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 49.
+
+Vendor-coupled skills (the only lock-ins today):
+
+- `security-issue-import` — `Google` → `contract:mail-draft`
+- `security-issue-import-via-forwarder` — `Google` → `contract:mail-draft`
+- `security-issue-invalidate` — `Google` → `contract:mail-draft`
+- `security-issue-sync` — `Google` → `contract:mail-draft`
+<!-- END vendor-neutrality-score -->
+
+### What the number means
+
+89% is not "89% done." It reads as: **eight of nine capabilities already
+work across more than one vendor, and the ninth — outbound mail drafting
+— is one adapter away.** The architecture privileges no vendor on any of
+the nine axes; the single red cell is a missing *implementation* (a
+second `mail-draft` backend), tracked in the open, not a design that
+assumes Gmail. Likewise only four skills touch that one lock-in, and each
+does so through the `mail-draft` contract — swap in a second backend and
+all four become portable without a line of skill code changing.
+
 ## What "vendor neutral" does and does not claim
 
 To keep the marketing honest and the engineering claim precise:

diff --git a/pyproject.toml b/pyproject.toml
@@ -114,4 +114,5 @@ members = [
   "tools/spec-validator",
   "tools/symlink-lint",
   "tools/vcs",
+  "tools/vendor-neutrality-score",
 ]
diff --git a/tools/apache-projects/README.md b/tools/apache-projects/README.md
@@ -14,6 +14,10 @@
 
 **Capability:** contract:project-metadata
 
+**Kind:** implementation
+
+**Vendor:** ASF
+
 **Organization:** ASF
 
 ASF project-metadata substrate. Read-only, unauthenticated client

diff --git a/tools/asf-svn/README.md b/tools/asf-svn/README.md
@@ -15,6 +15,10 @@
 
 **Capability:** contract:source-control
 
+**Kind:** implementation
+
+**Vendor:** Subversion
+
 **Organization:** ASF
 
 ASF SVN tool adapter — the Subversion counterpart to

diff --git a/tools/cve-org/README.md b/tools/cve-org/README.md
@@ -14,6 +14,10 @@
 
 **Capability:** contract:cve-authority
 
+**Kind:** implementation
+
+**Vendor:** CVE.org
+
 CVE.org publication client. Submits CVE records via the CVE.org REST API; consumed by `security-cve-allocate` once a CVE has been allocated via the ASF Vulnogram path. See [`tool.md`](tool.md) for the protocol detail and `cve.org` field mapping.
 
 ## Prerequisites

diff --git a/tools/cve-tool-vulnogram/README.md b/tools/cve-tool-vulnogram/README.md
@@ -14,6 +14,10 @@
 
 **Capability:** contract:cve-authority
 
+**Kind:** implementation
+
+**Vendor:** Vulnogram
+
 **Organization:** ASF
 
 ASF Vulnogram CVE-allocation client. OAuth-authenticated API that allocates a CVE ID through the ASF's Vulnogram instance and publishes the CVE record. Consumed by `security-cve-allocate`. See [`allocation.md`](allocation.md), [`record.md`](record.md), and [`bot-credits-policy.md`](bot-credits-policy.md) for the protocol.

diff --git a/tools/cve-tool/README.md b/tools/cve-tool/README.md
@@ -26,6 +26,10 @@
 
 **Capability:** contract:cve-authority
 
+**Kind:** interface
+
+**Vendor:** agnostic
+
 ## Prerequisites
 
 - **Runtime:** None — this directory is a Markdown contract spec; no executable code ships here. It is read by the framework, not run.

diff --git a/tools/forwarder-relay/README.md b/tools/forwarder-relay/README.md
@@ -30,6 +30,10 @@
 
 **Capability:** contract:report-relay
 
+**Kind:** interface
+
+**Vendor:** agnostic
+
 A forwarder-relay adapter is a pluggable seam that teaches the
 security skills how to recognise an inbound report that arrived
 **through a relay** (someone else forwarded the original

diff --git a/tools/github-body-field/README.md b/tools/github-body-field/README.md
@@ -21,6 +21,10 @@
 
 **Capability:** contract:tracker
 
+**Kind:** implementation
+
+**Vendor:** GitHub
+
 Read or rewrite a single `### Field` section of a GitHub issue
 body **without bringing the body into agent context**.
 

diff --git a/tools/github-rollup/README.md b/tools/github-rollup/README.md
@@ -20,6 +20,10 @@
 
 **Capability:** contract:tracker
 
+**Kind:** implementation
+
+**Vendor:** GitHub
+
 Append to (or create) the status-rollup comment on a GitHub
 issue **without bringing the rollup body into agent context**.
 

diff --git a/tools/github/README.md b/tools/github/README.md
@@ -14,6 +14,10 @@
 
 **Capability:** contract:tracker + contract:source-control
 
+**Kind:** implementation
+
+**Vendor:** GitHub
+
 GitHub REST + GraphQL substrate. Pure read/write wrapper used by every lifecycle phase (triage / intake / fix / resolve / stats). See [`tool.md`](tool.md) for the operation catalogue and the per-area files ([`issue-template.md`](issue-template.md), [`labels.md`](labels.md), [`operations.md`](operations.md), [`project-board.md`](project-board.md), [`status-rollup.md`](status-rollup.md)) for specifics.
 
 ## Prerequisites

diff --git a/tools/gmail/README.md b/tools/gmail/README.md
@@ -14,6 +14,10 @@
 
 **Capability:** contract:mail-source + contract:mail-draft + contract:mail-archive
 
+**Kind:** implementation
+
+**Vendor:** Google
+
 Gmail API substrate. Read + draft-only — never sends. Provides two
 contracts: `mail-source` for inbound report intake (search / read a
 uniform thread/message view) and `mail-draft` for outbound courtesy-reply

diff --git a/tools/jira/README.md b/tools/jira/README.md
@@ -32,6 +32,10 @@
 
 **Capability:** contract:tracker
 
+**Kind:** implementation
+
+**Vendor:** Atlassian
+
 JIRA REST helpers for the `issue-*` skill family.
 Adopters with JIRA-based issue trackers wire this in as their
 tracker bridge; adopters using GitHub Issues or other trackers

diff --git a/tools/mail-archive/README.md b/tools/mail-archive/README.md
@@ -24,6 +24,10 @@
 
 **Capability:** contract:mail-archive
 
+**Kind:** interface
+
+**Vendor:** agnostic
+
 This file defines the adapter contract for **public mail-archive
 backends** — the seam that lets adopting projects plug a non-ASF
 archive system (Hyperkitty, Discourse, Google Groups, GitHub

diff --git a/tools/mail-source/README.md b/tools/mail-source/README.md
@@ -14,6 +14,10 @@
 
 **Capability:** contract:mail-source
 
+**Kind:** interface
+
+**Vendor:** agnostic
+
 Mail-source backend abstraction. Pluggable backends (mbox, IMAP, the Gmail API via [`tools/gmail`](../gmail/), future Mailman 3 / Hyperkitty) that feed the security-issue-import intake pipeline a uniform thread/message view. See [`contract.md`](contract.md) for the backend interface.
 
 ## Prerequisites

diff --git a/tools/ponymail/README.md b/tools/ponymail/README.md
@@ -12,7 +12,11 @@
 
 # `tools/ponymail/`
 
-**Capability:** contract:mail-archive
+**Capability:** contract:mail-archive + contract:mail-source
+
+**Kind:** implementation
+
+**Vendor:** PonyMail
 
 **Organization:** ASF
 

diff --git a/tools/scan-format/README.md b/tools/scan-format/README.md
@@ -23,6 +23,10 @@
 
 **Capability:** contract:scan-format
 
+**Kind:** interface
+
+**Vendor:** agnostic
+
 A **scan-format adapter** teaches
 [`security-issue-import-from-scan`](../../skills/security-issue-import-from-scan/SKILL.md)
 how to read one security scanner's report layout. The skill is

diff --git a/tools/vcs/README.md b/tools/vcs/README.md
@@ -18,6 +18,10 @@
 
 **Capability:** contract:source-control
 
+**Kind:** implementation
+
+**Vendor:** Git
+
 Runnable implementation of the **source-control (VCS) capability**
 documented in
 [`tools/github/source-control.md`](../github/source-control.md). It