Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/labels-and-capabilities.md
Original file line number Diff line number Diff line change
Expand Up @@ -274,6 +274,7 @@ it implements multiple contracts (e.g. `tools/gmail` provides both
| [`tools/spec-loop`](../tools/spec-loop/) | `substrate:framework-dev` | Spec-driven build loop runner (Ralph-style) for framework development |
| [`tools/skill-evals`](../tools/skill-evals/) | `substrate:framework-dev` | Eval harness for skills; framework-dev infrastructure whose run output is governance evidence |
| [`tools/skill-and-tool-validator`](../tools/skill-and-tool-validator/) | `substrate:framework-dev` | Skill-frontmatter and convention validator |
| [`tools/spec-inventory`](../tools/spec-inventory/) | `substrate:framework-dev` + `substrate:analytics` | Compact routing inventory for spec-loop prompts — summarizes specs, skills, and tool metadata so agents can choose relevant files before direct verification |
| [`tools/spec-status-index`](../tools/spec-status-index/) | `substrate:framework-dev` + `substrate:analytics` | Index of spec / RFC implementation status — framework-dev substrate that also doubles as a governance/stats view (`analytics`) |
| [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/) | `substrate:framework-dev` + `substrate:analytics` | Deterministic vendor-neutrality score — reads each contract tool's `**Kind:**` / `**Vendor:**` metadata and scores per-contract + per-skill neutrality (`analytics`); backs the score block in [`docs/vendor-neutrality.md`](vendor-neutrality.md) |
| [`tools/spec-validator`](../tools/spec-validator/) | `substrate:framework-dev` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` |
Expand Down
7 changes: 3 additions & 4 deletions docs/vendor-neutrality.md
Original file line number Diff line number Diff line change
Expand Up @@ -544,7 +544,6 @@ approval on endpoint identity rather than vendor. Both appear in the
generated block below.

<!-- BEGIN vendor-neutrality-score — generated by `uv run --project tools/vendor-neutrality-score vendor-neutrality-score --markdown`; do not edit by hand -->

**Overall vendor-neutrality score: 10/10 capability contracts (100%).** Generated by [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/); re-run it to refresh this section.

| Capability contract | Neutral? | Class | Backends today | Basis |
Expand Down Expand Up @@ -572,7 +571,7 @@ Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 49.

**LLM / agent-integration neutrality**

**Agent harness: 20/20 substrate tools run under any harness unchanged (100%).** Substrate tools are Magpie's own machinery; each declares the agent harness it integrates with (`**Harness:**`), or `agnostic`. A tool is neutral when it is harness-agnostic or supports two or more harnesses; *coupled* when it targets a single harness.
**Agent harness: 21/21 substrate tools run under any harness unchanged (100%).** Substrate tools are Magpie's own machinery; each declares the agent harness it integrates with (`**Harness:**`), or `agnostic`. A tool is neutral when it is harness-agnostic or supports two or more harnesses; *coupled* when it targets a single harness.

| Substrate tool | Substrate | Harness support | Verdict |
|---|---|---|---|
Expand All @@ -591,6 +590,7 @@ Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 49.
| `security-tracker-stats-dashboard` | analytics | any | ✅ agnostic |
| `skill-and-tool-validator` | framework-dev | any | ✅ agnostic |
| `skill-evals` | framework-dev | any | ✅ agnostic |
| `spec-inventory` | framework-dev, analytics | any | ✅ agnostic |
| `spec-loop` | framework-dev | Claude Code, Codex, Cursor, Gemini CLI, OpenCode | ✅ portable |
| `spec-status-index` | framework-dev, analytics | any | ✅ agnostic |
| `spec-validator` | framework-dev | any | ✅ agnostic |
Expand All @@ -604,7 +604,7 @@ Harness → substrate tools it supports:
- **Cursor** (1): `spec-loop`
- **Gemini CLI** (1): `spec-loop`
- **OpenCode** (5): `agent-guard`, `agent-isolation`, `permission-audit`, `sandbox-lint`, `spec-loop`
- **any harness** (15): `dashboard-generator`, `dev`, `egress-gateway`, `pilot-report-validator`, `pr-management-stats`, `preflight-audit`, `privacy-llm`, `probe-templates`, `security-tracker-stats-dashboard`, `skill-and-tool-validator`, `skill-evals`, `spec-status-index`, `spec-validator`, `symlink-lint`, `vendor-neutrality-score`
- **any harness** (16): `dashboard-generator`, `dev`, `egress-gateway`, `pilot-report-validator`, `pr-management-stats`, `preflight-audit`, `privacy-llm`, `probe-templates`, `security-tracker-stats-dashboard`, `skill-and-tool-validator`, `skill-evals`, `spec-inventory`, `spec-status-index`, `spec-validator`, `symlink-lint`, `vendor-neutrality-score`

**Model endpoint: neutral by construction — 4 default-approved endpoint classes across independent trust domains, plus adopter opt-in.** From the [`privacy-llm` registry](../tools/privacy-llm/models.md): the framework keys approval on *endpoint identity*, not on who hosts the model, so no single LLM vendor is privileged.

Expand All @@ -616,7 +616,6 @@ Harness → substrate tools it supports:
| Air-gapped on-prem | A PMC-hosted inference appliance on a private VLAN |

Every other endpoint is **opt-in** — the adopting project's security team declares it in `<project-config>/privacy-llm.md` (endpoint URL, data-residency contract, approver), so the choice is local and audited.

<!-- END vendor-neutrality-score -->

### What the number means
Expand Down
1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,7 @@ members = [
"tools/skill-and-tool-validator",
"tools/skill-evals",
"tools/pilot-report-validator",
"tools/spec-inventory",
"tools/spec-status-index",
"tools/spec-validator",
"tools/symlink-lint",
Expand Down
51 changes: 51 additions & 0 deletions tools/spec-inventory/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
<!-- START doctoc generated TOC please keep comment here to allow auto update -->
<!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE -->
**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)*

- [spec-inventory](#spec-inventory)
- [Prerequisites](#prerequisites)
- [Usage](#usage)
- [Run tests](#run-tests)

<!-- END doctoc generated TOC please keep comment here to allow auto update -->

<!-- SPDX-License-Identifier: Apache-2.0
https://www.apache.org/licenses/LICENSE-2.0 -->

# spec-inventory

**Capability:** substrate:framework-dev + substrate:analytics

**Harness:** agnostic

A deterministic `uv` tool that emits a compact routing inventory for the
spec-loop prompts. It summarizes spec frontmatter, where-it-lives hints,
validation commands, known gaps, skill frontmatter, and tool/test
presence so agents can choose what to read next without first scanning
the whole repository.

The output is a routing aid, not proof. Prompts still require direct file
reads or code search before declaring behaviour present or absent.

## Prerequisites

- **Runtime:** Python 3.11+ run via `uv`; stdlib-only (no runtime
dependencies). The `dev` group pulls `pytest`, `ruff`, and `mypy`.
- **CLIs:** None beyond the runtime.
- **Credentials / auth:** None.
- **Network:** Runs fully offline; reads local specs, skills, and tool
metadata from the repository checkout.

## Usage

```bash
uv run --project tools/spec-inventory spec-inventory
uv run --project tools/spec-inventory spec-inventory --brief --max-where 1 --max-validation 1 --max-gaps 1
uv run --project tools/spec-inventory spec-inventory --json
```

## Run tests

```bash
uv run --project tools/spec-inventory --group dev pytest tools/spec-inventory/tests
```
75 changes: 75 additions & 0 deletions tools/spec-inventory/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "spec-inventory"
version = "0.1.0"
description = "Generate a compact routing inventory for the spec-loop prompts."
readme = "README.md"
requires-python = ">=3.11"
license = { text = "Apache-2.0" }
dependencies = []

[project.scripts]
spec-inventory = "spec_inventory:main"

[tool.hatch.build.targets.wheel]
packages = ["src/spec_inventory"]

[tool.ruff]
line-length = 110
target-version = "py311"
src = ["src", "tests"]

[tool.ruff.lint]
select = ["E", "W", "F", "I", "B", "UP", "SIM", "C4", "RUF"]
ignore = ["E501"]

[tool.ruff.lint.per-file-ignores]
"tests/**" = ["B", "SIM"]

[tool.mypy]
python_version = "3.11"
files = ["src", "tests"]
warn_unused_ignores = true
warn_redundant_casts = true
warn_unreachable = true
check_untyped_defs = true
no_implicit_optional = true
disallow_untyped_defs = true
disallow_incomplete_defs = true

[[tool.mypy.overrides]]
module = "tests.*"
disallow_untyped_defs = false
disallow_incomplete_defs = false

[dependency-groups]
dev = [
"mypy>=2.1.0",
"pytest>=9.1.1",
"ruff>=0.15.18",
]

[tool.pytest.ini_options]
minversion = "8.0"
addopts = "-ra -q"
testpaths = ["tests"]
Loading
Loading