PRD: Migrate skill-creator Python scripts to TypeScript

[MatthewYe]

Problem Statement

skills/skill-creator/ contains 2,404 lines of Python (10 .py files + pyproject.toml) while the rest of opencode-toolbox uses Bun/TypeScript. This dual-stack creates maintenance friction: contributors need Python 3.10+ installed just to use skill-creator scripts, two package managers to manage, and no shared tooling between the plugin core and its own skill toolchain.

Solution

Rewrite all Python scripts to TypeScript with Bun as the runtime, preserving identical functionality, CLI interfaces, and JSON output schemas. Keep the same directory layout (scripts/*.ts replacing scripts/*.py). Remove Python entirely — delete pyproject.toml and all .py files. The result is a single-stack project: Bun + TypeScript only.

User Stories

1. As a skill-creator user, I want to validate my skill's SKILL.md with bun run scripts/quick_validate.ts, so that I can catch frontmatter errors before publishing
2. As a skill-creator user, I want to package my skill into a .skill zip file with bun run scripts/package_skill.ts, so that I can distribute it
3. As a skill-creator user, I want to run trigger evaluations with bun run scripts/run_eval.ts, so that I can measure whether my skill's description correctly triggers for target queries
4. As a skill-creator user, I want to optimize a skill's description with bun run scripts/improve_description.ts, so that failing eval queries get better trigger accuracy
5. As a skill-creator user, I want to run the full optimize loop with bun run scripts/run_loop.ts, so that I can iterate eval → improve → re-eval automatically
6. As a skill-creator user, I want to generate an HTML report from run_loop output with bun run scripts/generate_report.ts, so that I can visualize optimization progress
7. As a skill-creator user, I want to aggregate benchmark results with bun run scripts/aggregate_benchmark.ts, so that I can compare skill performance across configurations
8. As a skill-creator user, I want to review eval outputs via a local HTTP server with bun run eval-viewer/generate_review.ts, so that I can inspect results and provide human feedback
9. As an AI agent executing the skill-creator skill, I want all scripts to require only Bun (no Python), so that I can run them in any environment without extra setup
10. As a contributor to opencode-toolbox, I want zero Python files in the repo, so that I only need to know TypeScript and Bun to work on the entire project
11. As a project maintainer, I want each migrated script to pass output-parity verification against its Python predecessor before merging, so that I am confident no functionality was lost

Implementation Decisions

Runtime: Bun

Use bun run scripts/<name>.ts to execute scripts. Bun is already the project runtime. No compilation step needed for dev scripts — Bun executes .ts directly.

Dependency mapping

Python	TypeScript	Category
pyyaml	gray-matter (already in project)	YAML frontmatter parsing
zipfile.ZipFile	adm-zip (new dependency)	ZIP packaging
http.server	Node.js built-in http	HTTP review server
concurrent.futures.ProcessPoolExecutor	child_process.fork + Promise.all	Parallel execution
subprocess	child_process.spawn / exec	AI CLI invocation
webbrowser.open()	child_process.exec('open') on macOS	Browser launch
mimetypes	mime-types or hand-rolled map	MIME type detection

All new dependencies go into dependencies (scripts are part of the published skill).

File structure — in-place migration

skills/skill-creator/
├── scripts/
│   ├── utils.ts              ← utils.py
│   ├── quick_validate.ts     ← quick_validate.py
│   ├── package_skill.ts      ← package_skill.py
│   ├── run_eval.ts           ← run_eval.py
│   ├── improve_description.ts ← improve_description.py
│   ├── run_loop.ts           ← run_loop.py
│   ├── generate_report.ts    ← generate_report.py
│   ├── aggregate_benchmark.ts ← aggregate_benchmark.py
│   └── __init__.py           ← DELETED (no equivalent needed)
├── eval-viewer/
│   ├── generate_review.ts    ← generate_review.py
│   └── viewer.html           (unchanged)
├── pyproject.toml            ← DELETED
└── ...                       (SKILL.md, agents/, assets/, references/ unchanged)

Migration phases

1. Phase 1 (utils, quick_validate, package_skill) — deterministic scripts, no CLI invocation
2. Phase 2 (run_eval, improve_description) — scripts that spawn AI CLIs
3. Phase 3 (run_loop, generate_report, aggregate_benchmark) — combinators and report generators
4. Phase 4 (eval-viewer/generate_review) — HTTP server + static HTML
5. Cleanup — delete Python files and pyproject.toml, update SKILL.md references

Verification discipline

Each phase must pass output parity before proceeding: run identical inputs through both the Python and TypeScript implementations and diff their outputs.

• Deterministic scripts (Phase 1, 4): byte-identical output. Compare file hashes, JSON content, zip file contents.
• AI-dependent scripts (Phase 2, 3): structurally identical output. Compare JSON keys, types, field presence, schema conformance. Values may differ due to AI non-determinism.

Testing Decisions

Every module gets automated tests:

• What makes a good test: Tests verify external behavior through public interfaces only — feed input, assert output. No tests on internal implementation details (which functions called, intermediate state). Mock at system boundaries only (filesystem via temp dirs, AI CLI via mocked child_process, network via in-process HTTP server).

• All 9 modules tested: utils, quick_validate, package_skill, run_eval, improve_description, run_loop, generate_report, aggregate_benchmark, eval-viewer/generate_review

• Prior art: No existing test framework in the project. Tests will use Bun's built-in test runner (bun test), which is Jest-compatible and requires zero additional dependencies.

• Verification as regression test suite: The output-parity tests developed during migration become the permanent regression test suite, ensuring future changes don't break the scripts.

Out of Scope

• Adding new features or changing script behavior — this is a pure 1:1 migration
• Changing the eval-viewer/viewer.html frontend — it works unchanged with the new backend
• Changing any markdown files under agents/, references/, assets/ — they remain as-is
• Altering the SKILL.md workflow instructions beyond updating script invocation commands
• Migrating upstream/ Python files (there are none)
• Performance optimization — match Python performance, don't exceed it

Further Notes

• ADR recorded at docs/adr/0003-python-to-typescript-migration.md
• The migration was designed through a grill-with-docs session covering scope, runtime, dependency management, file structure, verification strategy, migration order, and ADR worthiness
• All decisions documented in the ADR

[MatthewYe]

autopilot: This is a PRD containing 4 migration phases + cleanup across 9 scripts. It needs to be decomposed into per-phase (or per-script) implementable issues using /to-issues before autopilot can process. Current scope is too large for a single autonomous implementation.