diff --git a/.changeset/prebuilt-caplets-lockfile-update.md b/.changeset/prebuilt-caplets-lockfile-update.md new file mode 100644 index 00000000..818e6b46 --- /dev/null +++ b/.changeset/prebuilt-caplets-lockfile-update.md @@ -0,0 +1,6 @@ +--- +"@caplets/core": minor +"caplets": minor +--- + +Add lockfile-aware Caplet install, restore, and update workflows, including `caplets update`, JSON lifecycle output, remote-global catalog mutations, derived update risk checks, and new public catalog entries for browser, desktop, observability, and Google Workspace integrations. diff --git a/CONCEPTS.md b/CONCEPTS.md index 40b8d0ec..1bf3f2e6 100644 --- a/CONCEPTS.md +++ b/CONCEPTS.md @@ -8,6 +8,28 @@ Shared domain vocabulary for this project -- entities, named processes, and stat A configured capability surface that exposes a backend to agents through a stable handle, progressive wrapper tools, or direct tool operations. +### Prebuilt Caplets Catalog + +The repo-owned collection of installable Caplet files under `caplets/`. + +The Prebuilt Caplets Catalog is curated as a Code Mode-first capability catalog, not a generic marketplace. Catalog entries should be install-ready when promoted, with setup, auth, validation, safety, and Project Binding metadata appropriate to the backend. + +Install-ready catalog entries have an explicit verification status, a reproducible validation path, and a named primary Code Mode workflow. Unverified entries may exist as drafts or recipes, but they do not count as install-ready catalog coverage. + +### Catalog-Grade Caplet + +A Caplet that is ready to live in the Prebuilt Caplets Catalog. + +Catalog-Grade Caplets include enough frontmatter, setup or verification guidance, auth handling, least-privilege scope notes, safety notes, Code Mode workflow guidance, and local/project/runtime metadata for agents to use them without rediscovering private assumptions from the author's environment. + +### Caplets Lockfile + +A `caplets.lock.json` file that records installed catalog Caplets, their source repository, source path, destination, tracked source channel, resolved revision when available, content hash, and portability status. + +Caplets Lockfiles let `caplets install`, no-argument install restore, and `caplets update` manage installed Caplets from recorded provenance rather than from copied files alone. Project installs use `./.caplets.lock.json`; global installs use the target machine's Caplets state directory. + +Caplets Lockfiles are share-safe and integrity-aware. They strip credential-bearing source URLs, prefer project-relative paths where possible, verify recorded content before restore, and fail closed when local-source entries are unavailable or marked non-portable. + ### Namespace Shadowing Policy A Caplet shadowing policy where a local/upstream ID collision exposes both Caplets under qualified namespace IDs and removes the ambiguous bare ID. diff --git a/README.md b/README.md index 17ffa6ed..2848833d 100644 --- a/README.md +++ b/README.md @@ -55,6 +55,14 @@ Install a no-auth example Caplet and try it from your agent: caplets install spiritledsoftware/caplets osv ``` +Installs write a lockfile. Run `caplets install` with no source argument to restore the +selected project or global lockfile, and run `caplets update` to refresh tracked Caplets: + +```sh +caplets install +caplets update osv +``` + Or add Caplets manually to any MCP client: ```json diff --git a/apps/docs/src/content/docs/install.mdx b/apps/docs/src/content/docs/install.mdx index 4c7240a7..8f2ab715 100644 --- a/apps/docs/src/content/docs/install.mdx +++ b/apps/docs/src/content/docs/install.mdx @@ -31,6 +31,41 @@ caplets install spiritledsoftware/caplets osv `caplets setup` configures supported agent harnesses. The OSV Caplet is the recommended first install because it is public and does not require credentials. +## Install lockfiles and updates + +Successful installs write a lockfile so the same Caplets can be restored or updated later. +Project installs write `./.caplets.lock.json`. Global installs write +`caplets.lock.json` under the Caplets state directory, such as +`~/.local/state/caplets/caplets.lock.json` on Linux. + +Restore the Caplets recorded in the selected lockfile with no source argument: + +```sh +caplets install +caplets install --global +``` + +Update every tracked Caplet, or only named Caplets: + +```sh +caplets update +caplets update sentry github +``` + +Install, restore, and update refuse to overwrite local edits unless you pass `--force`. +Use `--json` when a script needs per-entry statuses and machine-readable errors. + +For explicit remote catalog lifecycle operations, `--remote` targets the remote machine's +global Caplets root and global lockfile: + +```sh +caplets install --remote spiritledsoftware/caplets sentry +caplets install --remote +caplets update --remote sentry +``` + +Remote project install and update semantics are intentionally separate from this workflow. + ## Manual MCP setup If your client does not support `caplets setup`, or if you are avoiding a global install, diff --git a/caplets/ast-grep/CAPLET.md b/caplets/ast-grep/CAPLET.md index c8ef0647..0114c0e0 100644 --- a/caplets/ast-grep/CAPLET.md +++ b/caplets/ast-grep/CAPLET.md @@ -6,6 +6,8 @@ tags: - mcp - code - search +projectBinding: + required: true setup: commands: - label: Install ast-grep MCP @@ -29,6 +31,8 @@ Use this Caplet to expose ast-grep's structural search, scan, rule testing, rewr The manifest uses the full `ast-grep-mcp` MCP server. +Project Binding is required because ast-grep reads and may rewrite files in the attached repository. The bound root defines the workspace that search and rewrite operations are allowed to target. + ## Setup This Caplet installs `ast-grep-mcp` globally with npm, then verifies the installed binary with diff --git a/caplets/browser-use/CAPLET.md b/caplets/browser-use/CAPLET.md new file mode 100644 index 00000000..3baa4867 --- /dev/null +++ b/caplets/browser-use/CAPLET.md @@ -0,0 +1,27 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Browser Use +description: Drive a local Playwright browser through the Playwright MCP server for web inspection and browser workflows. +tags: + - browser + - playwright + - mcp +mcpServer: + command: npx + args: + - -y + - "@playwright/mcp@latest" + - --browser=chromium +--- + +# Browser Use + +Use this Caplet when an agent needs a local browser to inspect pages, gather current web context, or exercise browser-based workflows. + +## Setup + +Install Playwright browser dependencies for the runtime where this Caplet runs. If you need a specific browser executable or profile, create a private variant that uses environment variables such as `DEFAULT_BROWSER_EXECUTABLE_PATH` and `DEFAULT_BROWSER_USER_DATA_DIR`. + +## Safety + +This is a local-control Caplet. Browser actions can sign in, submit forms, trigger purchases, or change account data. Prefer navigation, reading, and screenshots first; review mutating interactions before execution. diff --git a/caplets/computer-use/CAPLET.md b/caplets/computer-use/CAPLET.md new file mode 100644 index 00000000..891cfea9 --- /dev/null +++ b/caplets/computer-use/CAPLET.md @@ -0,0 +1,23 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Computer Use +description: Control local desktop applications and windows through open-computer-use for explicit desktop automation workflows. +tags: + - computer-use + - desktop + - local-control +mcpServer: + command: npx + args: + - -y + - open-computer-use@latest + - mcp +--- + +# Computer Use + +Use this Caplet only when an agent needs explicit access to the local desktop, application windows, or GUI workflows that cannot be completed through APIs or CLI tools. + +## Safety + +This is a high-risk local-control Caplet. It can operate real applications and may expose private screen content. Keep tasks narrow, identify the target application before acting, and do not use it for credential entry, payment flows, or irreversible actions without direct user instruction. diff --git a/caplets/gmail/CAPLET.md b/caplets/gmail/CAPLET.md new file mode 100644 index 00000000..9beaf02e --- /dev/null +++ b/caplets/gmail/CAPLET.md @@ -0,0 +1,32 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Gmail +description: Search, read, label, draft, and send Gmail messages through the Gmail API Discovery document. +tags: + - google + - gmail + - email +googleDiscoveryApi: + discoveryUrl: https://gmail.googleapis.com/$discovery/rest?version=v1 + auth: + type: oauth2 + issuer: https://accounts.google.com + clientId: $vault:GOOGLE_CLIENT_ID + clientSecret: $vault:GOOGLE_CLIENT_SECRET + scopes: + - https://www.googleapis.com/auth/gmail.metadata + - https://www.googleapis.com/auth/gmail.readonly + - https://www.googleapis.com/auth/gmail.modify +--- + +# Gmail + +Use this Caplet when an agent needs Gmail context for support, scheduling, customer communication, or inbox triage. + +## Scope Guidance + +Start with metadata or readonly access when possible. Add `gmail.modify` only when the workflow needs labels, archive, trash, drafts, or sending. Avoid the broad `https://mail.google.com/` scope unless a separate reviewed local Caplet genuinely needs permanent delete access. + +## Use Carefully + +Email often contains private or regulated content. Keep queries narrow, summarize minimally, and require explicit user intent before sending, modifying, trashing, or deleting messages. diff --git a/caplets/google-drive/CAPLET.md b/caplets/google-drive/CAPLET.md new file mode 100644 index 00000000..61b1e34d --- /dev/null +++ b/caplets/google-drive/CAPLET.md @@ -0,0 +1,31 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Google Drive +description: Search, read, download, upload, and manage Google Drive files through the Drive API Discovery document. +tags: + - google + - drive + - files +googleDiscoveryApi: + discoveryUrl: https://www.googleapis.com/discovery/v1/apis/drive/v3/rest + auth: + type: oauth2 + issuer: https://accounts.google.com + clientId: $vault:GOOGLE_CLIENT_ID + clientSecret: $vault:GOOGLE_CLIENT_SECRET + scopes: + - https://www.googleapis.com/auth/drive.file + - https://www.googleapis.com/auth/drive.metadata.readonly +--- + +# Google Drive + +Use this Caplet when an agent needs Drive files as context or needs to create/update files with explicit user direction. + +## Scope Guidance + +Prefer `drive.file` and `drive.metadata.readonly` for public-safe use. Google recommends narrow scopes where possible; broad Drive scopes such as `drive` and `drive.readonly` are restricted and should be added only in a reviewed private Caplet. + +## Use Carefully + +Search metadata before reading content. Confirm file IDs and names before upload, update, trash, or delete operations, especially on shared drives. diff --git a/caplets/google-tasks/CAPLET.md b/caplets/google-tasks/CAPLET.md new file mode 100644 index 00000000..c85da86f --- /dev/null +++ b/caplets/google-tasks/CAPLET.md @@ -0,0 +1,30 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Google Tasks +description: Read, create, update, organize, and complete Google Tasks through the Google Tasks API Discovery document. +tags: + - google + - tasks + - productivity +googleDiscoveryApi: + discoveryUrl: https://www.googleapis.com/discovery/v1/apis/tasks/v1/rest + auth: + type: oauth2 + issuer: https://accounts.google.com + clientId: $vault:GOOGLE_CLIENT_ID + clientSecret: $vault:GOOGLE_CLIENT_SECRET + scopes: + - https://www.googleapis.com/auth/tasks +--- + +# Google Tasks + +Use this Caplet when an agent needs to inspect or manage Google Tasks during planning, follow-up, or personal workflow coordination. + +## Scope Guidance + +Google Tasks offers `tasks.readonly` for read-only workflows and `tasks` for creating, editing, organizing, and deleting tasks. This install-ready Caplet includes the mutating `tasks` scope because normal task management requires it; use a private read-only variant if the agent should never mutate task state. + +## Use Carefully + +List existing tasklists and tasks before mutating. Confirm task names, due dates, and tasklist IDs before creating, completing, moving, or deleting tasks. diff --git a/caplets/lsp/CAPLET.md b/caplets/lsp/CAPLET.md index 5f3513b0..76879c3d 100644 --- a/caplets/lsp/CAPLET.md +++ b/caplets/lsp/CAPLET.md @@ -8,6 +8,8 @@ tags: - lsp - language-server - diagnostics +projectBinding: + required: true mcpServer: command: npx args: [-y, language-server-mcp] @@ -19,6 +21,8 @@ Use this Caplet to expose Language Server Protocol capabilities through `languag The server runs over stdio, starts local language servers lazily, and gives agents project-aware code intelligence for repositories that have LSP configuration or supported built-in language servers. +Project Binding is required because all useful LSP operations need a trustworthy bound project root for workspace-relative files, language-server startup, diagnostics, and edit containment. + ## Good Fits - Inspect hover/type information before editing unfamiliar code. diff --git a/caplets/posthog/CAPLET.md b/caplets/posthog/CAPLET.md new file mode 100644 index 00000000..8200ca3d --- /dev/null +++ b/caplets/posthog/CAPLET.md @@ -0,0 +1,29 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: PostHog +description: Inspect PostHog analytics, feature flags, experiments, session replay, and product telemetry through PostHog's hosted MCP server. +tags: + - analytics + - posthog + - product + - feature-flags +mcpServer: + transport: http + url: https://mcp.posthog.com/mcp + auth: + type: oauth2 +--- + +# PostHog + +Use this Caplet when an agent needs product analytics or feature-flag context from PostHog before planning, debugging, or validating a change. + +## Good Fits + +- Query trends, funnels, retention, or HogQL for a product question. +- Inspect feature flags, experiments, or rollout state before changing code. +- Search session replays and event data while investigating user-reported behavior. + +## Use Carefully + +PostHog MCP includes mutating tools for flags, insights, dashboards, and other project state. Prefer read-only inspection first, review planned mutations, and keep OAuth access scoped to the PostHog organization and project you intend to expose. diff --git a/caplets/repo-cli/CAPLET.md b/caplets/repo-cli/CAPLET.md index 73499bec..74c5529c 100644 --- a/caplets/repo-cli/CAPLET.md +++ b/caplets/repo-cli/CAPLET.md @@ -5,6 +5,8 @@ description: Inspect and run common local repository workflows through curated C tags: - cli - code +projectBinding: + required: true cliTools: actions: git_status: @@ -35,3 +37,5 @@ cliTools: # Repository CLI Use this Caplet to expose a small, typed set of local repository commands without giving an agent arbitrary shell access. + +Project Binding is required because every command is meant to run against the attached repository. The bound root prevents the agent from accidentally treating an unrelated working directory as the target project. diff --git a/caplets/sentry/CAPLET.md b/caplets/sentry/CAPLET.md new file mode 100644 index 00000000..b9d78301 --- /dev/null +++ b/caplets/sentry/CAPLET.md @@ -0,0 +1,29 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Sentry +description: Inspect Sentry issues, events, traces, releases, and AI debugging context through Sentry's hosted MCP server. +tags: + - observability + - sentry + - errors + - tracing +mcpServer: + transport: http + url: https://mcp.sentry.dev/mcp + auth: + type: oauth2 +--- + +# Sentry + +Use this Caplet when an agent needs live Sentry context while debugging production errors, investigating traces, or checking release health. + +## Good Fits + +- Find the highest-impact issues for a project and time range. +- Inspect event details, stack traces, tags, breadcrumbs, and suspect commits. +- Correlate deploys or releases with new errors before changing code. + +## Use Carefully + +Sentry data can contain user, request, and environment details. Ask for narrow projects and time windows, summarize only the needed debugging context, and review any mutating tool calls before applying changes to Sentry state. diff --git a/caplets/stealth-browser-use/CAPLET.md b/caplets/stealth-browser-use/CAPLET.md new file mode 100644 index 00000000..ac0f3872 --- /dev/null +++ b/caplets/stealth-browser-use/CAPLET.md @@ -0,0 +1,38 @@ +--- +# yaml-language-server: $schema=https://caplets.dev/caplet.schema.json +name: Stealth Browser Use +description: Drive a stealth-configured local Playwright browser for web workflows that need a non-default browser profile. +tags: + - browser + - playwright + - stealth + - local-control +mcpServer: + command: npx + args: + - -y + - "@playwright/mcp@latest" + - "--config=$env:CAPLETS_STEALTH_PLAYWRIGHT_CONFIG" + - --browser=firefox + - "--executable-path=$env:CAPLETS_STEALTH_BROWSER_EXECUTABLE" + - "--user-data-dir=$env:CAPLETS_STEALTH_BROWSER_USER_DATA_DIR" + - --headless +--- + +# Stealth Browser Use + +Use this Caplet when a workflow needs a dedicated local browser profile and the ordinary Browser Use Caplet is not suitable. + +## Setup + +Create a local Playwright MCP config file and set: + +- `CAPLETS_STEALTH_PLAYWRIGHT_CONFIG` +- `CAPLETS_STEALTH_BROWSER_EXECUTABLE` +- `CAPLETS_STEALTH_BROWSER_USER_DATA_DIR` + +Do not check browser profiles, cookies, or machine-specific paths into a public Caplet. + +## Safety + +This is a local-control Caplet. Follow site terms, avoid credential entry, and review any mutating browser interaction before it runs. diff --git a/docs/brainstorms/2026-06-26-prebuilt-caplets-catalog-requirements.md b/docs/brainstorms/2026-06-26-prebuilt-caplets-catalog-requirements.md new file mode 100644 index 00000000..fbe2d2ee --- /dev/null +++ b/docs/brainstorms/2026-06-26-prebuilt-caplets-catalog-requirements.md @@ -0,0 +1,213 @@ +--- +date: 2026-06-26 +topic: prebuilt-caplets-catalog +--- + +# Prebuilt Caplets Catalog Requirements + +## Summary + +Caplets should expand its public prebuilt catalog from a seed set into install-ready coverage for a real agent power-user stack, backed by lockfile-aware install and update commands. The milestone also adds a `writing-caplets` skill so agents can author catalog-grade Caplets with the same safety, setup, auth, and Project Binding expectations. + +--- + +## Problem Frame + +The current catalog already proves that Caplets can package useful backends, but it is still closer to a curated example set than a lived-in stack. The next catalog milestone should show that Caplets can carry the practical integrations an agent builder already uses across coding, planning, observability, Google workspace, browser automation, and desktop control. + +The install lifecycle is also too shallow for a growing catalog. `caplets install` copies files from a source repository, but it does not record the source revision, installed path, or update relationship needed for repeatable installs and safe updates. + +--- + +## Key Decisions + +- **Mirror the personal stack into public install-ready Caplets.** The first catalog expansion should include missing integrations from the user's working config instead of limiting high-risk surfaces to recipes. +- **Update existing catalog entries selectively.** Existing public-friendly entries stay unless the personal config is clearly more production-real, better aligned with hosted provider behavior, or the entry needs metadata changes to satisfy the catalog quality bar. +- **Promote install-ready entries only after validation.** A catalog entry should not count toward install-ready coverage until it has an explicit verification status and a reproducible read-only, dry-run, or no-op validation path. +- **Use Project Binding only for project-file consumers.** A Caplet requires Project Binding when its backend reads, edits, indexes, or runs commands against the current project; being a local process is not enough. +- **Treat lockfiles as part of the install contract.** `caplets install` and `caplets update` should operate from recorded source metadata, not from copied files alone. +- **Let install restore from the lockfile.** `caplets install` with no source arguments should install the Caplets described by the selected project or global lockfile. +- **Keep lockfiles share-safe and integrity-aware.** Lockfiles should prove what was installed without storing credentials or pretending non-portable local sources can be restored elsewhere. +- **Keep live smoke testing outside this brainstorm.** The requirements should specify setup and verification metadata, while the user will run live smoke tests in their own environment. + +--- + +## Actors + +- A1. **Agent power-user.** Installs prebuilt Caplets globally or into a project and expects agents to receive a compact, useful capability surface. +- A2. **Catalog authoring agent.** Writes or updates Caplet files and needs a repeatable quality bar. +- A3. **Caplets CLI.** Installs, records, and updates catalog Caplets. +- A4. **Catalog source repository.** Provides installable Caplet files under a `caplets/` directory. +- A5. **Project-bound Caplet.** Requires the current project root before it can run safely. +- A6. **Risky local capability.** Exposes browser, desktop, or other locally powerful behavior and needs explicit setup and safety guidance. + +--- + +## Requirements + +**Catalog Expansion** + +- R1. The public catalog must add install-ready entries for Browser Use, Computer Use, PostHog, Sentry, Gmail, Google Drive, Google Tasks, and Stealth Browser Use. +- R2. Each new catalog entry must be usable as a normal `caplets install` target rather than only as prose documentation. +- R3. Each new catalog entry must include a concise usage description, tags, auth or setup guidance, and safety notes appropriate to the backend's mutating or high-risk capabilities. +- R4. Existing catalog entries must be revised when the personal config provides a materially better public default, such as a hosted OAuth endpoint replacing a local package install, or when Project Binding, auth, setup, verification, safety, or lockfile metadata requirements make the current entry inconsistent with the catalog quality bar. +- R5. The catalog must keep Caplets framed as a Code Mode-first capability layer for coding agents, not as a generic marketplace of every possible integration. +- R6. Each install-ready entry must name its primary Code Mode workflow and the compact capability surface an agent should use first. +- R7. Each install-ready entry must record verification status and at least one reproducible validation path, preferring read-only or dry-run checks and using local no-op checks when the real provider action would mutate state. +- R8. Entries without successful validation may be documented as unverified drafts or recipes, but must not count toward install-ready catalog coverage. +- R9. The catalog must avoid committing user-specific secrets, machine paths, workspace names, tokens, browser profile paths, or private provider identifiers. + +**Project Binding** + +- R10. Catalog entries whose backends consume the current project files must declare Project Binding as required. +- R11. `lsp`, `ast-grep`, and repository-local CLI catalog entries must require Project Binding unless planning finds a stronger reason to split them into project-bound and non-project variants. +- R12. Browser, desktop-control, and hosted SaaS entries must not require Project Binding solely because they run locally or can affect a project indirectly. +- R13. A project-bound catalog entry must explain why it needs the bound project root and what behavior degrades or disappears without it. +- R14. Project-bound catalog entries must rely on the runtime's bound project context rather than hardcoded local paths. + +**Install And Update Lifecycle** + +- R15. `caplets install` must write or update an entry in the selected scope's lockfile for every installed catalog Caplet. +- R16. Project installs must write their lockfile at `./.caplets.lock.json`. +- R17. Global installs must write their lockfile in the target machine's Caplets state directory, with the Linux default path `~/.local/state/caplets/caplets.lock.json` and platform-equivalent paths on Windows and macOS; remote global installs therefore update the remote machine's global lockfile rather than a local project lockfile. +- R18. Each lock entry must record the Caplet ID, source repository, normalized credential-free source URL or local source identity, source path inside the repository, destination path, installed kind, tracked source ref or channel, resolved git revision when available, content hash, install time, update time, and portability status. +- R19. Installs from a Git repository must resolve and record the exact commit revision used for the copied Caplet and the source ref or channel future updates should track. +- R20. Installs from a local directory must record enough source metadata to explain provenance and portability, and must mark the entry as not reliably restorable or updatable when no stable git revision or available local source can be resolved. +- R21. `caplets install` with no source arguments must read the selected scope's lockfile and install the Caplets described there. +- R22. No-argument `caplets install` must honor project and global scope flags so project lockfiles restore project installs and global lockfiles restore global installs. +- R23. No-argument `caplets install` must be idempotent when installed files match the lockfile content and must surface conflicts when installed files differ. +- R24. No-argument `caplets install` must restore from the exact recorded revision when one is available and verify the recorded content hash before writing installed files. +- R25. `caplets update` must read the selected scope's lockfile, resolve updates against the recorded source repository and tracked source ref or channel, replace selected installed Caplets when updates are available, and update the lock entries. +- R26. `caplets update` must support updating all tracked Caplets and updating named Caplets. +- R27. `caplets update` must support project and global scopes consistently with `caplets install`. +- R28. `caplets update` must not silently overwrite local modifications that differ from the recorded content hash unless the user chooses a force path. +- R29. `caplets update` must detect risk-increasing changes before replacement, including backend family changes, broader auth requirements, new Project Binding requirements, new mutating operations, or changed high-risk safety metadata, and must require confirmation or a force path before applying them. +- R30. Lockfile writes must be atomic enough that interruption does not leave an unreadable or partially written lockfile. +- R31. Lockfile operations must expose machine-readable errors for missing lockfiles, missing tracked Caplets, deleted upstream repositories, unavailable revisions, content hash mismatches, unavailable local sources, non-portable local-source entries, and local modification conflicts. +- R32. Project lockfiles must be share-safe by default: they must not store credential-bearing URLs, must prefer project-relative destinations over absolute local paths, must mark local directory entries as non-portable when appropriate, and must warn when provenance metadata may expose private source identity. + +**Writing-Caplets Skill** + +- R33. The repo must add a `writing-caplets` skill under `skills/`. +- R34. The skill must teach agents how to author catalog-grade Caplets, not just how to use configured Caplets. +- R35. The skill must distinguish catalog-grade install-ready entries from local/private Caplets, unverified drafts, and recipes. +- R36. The skill must instruct agents to inspect the Caplet schema, existing catalog style, and relevant docs before writing a new Caplet. +- R37. The skill must include guidance for setup and verify metadata, auth and Vault references, least-privilege provider scopes, Project Binding, runtime features, Code Mode workflows, safety notes, and avoiding secrets. +- R38. The skill must tell agents when to create bundled reference files next to a directory Caplet. +- R39. The skill must tell agents to validate Caplet files and run the relevant focused checks after authoring. + +**Safety And Documentation** + +- R40. High-risk install-ready entries such as Computer Use and Stealth Browser Use must include explicit setup, scope, and safety guidance before they are promoted into the public catalog. +- R41. Local-control entries such as Browser Use, Computer Use, Stealth Browser Use, and desktop-control backends must treat installation as user acceptance of their local-control risk and must document bounded targets, browser profile or device assumptions, and credential isolation expectations. +- R42. Mutating SaaS entries such as Gmail, Google Drive, Google Tasks, Linear, GitHub, Sentry, and PostHog must guide agents to read first and write deliberately. +- R43. Catalog entries using OAuth, bearer tokens, or provider client credentials must document the expected Caplets auth or Vault setup without exposing credential values, list minimum required provider scopes or permissions, distinguish read-only from mutating scopes when providers support it, and warn against overbroad credentials. +- R44. Documentation examples must show lockfile-aware install and update workflows once the lifecycle commands exist. + +--- + +## Key Flows + +- F1. Install a project catalog Caplet + - **Trigger:** A user installs a Caplet from a source repo while working in a project. + - **Actors:** A1, A3, A4 + - **Steps:** The CLI resolves the source, copies the selected Caplet into the project Caplets root, records provenance in `./.caplets.lock.json`, and reports the installed ID and destination. + - **Covered by:** R15, R16, R18, R19 + +- F2. Update tracked project Caplets + - **Trigger:** A user runs `caplets update` in a project with a lockfile. + - **Actors:** A1, A3, A4 + - **Steps:** The CLI reads the project lockfile, checks each selected source channel for newer content, refuses local edits or risk-increasing changes without confirmation, replaces safe-to-update installed Caplets, and rewrites the lockfile with the new revision and content hash. + - **Covered by:** R25, R26, R27, R28, R29, R30 + +- F3. Restore project Caplets from a lockfile + - **Trigger:** A user runs `caplets install` with no source arguments in a project with `./.caplets.lock.json`. + - **Actors:** A1, A3, A4 + - **Steps:** The CLI reads the project lockfile, installs any missing tracked Caplets from their exact recorded source metadata, verifies content hashes before writing, leaves matching installed files alone, and reports conflicts or non-portable local sources. + - **Covered by:** R21, R22, R23, R24, R31 + +- F4. Use a project-bound catalog Caplet + - **Trigger:** An agent receives an installed Caplet whose backend needs current project files. + - **Actors:** A1, A5 + - **Steps:** The Caplet is exposed only when a valid Project Binding context exists, and its backend uses the bound project root rather than a hardcoded path. + - **Covered by:** R10, R13, R14 + +- F5. Author a new catalog-grade Caplet + - **Trigger:** An agent is asked to add or update a prebuilt catalog entry. + - **Actors:** A2, A4 + - **Steps:** The agent loads the `writing-caplets` skill, checks schema and existing catalog patterns, writes the Caplet and any nearby references, validates it, and reports verification. + - **Covered by:** R33, R34, R36, R37, R38, R39 + +- F6. Install a high-risk local capability + - **Trigger:** A user installs Computer Use, Browser Use, or Stealth Browser Use from the catalog. + - **Actors:** A1, A3, A6 + - **Steps:** The catalog entry installs normally, with installation treated as user acceptance of the local-control risk, and carries setup and safety guidance that makes the risk explicit. + - **Covered by:** R1, R2, R40, R41, R43 + +--- + +## Acceptance Examples + +- AE1. **Covers R15, R16, R18, R19.** Given a user runs `caplets install spiritledsoftware/caplets sentry` in a project, when install succeeds, then `./.caplets.lock.json` records the `sentry` entry with source repo, source path, destination, tracked source ref, git revision, content hash, and timestamps. +- AE2. **Covers R17, R22, R27.** Given a user installs, restores, or updates global Caplets, when the command succeeds, then the global lockfile is read from or written under the Caplets state directory rather than the current project. +- AE3. **Covers R21, R22, R23, R24.** Given a project lockfile tracks `posthog` and `sentry`, when the user runs `caplets install` with no source arguments, then Caplets installs missing tracked entries from their recorded revisions, verifies content hashes, and leaves matching installed entries unchanged. +- AE4. **Covers R25, R26.** Given a lockfile tracks `posthog` and `sentry`, when the user runs `caplets update sentry`, then only the selected tracked Caplet is considered for replacement and its recorded source channel determines what update is eligible. +- AE5. **Covers R28, R29.** Given an installed Caplet was locally edited after install or an update broadens its auth scope, when install restore or update would replace it, then the CLI refuses or asks for a force path instead of silently overwriting the edit or expanding risk. +- AE6. **Covers R10, R11, R14.** Given an agent exposes the `lsp` catalog Caplet without Project Binding context, then the Caplet is withheld from callable surfaces rather than starting against an arbitrary process working directory. +- AE7. **Covers R33, R37, R39.** Given an agent uses `writing-caplets` to add a Google Drive Caplet, then the skill guides it to document OAuth setup, least-privilege scopes, mutating-operation caution, schema validation, and focused checks. +- AE8. **Covers R5, R6, R7, R8.** Given a new provider entry has not completed a reproducible validation path, when the catalog is assessed for this milestone, then the entry is marked unverified and does not count toward install-ready coverage. +- AE9. **Covers R31, R32.** Given a project lockfile is written after installing from a credential-bearing URL or local directory, then the lockfile strips credentials, avoids absolute project paths where possible, marks local-source portability, and reports non-portable restore behavior with a machine-readable error. + +--- + +## Success Criteria + +- The catalog includes install-ready entries for the missing personal-stack integrations named in R1, each with verification status and a reproducible validation path. +- Each install-ready entry names the primary Code Mode workflow and compact capability surface an agent should use first. +- Project-file consumer Caplets declare and use Project Binding consistently with the runtime contract. +- High-risk local-control entries treat installation as user acceptance of local-control risk and make that risk explicit in setup and safety guidance. +- `caplets install` produces a share-safe, integrity-aware lockfile entry for local project and global installs. +- `caplets install` with no source arguments restores the Caplets described by the selected project or global lockfile and verifies recorded content before writing files. +- `caplets update` can refresh tracked Caplets from lockfile provenance and the recorded source channel without requiring the user to remember the original install command. +- `caplets update` refuses local modifications and risk-increasing changes unless the user chooses a confirmation or force path. +- `writing-caplets` gives a future agent enough guidance to author a safe catalog entry without rediscovering the quality bar from scattered docs. +- The user can perform live smoke tests against their own provider accounts and machine-local dependencies after implementation. + +--- + +## Scope Boundaries + +- Hosted registry search, ratings, popularity, or marketplace UX are outside this milestone. +- Recipe-only treatment for Computer Use or Stealth Browser Use is outside this milestone; those entries should be install-ready with strong safety guidance. +- Remote-control project install semantics may be planned separately if they add risk beyond the local project and global lockfile behavior. +- Live provider smoke testing is owned by the user, not by this requirements artifact. + +--- + +## Dependencies / Assumptions + +- The current Caplet file schema supports `projectBinding.required`, setup metadata, runtime requirements, and the backend families needed for the catalog entries. +- The current install command already resolves a source repo and copies selected Caplet files, but it does not maintain lock state. +- The target machine's Caplets state directory remains the right home for global install metadata, including global installs performed against a remote host. +- Provider endpoints and auth models may change; implementation should verify current public endpoints and auth requirements before finalizing each catalog entry. +- Local-directory installs are allowed, but lockfile restore and update must fail closed when the local source cannot be proven available and unchanged. +- The first lockfile version can be Caplets-specific and does not need to copy the skills CLI schema exactly. + +--- + +## Implementation Discovery + +- Inventory existing catalog entries during implementation, starting with Context7, and update only the entries whose endpoint, auth, setup, Project Binding, verification, safety, or lockfile metadata falls short of the catalog quality bar. + +--- + +## Sources / Research + +- `STRATEGY.md` for the Code Mode-first product frame. +- `CONTEXT.md` and `CONCEPTS.md` for Caplet and Project Binding vocabulary. +- `apps/docs/src/content/docs/reference/caplet-files.mdx` for Caplet file fields and the Project Binding example. +- `docs/project-binding.md` for the runtime Project Binding contract. +- `packages/core/src/cli/install.ts` and `packages/core/src/cli.ts` for current install behavior. +- `skills/caplets/SKILL.md` for the existing Caplets usage skill that `writing-caplets` must complement rather than duplicate. +- `/tmp/compound-engineering/ce-brainstorm/caplets-catalog-1782465510/grounding.md` for the extraction dossier used during brainstorming. +- Public skills-lock discussions in `vercel-labs/skills` issues 283 and 549 for the lockfile lifecycle analogy. diff --git a/docs/plans/2026-06-26-001-feat-prebuilt-caplets-catalog-and-lockfile-update-plan.md b/docs/plans/2026-06-26-001-feat-prebuilt-caplets-catalog-and-lockfile-update-plan.md new file mode 100644 index 00000000..76eca2d5 --- /dev/null +++ b/docs/plans/2026-06-26-001-feat-prebuilt-caplets-catalog-and-lockfile-update-plan.md @@ -0,0 +1,316 @@ +--- +title: "feat: Build prebuilt Caplets catalog and lockfile update" +type: feat +date: 2026-06-26 +origin: docs/brainstorms/2026-06-26-prebuilt-caplets-catalog-requirements.md +--- + +# feat: Build prebuilt Caplets catalog and lockfile update + +## Summary + +Build the next public catalog milestone by expanding install-ready Caplets, adding structured catalog-readiness metadata, and wrapping the existing safe install copier with lockfile-aware install, restore, and update workflows. The implementation keeps Caplets Code Mode-first, uses existing Project Binding runtime behavior for project-file consumers, and treats remote catalog install/update as operating on the remote machine's global Caplets state for this milestone. + +--- + +## Problem Frame + +The current prebuilt catalog proves the Caplet file format, but it still reads like a seed set rather than a real agent power-user stack. The user's personal configuration already demonstrates useful integrations across browser automation, desktop control, observability, Google workspace APIs, repository tooling, and hosted MCP providers. This milestone turns that working stack into public, install-ready Caplets without copying user-specific secrets, paths, provider identifiers, or local assumptions. + +The install lifecycle is also too shallow for a public catalog. `caplets install` currently resolves a local path or Git source, validates selected Caplet files, preflights destination safety, and copies files. It does not record the source revision, future update channel, installed content hash, portability, or risk metadata needed for repeatable restore and safe update. The plan preserves the existing destination-safety work and adds a lockfile lifecycle around it. + +The adjacent product constraint is important: Caplets should remain a Code Mode-first capability layer for coding agents, not a generic marketplace. Catalog entries should teach the agent which compact workflow to use first, and high-risk local-control entries should be explicit about setup and scope without adding extra install-time confirmation beyond the user's decision to install. + +--- + +## Requirements + +Plan requirement IDs use `P-R*` so they do not collide with the origin brainstorm's `R1`-`R44` IDs. + +**Catalog Expansion And Authoring** + +- P-R1. Add install-ready public catalog entries for Browser Use, Computer Use, PostHog, Sentry, Gmail, Google Drive, Google Tasks, and Stealth Browser Use. Origin refs: R1, R2. +- P-R2. Each install-ready catalog entry includes public-safe description, tags, auth or setup guidance, safety notes, primary Code Mode workflow, compact capability surface, verification status, and reproducible validation path. Origin refs: R3, R5, R6, R7, R8, R9, R40, R41, R42, R43. +- P-R3. Existing catalog entries are revised only when they fail the new catalog quality bar or the personal config reveals a materially better public default. Origin refs: R4, Implementation Discovery. +- P-R4. Project-file consumer catalog entries declare Project Binding and explain why the bound project root is required. Origin refs: R10, R11, R12, R13, R14. +- P-R5. Add a `writing-caplets` skill that teaches agents how to author, review, and validate catalog-grade Caplets without duplicating the existing Caplets usage skill. Origin refs: R33-R39, F5, AE7. + +**Install, Restore, And Update** + +- P-R6. Successful `caplets install` writes or updates a selected-scope lock entry for every installed Caplet. Origin refs: R15, R16, R17, R18, AE1, AE2. +- P-R7. Project lockfiles live at `./.caplets.lock.json`; global lockfiles live under the target machine's Caplets state directory; remote catalog installs and updates use the remote machine's global Caplets state. Origin refs: R16, R17, R22, R27, AE2. +- P-R8. Lock entries record Caplet ID, source repository, credential-free source identity, source path, destination path, installed kind, tracked source ref or channel, resolved git revision when available, content hash over the installed artifact, install/update timestamps, portability, and parsed risk summary. Origin refs: R18, R19, R20, R32, AE1, AE9. +- P-R9. `caplets install` with no source arguments restores the selected scope from its lockfile, is idempotent for matching installed content, restores exact recorded revisions when available, verifies recorded content hashes, and fails closed for conflicts or non-portable sources. Origin refs: R21, R22, R23, R24, R31, AE3, AE9. +- P-R10. `caplets update` reads the selected scope's lockfile, updates all or named Caplets from their recorded tracked source channels, and rewrites lock entries only after a successful replacement. Origin refs: R25, R26, R27, AE4. +- P-R11. Restore and update do not silently overwrite local modifications; update also blocks risk-increasing changes unless the user uses an explicit force path. Origin refs: R28, R29, AE5. +- P-R12. Lockfile operations are atomic enough to avoid unreadable partial writes and avoid losing a previously working installed Caplet during failed replacement. Origin refs: R30, R31, AE9. +- P-R13. Install, restore, and update support `--json` result output with stable per-entry statuses and machine-readable errors for missing lockfiles, missing tracked entries, deleted sources, unavailable revisions, content hash mismatches, unavailable local sources, non-portable local-source entries, local modification conflicts, risk blocks, and partial-state failures. Origin refs: R31, AE9. +- P-R14. Project lockfiles are share-safe by default and never persist credential-bearing URLs, user-specific local paths, tokens, browser profile paths, or provider secrets. Origin refs: R9, R32, AE9. + +**Documentation And Verification** + +- P-R15. Public docs show lockfile-aware install, no-argument restore, and update workflows once the commands exist. Origin refs: R44. +- P-R16. The implementation includes focused automated coverage for lockfile paths, lockfile integrity, install restore, update safety, remote-global routing, catalog schema, Project Binding metadata, and catalog guardrails. Origin refs: Success Criteria. +- P-R17. Live provider smoke testing is documented as user-owned and is not required for agent-side implementation completion. Origin refs: Key Decisions, Scope Boundaries. + +--- + +## Key Technical Decisions + +- **KTD1. Add a lockfile lifecycle service around the existing install copier.** `packages/core/src/cli/install.ts` already has useful source discovery, validation, destination preflight, symlink protection, duplicate detection, and copy logic. The plan should keep that behavior and add provenance, restore, update, hashing, and lock writes in a separate lifecycle layer rather than burying state management in command parsing. +- **KTD2. Use installed-artifact hashes, not source-tree hashes.** The lockfile should hash the materialized file or directory exactly as Caplets installed it, with sorted relative paths, file type, executable mode where meaningful, and file bytes after directory Caplet symlink materialization. This avoids the integrity gap raised in `vercel-labs/skills` issue 806, where a source-folder hash could not be recomputed from the installed copy. +- **KTD3. Store parsed risk summaries in lock entries.** Update risk checks should compare structured facts from parsed Caplet metadata: backend family, auth type and scopes, Project Binding requirement, runtime features, catalog safety classification, mutating/destructive hints, and instruction/body/reference manifest changes. Markdown body text can carry context, but update safety cannot depend on prose parsing. +- **KTD4. Add structured catalog metadata.** Verification status, validation path, primary Code Mode workflow, compact capability surface, and safety/risk classification become schema-backed Caplet metadata. This supports catalog quality checks and future update-risk detection better than prose conventions alone, and the primary workflow metadata must reach the agent-facing inspection/discovery surface rather than living only in schema validation. +- **KTD5. Keep remote project installs out of this milestone.** Internally separate host selection from install scope so the remote control server can choose the remote global root and remote global lockfile explicitly. Externally, `--remote` remains the remote catalog install/update form for this milestone and targets the remote machine's global Caplets state; remote project install semantics stay deferred. +- **KTD6. Use `caplets update`, not `caplets upgrade`.** The command should be added to the top-level command registry, completion logic, remote-control command set, docs, and tests with no `upgrade` alias in this milestone. +- **KTD7. Use `--force` as the non-interactive risk override.** High-risk local-control installation does not require extra confirmation because installation is user acceptance. Update risk increases and local modification overwrites still require an explicit force path; interactive prompting is not necessary for the first version. +- **KTD8. Project Binding catalog work is metadata cleanup, not runtime redesign.** The runtime already hides project-bound Caplets without valid Project Binding context. This milestone updates `lsp`, `ast-grep`, and repository-local CLI catalog entries to declare Project Binding and explain the bound-root dependency, then relies on existing exposure behavior. +- **KTD9. Treat local-source installs as provenance with portability limits.** Local paths may be useful for development, but lockfiles must mark them non-portable unless a stable Git revision and credential-free remote identity can be resolved. Restore and update fail closed when a local source cannot be proven available and unchanged. +- **KTD10. Keep provider smoke tests out of agent verification.** Implementation should verify public endpoints and schema validity before writing entries, but live account, OAuth, browser, and desktop smoke tests are user-owned after implementation. +- **KTD11. Treat lockfiles as untrusted input without adding a prompt gate.** Running install or update is user acceptance of the Caplet source risk, but restore/update still must validate destination containment, source identity syntax, content hashes, and risk summaries before writing. Human and JSON output should show source repo, ref, resolved revision, hash, and lock path for new or restored entries. +- **KTD12. Stage replacement before removing working artifacts.** Restore and update should stage new artifacts, validate and hash them, then swap into place with rollback or backup restore for directory replacements. A failed replacement should not delete a previously working installed Caplet while leaving the old lock entry behind. + +--- + +## High-Level Technical Design + +```mermaid +flowchart TB + CLI["caplets install/update"] --> Target["target resolver: local project, local global, remote global"] + Target --> Lock["lockfile store"] + CLI --> Source["source resolver"] + Lock --> Restore["restore/update plan"] + Source --> Select["discover selected Caplets"] + Select --> Validate["schema + catalog validation"] + Validate --> Risk["risk summary"] + Risk --> Preflight["destination and local-mod preflight"] + Preflight --> Copy["copy or replace Caplet"] + Copy --> Hash["hash installed artifact"] + Hash --> Write["atomic lock write"] + Write --> Report["human + JSON result"] +``` + +```mermaid +stateDiagram-v2 + [*] --> MissingLock + [*] --> Locked + MissingLock --> InstallFromSource: install repo + InstallFromSource --> Locked: copy succeeds and lock writes + Locked --> RestoreNoop: install with no args and hash matches + Locked --> RestoreConflict: installed hash differs + Locked --> RestoreMissing: installed artifact missing + RestoreMissing --> Locked: exact revision restored and hash verified + Locked --> UpdateCheck: update all or named + UpdateCheck --> UpdateNoop: source channel unchanged + UpdateCheck --> UpdateBlocked: local edit or risk increase + UpdateCheck --> Locked: replacement succeeds and lock writes + RestoreConflict --> Locked: force replacement succeeds + UpdateBlocked --> Locked: force replacement succeeds +``` + +```mermaid +sequenceDiagram + participant User + participant CLI + participant Remote as Remote control server + participant Lock as Remote global lockfile + participant Store as Remote global Caplets root + + User->>CLI: caplets install --remote repo sentry + CLI->>Remote: install from source + Remote->>Store: copy selected Caplet + Remote->>Lock: write remote global lock entry + Remote-->>CLI: installed id, destination, lock path + CLI-->>User: report remote global install +``` + +The command path has three separable concerns. The target resolver decides where the operation executes and which lockfile/destination root is authoritative. The source resolver decides which source revision and Caplet path are being installed or updated. The lifecycle service performs destination preflight, local modification checks, content hashing, and atomic lock writes. + +--- + +## Implementation Units + +### U1. Add lockfile path helpers, schema, and atomic store + +- **Goal:** Establish a reusable Caplets lockfile model before changing install behavior. +- **Requirements:** P-R6, P-R7, P-R8, P-R12, P-R14 +- **Origin refs:** R15-R18, R30-R32, AE1, AE2, AE9 +- **Dependencies:** None +- **Files:** `packages/core/src/config/paths.ts`, `packages/core/src/config.ts`, `packages/core/src/cli/lockfile.ts`, `packages/core/src/errors.ts`, `packages/core/test/config-paths.test.ts`, `packages/core/test/caplets-lockfile.test.ts` +- **Approach:** Add path helpers for the global lockfile under the Caplets state directory and the project lockfile at the current project's `./.caplets.lock.json`. The project helper should explicitly test default `.caplets/config.json` behavior and custom `CAPLETS_PROJECT_CONFIG` behavior so it never accidentally writes `.caplets/caplets.lock.json`. Define lockfile versioning, entry parsing, unknown-version failure, credential-free source identity, project-relative destination display where possible, local-source portability status, destination containment validation, and atomic JSON writes using the repo's existing private-dir/temp-file/rename pattern. Treat recorded destinations as display/provenance where possible; restore/update must resolve the final destination from the selected Caplets root and Caplet ID. +- **Test scenarios:** Linux default global path is `~/.local/state/caplets/caplets.lock.json`; Windows and macOS use platform-equivalent state locations; XDG and `LOCALAPPDATA` overrides are honored; project lock path is `./.caplets.lock.json`; malformed JSON, missing `version`, unsupported version, duplicate IDs, credential-bearing URLs, absolute project destinations in shareable fields, `..` traversal, symlinked parents, symlinked targets, and cross-root realpaths fail with machine-readable errors; interrupted temp writes never leave an unreadable final lockfile. +- **Verification:** Lockfile state can be read, validated, written atomically, and addressed by project/global/remote-global callers without invoking install. + +### U2. Refactor source resolution and content hashing for provenance + +- **Goal:** Make install sources produce exact provenance and installed-artifact hashes for lock entries. +- **Requirements:** P-R6, P-R8, P-R9, P-R12, P-R14 +- **Origin refs:** R18-R20, R23, R24, R31, R32, AE1, AE3, AE9 +- **Dependencies:** U1 +- **Files:** `packages/core/src/cli/install.ts`, `packages/core/src/cli/lockfile.ts`, `packages/core/src/caplet-source/parse.ts`, `packages/core/src/setup/hash.ts`, `packages/core/src/stable-json.ts`, `packages/core/test/cli.test.ts`, `packages/core/test/caplets-lockfile.test.ts`, `packages/core/test/caplet-source.test.ts` +- **Approach:** Replace the current source string convention of `repo#caplets/id` with separate source fields for repository identity, source path, tracked ref or channel, and resolved revision. Git installs should record the exact commit used and the future update channel; restore should fetch and checkout that exact revision when available and fail with an unavailable-revision error when the commit cannot be retrieved. Local installs should record whether the source is a Git worktree with a stable remote/revision, a dirty worktree, a detached commit, a local path inside the project, or a local path outside the project. Hash installed content after copy or restore, not the raw source tree. +- **Test scenarios:** Owner/repo shorthand records normalized GitHub URL without credentials; Git URL with branch or ref records the tracked source channel separately from the Caplet source path; restore uses the exact recorded commit; deleted or unavailable revisions return machine-readable errors; directory Caplet hashes are stable across filesystem ordering; symlink-materialized directory Caplets hash the installed files plus relevant type/mode metadata; extra, deleted, edited, or executable-mode-changed installed files produce hash mismatches; non-Git local directories are marked non-portable and fail closed on restore/update from a different machine. +- **Verification:** Every install plan can produce a lock entry with stable source provenance and a hash that can be recomputed from the installed artifact. + +### U3. Make `caplets install` write locks and restore from locks + +- **Goal:** Add lockfile writes to source installs and implement no-argument restore. +- **Requirements:** P-R6, P-R7, P-R8, P-R9, P-R11, P-R12, P-R13, P-R14 +- **Origin refs:** R15-R24, R28, R30-R32, F1, F3, AE1, AE2, AE3, AE5, AE9 +- **Dependencies:** U1, U2 +- **Files:** `packages/core/src/cli.ts`, `packages/core/src/cli/commands.ts`, `packages/core/src/cli/install.ts`, `packages/core/src/cli/lockfile.ts`, `packages/core/src/errors.ts`, `packages/core/test/cli.test.ts`, `packages/core/test/cli-completion.test.ts` +- **Approach:** Change the install command to accept an optional source argument and add `--json` output. When a source is present, preserve existing selected-Caplet install behavior and write lock entries after successful copy/hash. When no source is present, read the selected scope's lockfile and restore missing tracked Caplets from recorded metadata. Matching installed artifacts are no-ops; differing installed artifacts are local modification conflicts unless `--force` is used. Lockfile restore treats the shared lockfile as untrusted input, validates source identity and destination containment, and reports source repo/ref/revision/hash before or as part of installing new entries. Replacement should stage the new artifact, hash it, then swap or restore a backup so failed copy/hash/lock writes do not remove a previously working Caplet. +- **Test scenarios:** `caplets install repo sentry` in a project copies `sentry` and writes `./.caplets.lock.json`; `caplets install --global repo sentry` writes the global state lock; `caplets install --json` returns a stable success envelope with installed/restored/no-op entries; `caplets install` with no args restores missing entries from the selected lockfile; restore leaves matching entries untouched; restore refuses local edits; restore from a non-portable local-source entry fails with a machine-readable error; `--force` replaces a locally modified installed artifact and updates the lock entry; malformed lockfile destinations cannot escape the selected Caplets root; replacement failure rolls back to the prior artifact; missing lockfile and missing tracked Caplet errors are stable in JSON output; install argument completion continues to work without treating no-arg restore as an error. +- **Verification:** Install from source and install from lock both preserve existing destination safety and produce deterministic lock state. + +### U4. Add `caplets update` + +- **Goal:** Refresh tracked Caplets from lockfile provenance and recorded source channels. +- **Requirements:** P-R8, P-R10, P-R11, P-R12, P-R13, P-R14 +- **Origin refs:** R25-R32, F2, AE4, AE5, AE9 +- **Dependencies:** U1, U2, U3, U6 for risk-increase enforcement. Command plumbing can start after U3, but update safety cannot ship until the U6 risk-summary contract exists. +- **Files:** `packages/core/src/cli.ts`, `packages/core/src/cli/commands.ts`, `packages/core/src/cli/install.ts`, `packages/core/src/cli/lockfile.ts`, `packages/core/src/caplet-files.ts`, `packages/core/test/cli.test.ts`, `packages/core/test/cli-completion.test.ts` +- **Approach:** Add top-level `caplets update [caplets...]` with project/global scope flags, `--json`, and `--force`. The update lifecycle reads the selected lockfile, filters to named entries when provided, resolves each tracked source channel, compares the available source revision and installed artifact hash, blocks local modification conflicts, computes a new parsed risk summary, and replaces only entries that are safe or forced. Missing, partial, schema-version-incompatible, or otherwise incomparable old/new risk summaries are treated as risk-increasing and require `--force`. Body and bundled-reference changes are always surfaced in human/JSON output; for local-control and mutating SaaS Caplets, instruction/body/reference changes require `--force` even when backend metadata is unchanged. Replacement uses the same stage/hash/swap-or-rollback strategy as restore. +- **Test scenarios:** Updating with no names considers all tracked entries; updating with one name considers only that entry; missing tracked names produce machine-readable errors; unchanged source channels are reported as no-op; newer source revisions replace installed files and update timestamps/hash/revision; `caplets update --json` returns per-entry statuses for unchanged, updated, blocked, failed, and partial-state entries; local edits block replacement; backend family changes, broader auth scopes, new Project Binding, new runtime features, changed safety classification, missing/incomparable risk summaries, mutating/destructive hint changes, and risky instruction/body/reference changes block replacement without `--force`; `--force` applies the update and records the new risk summary; partial update failures restore the previous artifact and do not corrupt unrelated lock entries. +- **Verification:** Users can refresh catalog Caplets without remembering original install commands, and risk-increasing updates cannot apply silently. + +### U5. Route install and update through remote control as remote global operations + +- **Goal:** Make remote catalog install, restore, and update use the remote machine's global Caplets root and global lockfile. +- **Requirements:** P-R7, P-R9, P-R10, P-R12, P-R13 +- **Origin refs:** R17, R21, R22, R25, R27, R31, AE2, AE3, AE4 +- **Dependencies:** U1, U3, U4 +- **Files:** `packages/core/src/remote-control/types.ts`, `packages/core/src/remote-control/dispatch.ts`, `packages/core/src/remote-control/client.ts`, `packages/core/src/serve/http.ts`, `packages/core/src/cli.ts`, `packages/core/test/cli-remote.test.ts`, `packages/core/test/remote-control-dispatch.test.ts`, `packages/core/test/remote-control-client.test.ts`, `packages/core/test/serve-http.test.ts` +- **Approach:** Extend remote-control command types with install-from-source, install-from-lock, and update requests. The dispatch context and HTTP server control-context construction should expose the server's global Caplets root and global lockfile path rather than reusing `projectCapletsRoot` for catalog installs. The local CLI keeps remote/local overlay behavior for ordinary command execution, but explicit `--remote` install/update mutations are sent through the existing protected remote-control endpoint and operate on remote global state. Keep `--project --remote` out of this milestone; if `--global --remote` is accepted, treat it as an explicit spelling of the same remote-global target rather than a new project-scope feature. +- **Test scenarios:** `CAPLETS_MODE=remote caplets install repo sentry` without `--remote` still installs to the local project by default; `--project` and `--global` still target local project/global in remote mode; `--remote` sends an install request to the remote server; remote dispatch writes the remote global lock and remote global destination; remote no-arg install restores from the remote global lock; remote update reads and writes remote global lock entries; `serveHttp`, `serveHttpWithSessionFactory`, and control-context tests populate remote-global paths; unauthenticated, expired, revoked, or unauthorized remote clients cannot mutate remote global catalog state; remote errors propagate with stable codes and redacted messages; invalid remote project-scope combinations fail clearly. +- **Verification:** Remote catalog lifecycle state lives on the remote machine and does not mutate the caller's local project lockfile. + +### U6. Add structured catalog-readiness metadata to Caplet files + +- **Goal:** Make install-ready status, verification, Code Mode workflow, and risk metadata machine-readable. +- **Requirements:** P-R2, P-R3, P-R8, P-R11, P-R16 +- **Origin refs:** R3-R8, R29, R40-R43, AE5, AE8 +- **Dependencies:** U1. This unit should land before U4's risk-increase enforcement, though U4 command plumbing can start earlier. +- **Files:** `packages/core/src/config.ts`, `packages/core/src/config-runtime.ts`, `packages/core/src/caplet-files-bundle.ts`, `packages/core/src/capability-description.ts`, `packages/core/src/cli/inspection.ts`, `packages/core/src/code-mode/api.ts`, `packages/core/src/code-mode/runtime-api.d.ts`, `packages/core/src/code-mode/platform-entry.ts`, `schemas/caplet.schema.json`, `apps/landing/public/caplet.schema.json`, `apps/docs/src/content/docs/reference/caplet-files.mdx`, `packages/core/test/config.test.ts`, `packages/core/test/config-validation.test.ts`, `packages/core/test/caplet-files.test.ts`, `packages/core/test/code-mode-api.test.ts`, `apps/landing/test/schema-assets.test.ts` +- **Approach:** Add a small `catalog` metadata object shared by Caplet backend families. It should distinguish `install_ready`, `draft`, and `recipe`; include verification status and validation method; name the primary Code Mode workflow and compact surface; classify safety risk as standard, mutating SaaS, or local control; and expose mutating/destructive capability hints or deterministic backend-specific derivation inputs. Backend-specific derivation should be explicit for HTTP method families, OpenAPI/Google Discovery operations, GraphQL operations, CLI annotations, and dynamic MCP tools. Keep credential values and machine paths out of metadata. Do not reuse `setup.verify` as validation status because setup requirements can hide Caplets as setup-required, while catalog validation may be a read-only provider call or no-op workflow description. Thread the primary Code Mode workflow and compact surface through agent-facing inspection/card output so agents can discover it without reading full prose. +- **Test scenarios:** Caplet frontmatter accepts valid catalog metadata and rejects unknown or inconsistent values; install-ready entries require verification status, validation path, primary Code Mode workflow, compact surface, safety classification, and mutating/destructive summary or derivation data; draft/recipe entries can be explicitly unverified but do not count as install-ready; generated JSON schema and docs include the new metadata; runtime config parsing preserves catalog metadata for risk summaries without exposing secret-like values; Code Mode `inspect()` or the equivalent card/inspection path exposes primary workflow and compact surface; schema assets in docs and landing stay in sync. +- **Verification:** Catalog quality can be enforced by parsed metadata rather than Markdown conventions. + +### U7. Expand the public catalog and fix Project Binding declarations + +- **Goal:** Add the requested personal-stack integrations and bring existing project-file consumers up to the quality bar. +- **Requirements:** P-R1, P-R2, P-R3, P-R4, P-R14, P-R16, P-R17 +- **Origin refs:** R1-R14, R40-R43, F4, F6, AE6, AE8 +- **Dependencies:** U6 should land before final catalog entries; Project Binding edits can be prepared earlier +- **Files:** `caplets/browser-use/CAPLET.md`, `caplets/computer-use/CAPLET.md`, `caplets/posthog/CAPLET.md`, `caplets/sentry/CAPLET.md`, `caplets/gmail/CAPLET.md`, `caplets/google-drive/CAPLET.md`, `caplets/google-tasks/CAPLET.md`, `caplets/stealth-browser-use/CAPLET.md`, `caplets/lsp/CAPLET.md`, `caplets/ast-grep/CAPLET.md`, `caplets/repo-cli/CAPLET.md`, `caplets/coding-agent-toolkit/CAPLET.md`, `packages/core/test/catalog-vault.test.ts`, `packages/core/test/caplet-files.test.ts`, `packages/core/test/exposure-discovery.test.ts` +- **Approach:** Convert the user's useful integrations into public-safe, installable Caplets. Use hosted OAuth MCP endpoints for PostHog and Sentry when current public provider docs confirm them; if a named provider cannot be validated as install-ready through a public-safe endpoint/backend during implementation, stop and record the blocker rather than silently downgrading the required coverage. Use Google Discovery API entries for Gmail, Drive, and Tasks with least-privilege scopes documented and mutating actions framed read-first/write-deliberately. Use local-control setup and safety guidance for Browser Use, Computer Use, and Stealth Browser Use without committing personal executable paths, browser profile paths, tokens, or private provider IDs. Add `projectBinding.required` and bound-root explanations to `lsp`, `ast-grep`, and repository-local CLI entries unless implementation finds a stronger reason to split variants. Add new integrations to `coding-agent-toolkit` only where the grouping has a clear product role and does not make high-risk local-control tools feel silently bundled. +- **Test scenarios:** Every catalog `CAPLET.md` parses under the generated schema; install-ready entries include catalog metadata; named provider entries are not marked install-ready unless their public endpoint/backend and auth model were verified from current provider docs; secret-like env references in checked-in catalog files use Vault syntax or documented non-secret placeholders; browser/desktop entries include local-control safety guidance and do not require Project Binding solely because they run locally; Google entries document least-privilege scopes and mutating-operation caution; `lsp`, `ast-grep`, and repo-local CLI are withheld by exposure discovery without Project Binding context; project-bound entries do not hardcode local absolute paths. +- **Verification:** The public catalog contains the named entries as normal install targets, and project-file consumers declare Project Binding consistently. + +### U8. Add the `writing-caplets` skill + +- **Goal:** Give future agents a durable authoring workflow for catalog-grade Caplets. +- **Requirements:** P-R5, P-R16 +- **Origin refs:** R33-R39, F5, AE7 +- **Dependencies:** U6 informs the metadata guidance; U7 provides examples +- **Files:** `skills/writing-caplets/SKILL.md`, `skills/caplets/SKILL.md`, `packages/core/test/agent-plugins.test.ts` +- **Approach:** Add a sibling skill focused on authoring and reviewing Caplet files. The trigger should cover creating or updating catalog entries, setup/verify metadata, auth/Vault, least-privilege scopes, Project Binding, runtime requirements, Code Mode workflow notes, safety notes, bundled reference files, and validation checks. The body should tell agents to inspect the schema, generated docs, and nearby catalog examples before writing; distinguish install-ready catalog entries from local/private Caplets, drafts, and recipes; and defer usage/execution workflows to the existing `caplets` skill. Add a light structural test only if the repo already has skill validation coverage; avoid brittle body-substring tests. +- **Test scenarios:** The skill exists under `skills/writing-caplets/SKILL.md` with valid frontmatter; it directs agents to schema/docs/catalog examples; it covers catalog-grade vs private/draft/recipe distinctions; it includes setup/auth/Vault/Project Binding/runtime/Code Mode/safety/reference-file/check guidance; it does not duplicate long Code Mode handle API tables from `skills/caplets/SKILL.md`. +- **Verification:** A future agent can load one skill and write a catalog-grade Caplet without rediscovering the quality bar from scattered docs. + +### U9. Update public docs, generated artifacts, and release metadata + +- **Goal:** Document the new lifecycle and keep generated artifacts current. +- **Requirements:** P-R15, P-R16, P-R17 +- **Origin refs:** R44, Success Criteria +- **Dependencies:** U1 through U8 +- **Files:** `apps/docs/src/content/docs/install.mdx`, `apps/docs/src/content/docs/capabilities.mdx`, `apps/docs/src/content/docs/troubleshooting.mdx`, `apps/docs/src/content/docs/reference/caplet-files.mdx`, `README.md`, `schemas/caplet.schema.json`, `apps/landing/public/caplet.schema.json`, `.changeset/*.md` +- **Approach:** Add user-facing examples for install from source, install with no arguments from a lockfile, global lockfile location, remote-global behavior, update all, update named, local modification conflicts, and force update. Regenerate schema/docs artifacts when U6 changes schema. Add a changeset because CLI behavior, command surface, schema, and public package behavior change. +- **Test scenarios:** Docs examples use `caplets update`, not `upgrade`; global lockfile docs refer to state directories, not config directories; remote examples make clear that `--remote` targets remote global state; generated schema/docs are in sync; docs do not claim live provider smoke tests were performed by the implementing agent. +- **Verification:** Public docs and generated assets reflect the implemented command behavior and catalog schema. + +--- + +## System-Wide Impact + +- **CLI contract:** `install` gains a no-argument restore mode, `update` becomes a new top-level command, and command completion/help must reflect both without breaking existing source-install usage. +- **Remote control:** Remote catalog mutations move from project-root install behavior to remote-global state. This touches command types, dispatch context, client routing, and error propagation. +- **Persistent state:** Caplets gains a project lockfile and global state lockfile. Atomic writes, credential stripping, and share-safe path rendering become part of the install contract. +- **Config/schema:** Caplet file schema gains catalog-readiness metadata that must be threaded through parsed config, runtime config, generated JSON schema, landing schema assets, and docs references. +- **Agent-facing catalog:** Code Mode-first workflows and compact capability surfaces become catalog metadata, so agents can learn intended usage without reading long prose first. +- **Project Binding:** Existing project-bound runtime gating becomes more visible because public catalog entries will now declare the requirement where appropriate. +- **Security/privacy:** Lockfiles, catalog entries, and docs must avoid credentials, private source URLs, personal browser profiles, local machine paths, and provider account identifiers. +- **Release process:** CLI command changes, schema changes, catalog expansion, and lockfile behavior require focused tests, generated artifacts, a changeset, and full verification before release. + +--- + +## Risks & Dependencies + +- **Partial state between copy and lock write.** Mitigation: copy/hash before lock write, write lock atomically, and return explicit partial-state errors if the lock write fails after files were copied. +- **Credential leakage through source URLs or catalog metadata.** Mitigation: normalize and sanitize source identities before lock write, reject credential-bearing persisted URLs, extend catalog guardrail tests, and keep user-specific paths out of checked-in entries. +- **Risk-increase false negatives.** Mitigation: compare parsed metadata rather than prose; include backend family, auth, scopes, Project Binding, runtime, safety classification, and mutating/destructive hints in lock summaries. +- **Provider endpoint drift.** Mitigation: verify current public provider endpoints and auth models during implementation before finalizing entries; keep live account smoke tests user-owned. +- **Local-source portability ambiguity.** Mitigation: record portability explicitly and fail closed for non-portable restore/update instead of pretending local directories can be reproduced on another machine. +- **Remote scope confusion.** Mitigation: document that `--remote` catalog install/update targets remote global state in this milestone, and keep remote project install semantics deferred. +- **Schema churn across duplicated definitions.** Mitigation: update `config.ts`, `config-runtime.ts`, `caplet-files-bundle.ts`, generated schema, landing schema, generated docs, and schema tests together. +- **High-risk local-control catalog entries feeling silently bundled.** Mitigation: make safety metadata explicit and avoid adding those entries to bundled toolkit groupings unless the grouping's purpose clearly calls for them. + +--- + +## Scope Boundaries + +- Hosted registry search, marketplace ranking, popularity, ratings, and discovery UX are out of scope. +- `caplets upgrade` is out of scope; the command is `caplets update`. +- Remote project install/update semantics are out of scope unless required only as internal groundwork for remote-global behavior. +- Live provider OAuth/browser/desktop smoke testing is out of scope for the implementing agent; the user will perform it. +- Auto-updating Caplets without an explicit command is out of scope. +- Secret provisioning, OAuth client creation, browser profile creation, and provider account setup are documented but not automated by this milestone. +- Recipe-only treatment for Computer Use and Stealth Browser Use is out of scope; they should be install-ready with explicit safety guidance. + +--- + +## Open Questions + +### Resolved During Planning + +- **Command name:** Use `caplets update`, not `caplets upgrade`. +- **Remote global lockfile location:** Remote catalog install/update writes the global lockfile on the remote machine. +- **Local-control confirmation:** Installing a local-control Caplet is user acceptance of that risk; no extra confirmation prompt is required. +- **Live smoke testing:** User owns live provider and local machine smoke testing after implementation. +- **Machine-readable output:** Install, restore, and update add `--json` and return stable per-entry result/status data plus `CapletsError` codes for command failures. +- **Project lockfile path:** The default project lockfile is `./.caplets.lock.json`; custom project config tests must verify this does not drift to `.caplets/caplets.lock.json`. + +### Deferred To Implementation Discovery + +- **Exact provider scopes and endpoints:** Confirm current public provider endpoints and least-privilege scopes for PostHog, Sentry, Gmail, Drive, and Tasks before writing final catalog entries. +- **Toolkit grouping:** Decide entry-by-entry whether new catalog Caplets belong in `coding-agent-toolkit`; default to top-level installable entries unless the grouping has a clear product role. + +--- + +## Validation Plan + +- Run focused core tests for lockfile paths, lockfile store, install/restore/update, remote routing, config/schema parsing, catalog guardrails, Project Binding exposure, and CLI completion: + - `pnpm --filter @caplets/core test -- test/config-paths.test.ts test/caplets-lockfile.test.ts test/cli.test.ts test/cli-remote.test.ts test/remote-control-dispatch.test.ts test/remote-control-client.test.ts test/serve-http.test.ts test/config.test.ts test/config-validation.test.ts test/caplet-files.test.ts test/catalog-vault.test.ts test/exposure-discovery.test.ts test/code-mode-api.test.ts test/cli-completion.test.ts` +- When schema metadata changes, run `pnpm schema:generate` and `pnpm schema:check`. +- When docs references change, run `pnpm docs:generate` and `pnpm docs:check`. +- Run `pnpm format:check`, `pnpm lint`, `pnpm typecheck`, and targeted package builds after implementation. +- Run full `pnpm verify` before release or PR handoff. +- User performs live smoke tests against provider accounts, OAuth grants, browser automation, stealth browser setup, and desktop-control dependencies. + +--- + +## Sources / Research + +- `docs/brainstorms/2026-06-26-prebuilt-caplets-catalog-requirements.md` for the source requirements. +- `STRATEGY.md` for the Code Mode-first product frame. +- `CONTEXT.md`, `CONCEPTS.md`, and `docs/project-binding.md` for Caplets and Project Binding vocabulary. +- `packages/core/src/cli/install.ts` and `packages/core/src/cli.ts` for current install and command behavior. +- `packages/core/src/config/paths.ts` for platform state/config path patterns. +- `packages/core/src/remote-control/types.ts` and `packages/core/src/remote-control/dispatch.ts` for current remote install routing. +- `packages/core/src/config.ts`, `packages/core/src/config-runtime.ts`, and `packages/core/src/caplet-files-bundle.ts` for Caplet schema and runtime config threading. +- `apps/docs/src/content/docs/reference/caplet-files.mdx` for generated Caplet file documentation. +- `skills/caplets/SKILL.md` for the existing usage skill that `writing-caplets` complements. +- `~/.config/caplets/config.json` and `~/.config/caplets/*` inspected with secret/path redaction for personal-stack grounding. +- `vercel-labs/skills` issue 283 for lockfile install/sync expectations: https://github.com/vercel-labs/skills/issues/283 +- `vercel-labs/skills` issue 549 for restore-from-lock workflow pressure: https://github.com/vercel-labs/skills/issues/549 +- `vercel-labs/skills` issue 806 for installed-layout hash verification risk: https://github.com/vercel-labs/skills/issues/806 diff --git a/packages/core/src/caplet-files-bundle.ts b/packages/core/src/caplet-files-bundle.ts index d1a9c487..c5c4d3b5 100644 --- a/packages/core/src/caplet-files-bundle.ts +++ b/packages/core/src/caplet-files-bundle.ts @@ -107,7 +107,6 @@ const capletRuntimeRequirementsSchema = z }) .strict() .describe("Runtime feature and resource requirements for hosted execution."); - const capletAgentSelectionHintSchema = z .string() .trim() diff --git a/packages/core/src/cli.ts b/packages/core/src/cli.ts index d57a1921..8c0b5e76 100644 --- a/packages/core/src/cli.ts +++ b/packages/core/src/cli.ts @@ -2,7 +2,7 @@ import { Command, CommanderError, Option } from "commander"; import { Buffer } from "node:buffer"; import { existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { arch, platform } from "node:os"; -import { dirname, join } from "node:path"; +import { dirname, isAbsolute, join } from "node:path"; import { createInterface } from "node:readline/promises"; import { Writable } from "node:stream"; import { version as packageJsonVersion } from "../package.json"; @@ -54,7 +54,12 @@ import { formatVaultValueList, formatVaultValueStatus, } from "./cli/vault"; -import { installCaplets } from "./cli/install"; +import { + installCaplets, + restoreCapletsFromLockfile, + updateCapletsFromLockfile, +} from "./cli/install"; +import { readCapletsLockfile } from "./cli/lockfile"; import { formatSetupMenu, runInteractiveSetup, @@ -70,12 +75,14 @@ import { type LocalOverlayConfigWithSources, defaultUpdateCheckCacheDir, defaultUpdateCheckStateDir, + defaultCapletsLockfilePath, loadConfigWithSources, loadLocalOverlayConfigWithSources, resolveCapletsRoot, resolveConfigPath, resolveProjectCapletsRoot, resolveProjectConfigPath, + resolveProjectLockfilePath, vaultBootstrapResolver, } from "./config"; import { CapletsEngine } from "./engine"; @@ -406,6 +413,7 @@ function telemetryCommandFamilyFromArgs( if (command === cliCommands.setup) return { commandFamily: "setup", surface: "cli" }; if (command === cliCommands.init) return { commandFamily: "init", surface: "cli" }; if (command === cliCommands.install) return { commandFamily: "install", surface: "cli" }; + if (command === cliCommands.update) return { commandFamily: "install", surface: "cli" }; if (command === cliCommands.add) return { commandFamily: "add", surface: "cli" }; if (command === cliCommands.doctor) return { commandFamily: "doctor", surface: "cli" }; if (command === cliCommands.auth) return { commandFamily: "auth", surface: "cli" }; @@ -2897,41 +2905,86 @@ export function createProgram(io: CliIO = {}): Command { program .command(cliCommands.install) - .description("Install Caplets from a repo's caplets directory.") - .argument("", "local repo path, Git URL, or GitHub owner/repo") + .description("Install Caplets from a repo's caplets directory or restore from a lockfile.") + .argument("[repo]", "local repo path, Git URL, or GitHub owner/repo") .argument("[caplets...]", "optional Caplet IDs to install") .option("--project", "install to the project Caplets root") .option("-g, --global", "install to the user Caplets root") .option("--remote", "install through remote control") .option("--force", "overwrite installed Caplets") + .option("--json", "print JSON output") .action( async ( - repo: string, + repo: string | undefined, capletIds: string[], - options: MutationTargetOptions & { force?: boolean }, + options: MutationTargetOptions & { force?: boolean; json?: boolean }, ) => { printTelemetryNotice("cli"); - const target = parseMutationTarget(options); + const target = parseCatalogLifecycleTarget(options); + const localLockfilePath = + target === "remote" + ? undefined + : target === "global" + ? defaultCapletsLockfilePath(env) + : resolveProjectLockfilePath(process.cwd()); + const installSource = + repo && + isInstallSourceArgument(repo, { + allowImplicitLocalPath: target !== "remote", + lockfilePath: localLockfilePath, + }) + ? repo + : undefined; + const selectedCapletIds = repo && !installSource ? [repo, ...capletIds] : capletIds; if (target === "remote") { const remote = requireRemoteClientForTarget(io); const result = (await remote.request("install", { - repo, - capletIds, + ...(installSource ? { repo: installSource } : {}), + capletIds: selectedCapletIds, force: Boolean(options.force), })) as { installed: Array<{ id: string; destination: string }> }; + if (options.json) { + writeOut(`${JSON.stringify(installJsonResult(result.installed), null, 2)}\n`); + return; + } for (const caplet of result.installed) { writeOut(`Installed ${caplet.id} to remote ${caplet.destination}\n`); } return; } - const result = installCaplets(repo, { - capletIds, + const destinationRoot = + target === "global" + ? resolveCapletsRoot(resolveConfigPath(currentConfigPath())) + : envProjectCapletsRoot(env); + const lockfilePath = localLockfilePath ?? resolveProjectLockfilePath(process.cwd()); + if (!installSource) { + const result = restoreCapletsFromLockfile({ + capletIds: selectedCapletIds, + force: Boolean(options.force), + destinationRoot, + lockfilePath, + }); + if (options.json) { + writeOut(`${JSON.stringify(installJsonResult(result.installed), null, 2)}\n`); + return; + } + for (const caplet of result.installed) { + writeOut( + `${caplet.status === "noop" ? "Already installed" : "Restored"} ${caplet.id} to ${localMutationTargetLabel(target, io)}${caplet.destination}\n`, + ); + } + return; + } + const result = installCaplets(installSource, { + capletIds: selectedCapletIds, force: Boolean(options.force), - destinationRoot: - target === "global" - ? resolveCapletsRoot(resolveConfigPath(currentConfigPath())) - : envProjectCapletsRoot(env), + destinationRoot, + lockfilePath, }); + if (options.json) { + writeOut(`${JSON.stringify(installJsonResult(result.installed), null, 2)}\n`); + return; + } for (const caplet of result.installed) { writeOut( `Installed ${caplet.id} to ${localMutationTargetLabel(target, io)}${caplet.destination}\n`, @@ -2940,6 +2993,65 @@ export function createProgram(io: CliIO = {}): Command { }, ); + program + .command(cliCommands.update) + .description("Update installed Caplets from the selected lockfile.") + .argument("[caplets...]", "optional Caplet IDs to update") + .option("--project", "update the project Caplets root") + .option("-g, --global", "update the user Caplets root") + .option("--remote", "update through remote control") + .option("--force", "replace local modifications and risk-increasing changes") + .option("--json", "print JSON output") + .action( + async ( + capletIds: string[], + options: MutationTargetOptions & { force?: boolean; json?: boolean }, + ) => { + printTelemetryNotice("cli"); + const target = parseCatalogLifecycleTarget(options); + if (target === "remote") { + const remote = requireRemoteClientForTarget(io); + const result = (await remote.request("update", { + capletIds, + force: Boolean(options.force), + })) as { installed: Array<{ id: string; destination: string; status?: string }> }; + if (options.json) { + writeOut(`${JSON.stringify(installJsonResult(result.installed), null, 2)}\n`); + return; + } + for (const caplet of result.installed) { + writeOut( + `${caplet.status === "noop" ? "Already current" : "Updated"} ${caplet.id} at remote ${caplet.destination}\n`, + ); + } + return; + } + const destinationRoot = + target === "global" + ? resolveCapletsRoot(resolveConfigPath(currentConfigPath())) + : envProjectCapletsRoot(env); + const lockfilePath = + target === "global" + ? defaultCapletsLockfilePath(env) + : resolveProjectLockfilePath(process.cwd()); + const result = updateCapletsFromLockfile({ + capletIds, + force: Boolean(options.force), + destinationRoot, + lockfilePath, + }); + if (options.json) { + writeOut(`${JSON.stringify(installJsonResult(result.installed), null, 2)}\n`); + return; + } + for (const caplet of result.installed) { + writeOut( + `${caplet.status === "noop" ? "Already current" : "Updated"} ${caplet.id} at ${localMutationTargetLabel(target, io)}${caplet.destination}\n`, + ); + } + }, + ); + const add = program.command(cliCommands.add).description("Add generated Caplet files."); add @@ -3773,6 +3885,52 @@ function parseMutationTarget(options: MutationTargetOptions): MutationTarget { return "project"; } +function parseCatalogLifecycleTarget(options: MutationTargetOptions): MutationTarget { + const selected = [ + options.project ? "--project" : undefined, + options.global ? "--global" : undefined, + options.remote ? "--remote" : undefined, + ].filter((value): value is string => value !== undefined); + const allowedRemoteGlobal = options.remote && options.global && !options.project; + if (selected.length > 1 && !allowedRemoteGlobal) { + throw new CapletsError( + "REQUEST_INVALID", + `Cannot combine mutation target flags: ${selected.join(", ")}`, + ); + } + if (options.remote) return "remote"; + if (options.global) return "global"; + return "project"; +} + +function isInstallSourceArgument( + value: string, + options: { allowImplicitLocalPath?: boolean; lockfilePath?: string | undefined } = {}, +): boolean { + if (isExplicitLocalPath(value)) return true; + if (/^[a-z][a-z0-9+.-]*:\/\//i.test(value)) return true; + if (/^[^@\s]+@[^:\s]+:.+/.test(value)) return true; + if (/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+(?:\.git)?$/.test(value)) return true; + return ( + options.allowImplicitLocalPath !== false && + existsSync(value) && + !lockfileContainsCapletId(options.lockfilePath, value) + ); +} + +function isExplicitLocalPath(value: string): boolean { + return isAbsolute(value) || /^\.{1,2}(?:[\\/]|$)/.test(value) || /[\\/]/.test(value); +} + +function lockfileContainsCapletId(path: string | undefined, capletId: string): boolean { + if (!path) return false; + try { + return readCapletsLockfile(path).entries.some((entry) => entry.id === capletId); + } catch { + return false; + } +} + function parseVaultTarget(options: VaultTargetOptions): VaultTarget { const selected = [ options.global ? "--global" : undefined, @@ -3968,6 +4126,28 @@ function localMutationTargetLabel(target: Exclude, io: return remoteClientForCli(io) ? `${target} ` : ""; } +function installJsonResult( + installed: Array<{ + id: string; + destination: string; + status?: string | undefined; + hash?: string | undefined; + lockfile?: string | undefined; + source?: string | undefined; + }>, +) { + return { + entries: installed.map((entry) => ({ + id: entry.id, + status: entry.status ?? "installed", + destination: entry.destination, + ...(entry.lockfile ? { lockfile: entry.lockfile } : {}), + ...(entry.hash ? { hash: entry.hash } : {}), + ...(entry.source ? { source: entry.source } : {}), + })), + }; +} + function parseAuthFlagTarget(options: AuthTargetOptions): AuthTarget | undefined { const selected = [ options.project ? "--project" : undefined, diff --git a/packages/core/src/cli/commands.ts b/packages/core/src/cli/commands.ts index 7855338d..69337b8f 100644 --- a/packages/core/src/cli/commands.ts +++ b/packages/core/src/cli/commands.ts @@ -15,6 +15,7 @@ export const cliCommands = { doctor: "doctor", list: "list", install: "install", + update: "update", add: "add", inspect: "inspect", checkBackend: "check-backend", @@ -48,6 +49,7 @@ export const topLevelCommandNames = [ cliCommands.doctor, cliCommands.list, cliCommands.install, + cliCommands.update, cliCommands.add, cliCommands.inspect, cliCommands.checkBackend, diff --git a/packages/core/src/cli/install.ts b/packages/core/src/cli/install.ts index a319cdd0..c78301ca 100644 --- a/packages/core/src/cli/install.ts +++ b/packages/core/src/cli/install.ts @@ -1,4 +1,5 @@ import { execFileSync } from "node:child_process"; +import { createHash } from "node:crypto"; import { accessSync, constants, @@ -9,24 +10,37 @@ import { mkdirSync, mkdtempSync, readdirSync, + readFileSync, readlinkSync, realpathSync, + renameSync, rmSync, type Stats, statSync, } from "node:fs"; import { tmpdir } from "node:os"; import { basename, dirname, isAbsolute, join, parse, relative, resolve, sep } from "node:path"; +import { parse as parseYaml } from "yaml"; import { discoverCapletFiles, validateCapletFile } from "../caplet-files"; import { resolveProjectCapletsRoot } from "../config"; import { SERVER_ID_PATTERN } from "../config/validation"; import { CapletsError, toSafeError } from "../errors"; +import { + readCapletsLockfile, + validateLockfileDestination, + writeCapletsLockfile, + type CapletsLockEntry, + type CapletsLockSource, +} from "./lockfile"; type InstallableCaplet = { id: string; source: string; destination: string; kind: "file" | "directory"; + hash?: string | undefined; + status?: "installed" | "restored" | "updated" | "noop" | undefined; + lockfile?: string | undefined; }; type InstallPlan = InstallableCaplet & { @@ -34,12 +48,21 @@ type InstallPlan = InstallableCaplet & { sourceBoundary: string; }; +type LockedSourceResolution = { + sourcePath: string; + repoRoot: string; + resolvedRevision?: string | undefined; + cleanup: () => void; +}; + export function installCaplets( repo: string, options: { capletIds?: string[]; destinationRoot?: string; force?: boolean; + lockfilePath?: string | undefined; + now?: Date | undefined; } = {}, ): { installed: InstallableCaplet[] } { const source = resolveInstallSource(repo); @@ -73,18 +96,318 @@ export function installCaplets( for (const caplet of selected) { validateCapletFile(caplet.path); } - const installed = preflightInstallCaplets(selected, { + const plans = preflightInstallCaplets(selected, { destinationRoot, force: Boolean(options.force), repoRoot: source.repoRoot, sourceId: source.id, - }).map((plan) => installOneCaplet(plan, { force: Boolean(options.force) })); + }); + const now = options.now ?? new Date(); + const installed: InstallableCaplet[] = []; + for (const plan of plans) { + const caplet = installOneCaplet(plan, { force: Boolean(options.force) }); + const installedWithHash = { + ...caplet, + hash: hashInstalledArtifact(caplet.destination), + status: "installed" as const, + ...(options.lockfilePath ? { lockfile: options.lockfilePath } : {}), + }; + installed.push(options.lockfilePath ? installedWithHash : caplet); + if (options.lockfilePath) { + updateLockfileAfterInstall(options.lockfilePath, [plan], [installedWithHash], { + source, + now, + }); + } + } return { installed }; } finally { source.cleanup(); } } +export function restoreCapletsFromLockfile(options: { + destinationRoot?: string; + lockfilePath: string; + force?: boolean; + capletIds?: string[] | undefined; + now?: Date | undefined; +}): { installed: InstallableCaplet[] } { + const destinationRoot = options.destinationRoot ?? resolveProjectCapletsRoot(); + const lockfile = readCapletsLockfile(options.lockfilePath); + const selectedIds = new Set(options.capletIds ?? []); + const entries = lockfile.entries.filter( + (entry) => selectedIds.size === 0 || selectedIds.has(entry.id), + ); + const missing = [...selectedIds].filter( + (id) => !lockfile.entries.some((entry) => entry.id === id), + ); + if (missing.length > 0) { + throw new CapletsError( + "CONFIG_NOT_FOUND", + `Caplet ${missing.join(", ")} not found in lockfile`, + ); + } + if (entries.length === 0) { + throw new CapletsError( + "CONFIG_NOT_FOUND", + `No Caplets found in lockfile ${options.lockfilePath}`, + ); + } + + const nextEntries = new Map(lockfile.entries.map((entry) => [entry.id, entry])); + const results: InstallableCaplet[] = []; + for (const entry of entries) { + const destination = validateLockfileDestination(destinationRoot, entry.destination); + const existing = lstatIfExists(destination); + if (existing) { + const currentHash = hashInstalledArtifact(destination); + if (currentHash === entry.installedHash) { + results.push({ + id: entry.id, + source: lockSourceDisplay(entry.source), + destination, + kind: entry.kind, + hash: currentHash, + status: "noop", + lockfile: options.lockfilePath, + }); + continue; + } + if (!options.force) { + throw new CapletsError( + "CONFIG_EXISTS", + `Caplet ${entry.id} has local modifications at ${destination}; pass --force to replace it`, + ); + } + } + + const lockedSource = resolveLockedSource(entry.source); + try { + const plan: InstallPlan = { + id: entry.id, + source: lockSourceDisplay(entry.source), + sourcePath: lockedSource.sourcePath, + sourceBoundary: dirname(lockedSource.sourcePath), + destination, + kind: entry.kind, + }; + preflightInstallCaplets( + [ + { + id: entry.id, + path: + entry.kind === "directory" + ? join(lockedSource.sourcePath, "CAPLET.md") + : lockedSource.sourcePath, + }, + ], + { + destinationRoot, + force: true, + repoRoot: lockedSource.repoRoot, + sourceId: lockSourceDisplay(entry.source), + }, + ); + installOneCaplet(plan, { force: true }); + const hash = hashInstalledArtifact(destination); + if (hash !== entry.installedHash) { + nextEntries.set(entry.id, { + ...entry, + installedHash: hash, + updatedAt: (options.now ?? new Date()).toISOString(), + }); + writeCapletsLockfile(options.lockfilePath, { + version: 1, + entries: [...nextEntries.values()], + }); + } + results.push({ + id: entry.id, + source: lockSourceDisplay(entry.source), + destination, + kind: entry.kind, + hash, + status: "restored", + lockfile: options.lockfilePath, + }); + } finally { + lockedSource.cleanup(); + } + } + return { installed: results }; +} + +export function updateCapletsFromLockfile(options: { + destinationRoot?: string; + lockfilePath: string; + force?: boolean; + capletIds?: string[] | undefined; + now?: Date | undefined; +}): { installed: InstallableCaplet[] } { + const destinationRoot = options.destinationRoot ?? resolveProjectCapletsRoot(); + const lockfile = readCapletsLockfile(options.lockfilePath); + const selectedIds = new Set(options.capletIds ?? []); + const entries = lockfile.entries.filter( + (entry) => selectedIds.size === 0 || selectedIds.has(entry.id), + ); + const missing = [...selectedIds].filter( + (id) => !lockfile.entries.some((entry) => entry.id === id), + ); + if (missing.length > 0) { + throw new CapletsError( + "CONFIG_NOT_FOUND", + `Caplet ${missing.join(", ")} not found in lockfile`, + ); + } + if (entries.length === 0) { + throw new CapletsError( + "CONFIG_NOT_FOUND", + `No Caplets found in lockfile ${options.lockfilePath}`, + ); + } + + const nextEntries = new Map(lockfile.entries.map((entry) => [entry.id, entry])); + const results: InstallableCaplet[] = []; + for (const entry of entries) { + const destination = validateLockfileDestination(destinationRoot, entry.destination); + const existing = lstatIfExists(destination); + if (existing) { + const currentHash = hashInstalledArtifact(destination); + if (currentHash !== entry.installedHash && !options.force) { + throw new CapletsError( + "CONFIG_EXISTS", + `Caplet ${entry.id} has local modifications at ${destination}; pass --force to update it`, + ); + } + } + + const lockedSource = resolveLockedSource(entry.source, { useResolvedRevision: false }); + try { + if (!existsSync(lockedSource.sourcePath)) { + throw new CapletsError("CONFIG_NOT_FOUND", `Locked source for ${entry.id} is unavailable`); + } + const nextSource = refreshedLockSource(entry.source, lockedSource); + const sourceHash = + entry.kind === "directory" + ? hashDirectoryCapletInstallSource( + lockedSource.sourcePath, + realpathSync(dirname(lockedSource.sourcePath)), + ) + : hashInstalledArtifact(lockedSource.sourcePath); + const nextRisk = riskSummaryForSourcePath(lockedSource.sourcePath); + if (existing && sourceHash === entry.installedHash && !options.force) { + const sourceChanged = !sameLockSource(entry.source, nextSource); + const riskChanged = !sameLockRisk(entry.risk, nextRisk); + if (sourceChanged || riskChanged) { + nextEntries.set(entry.id, { + ...entry, + source: nextSource, + risk: nextRisk, + updatedAt: (options.now ?? new Date()).toISOString(), + }); + writeCapletsLockfile(options.lockfilePath, { + version: 1, + entries: [...nextEntries.values()], + }); + } + results.push({ + id: entry.id, + source: lockSourceDisplay(entry.source), + destination, + kind: entry.kind, + hash: entry.installedHash, + status: "noop", + lockfile: options.lockfilePath, + }); + continue; + } + if (!options.force && riskIncrease(entry.risk, nextRisk)) { + throw new CapletsError( + "REQUEST_INVALID", + `Caplet ${entry.id} update changes its risk profile; pass --force to update it`, + ); + } + + const plan: InstallPlan = { + id: entry.id, + source: lockSourceDisplay(entry.source), + sourcePath: lockedSource.sourcePath, + sourceBoundary: dirname(lockedSource.sourcePath), + destination, + kind: entry.kind, + }; + preflightInstallCaplets( + [ + { + id: entry.id, + path: + entry.kind === "directory" + ? join(lockedSource.sourcePath, "CAPLET.md") + : lockedSource.sourcePath, + }, + ], + { + destinationRoot, + force: true, + repoRoot: lockedSource.repoRoot, + sourceId: lockSourceDisplay(entry.source), + }, + ); + installOneCaplet(plan, { force: true }); + const hash = hashInstalledArtifact(destination); + const now = (options.now ?? new Date()).toISOString(); + nextEntries.set(entry.id, { + ...entry, + source: nextSource, + installedHash: hash, + updatedAt: now, + risk: nextRisk, + }); + writeCapletsLockfile(options.lockfilePath, { + version: 1, + entries: [...nextEntries.values()], + }); + results.push({ + id: entry.id, + source: lockSourceDisplay(entry.source), + destination, + kind: entry.kind, + hash, + status: "updated", + lockfile: options.lockfilePath, + }); + } finally { + lockedSource.cleanup(); + } + } + return { installed: results }; +} + +function refreshedLockSource( + source: CapletsLockSource, + lockedSource: LockedSourceResolution, +): CapletsLockSource { + if (source.type === "git") { + return { + ...source, + ...(lockedSource.resolvedRevision ? { resolvedRevision: lockedSource.resolvedRevision } : {}), + }; + } + return { + ...source, + ...localGitInfo(lockedSource.repoRoot), + }; +} + +function sameLockSource(left: CapletsLockSource, right: CapletsLockSource): boolean { + return JSON.stringify(left) === JSON.stringify(right); +} + +function sameLockRisk(left: CapletsLockEntry["risk"], right: CapletsLockEntry["risk"]): boolean { + return JSON.stringify(left) === JSON.stringify(right); +} + function discoverSelectedCapletFiles( sourceRoot: string, selectedIds: Set, @@ -108,21 +431,33 @@ function discoverSelectedCapletFiles( return candidates.sort((left, right) => left.id.localeCompare(right.id)); } -function resolveInstallSource(repo: string): { id: string; repoRoot: string; cleanup: () => void } { +function resolveInstallSource(repo: string): { + id: string; + repoRoot: string; + cleanup: () => void; + sourceKind: "local" | "git"; + repository?: string | undefined; + resolvedRevision?: string | undefined; +} { if (existsSync(repo) && statSync(repo).isDirectory()) { - return { id: repo, repoRoot: repo, cleanup: () => {} }; + return { id: repo, repoRoot: repo, cleanup: () => {}, sourceKind: "local" }; } const normalizedRepo = normalizeGitRepo(repo); const repoRoot = mkdtempSync(join(tmpdir(), "caplets-install-")); try { execFileSync("git", ["clone", "--depth", "1", "--", normalizedRepo, repoRoot], { + env: externalGitEnv(), stdio: "ignore", timeout: 60_000, }); + const resolvedRevision = gitRevision(repoRoot); return { id: normalizedRepo, repoRoot, + sourceKind: "git", + repository: normalizedRepo, + resolvedRevision, cleanup: () => removeInstallPath(repoRoot, `temporary install source ${repoRoot}`, true), }; } catch (error) { @@ -131,6 +466,464 @@ function resolveInstallSource(repo: string): { id: string; repoRoot: string; cle } } +function updateLockfileAfterInstall( + lockfilePath: string, + plans: InstallPlan[], + installed: InstallableCaplet[], + options: { + source: ReturnType; + now: Date; + }, +): void { + const existing = existsSync(lockfilePath) + ? readCapletsLockfile(lockfilePath) + : { version: 1 as const, entries: [] }; + const installedById = new Map(installed.map((caplet) => [caplet.id, caplet])); + const next = new Map(existing.entries.map((entry) => [entry.id, entry])); + for (const plan of plans) { + const caplet = installedById.get(plan.id); + if (!caplet?.hash) continue; + const now = options.now.toISOString(); + const previous = next.get(plan.id); + next.set(plan.id, { + id: plan.id, + destination: destinationDisplay(plan, caplet.destination), + kind: plan.kind, + source: lockSourceForPlan(plan, options.source), + installedHash: caplet.hash, + installedAt: previous?.installedAt ?? now, + updatedAt: now, + risk: riskSummaryForSourcePath(plan.sourcePath), + }); + } + writeCapletsLockfile(lockfilePath, { version: 1, entries: [...next.values()] }); +} + +function destinationDisplay(plan: InstallPlan, destination: string): string { + return plan.kind === "file" ? `${plan.id}.md` : basename(destination); +} + +function lockSourceForPlan( + plan: InstallPlan, + source: ReturnType, +): CapletsLockSource { + const sourcePath = relative(source.repoRoot, plan.sourcePath).replace(/\\/g, "/"); + if (source.sourceKind === "git") { + return { + type: "git", + repository: source.repository ?? source.id, + path: sourcePath, + trackedRef: "HEAD", + resolvedRevision: source.resolvedRevision, + portability: "portable", + }; + } + return { + type: "local", + path: plan.sourcePath, + portability: "non_portable", + ...localGitInfo(source.repoRoot), + }; +} + +function resolveLockedSource( + source: CapletsLockSource, + options: { useResolvedRevision?: boolean } = {}, +): LockedSourceResolution { + const useResolvedRevision = options.useResolvedRevision ?? true; + if (source.type === "local") { + if (!existsSync(source.path)) { + throw new CapletsError( + "CONFIG_NOT_FOUND", + `Locked local source ${source.path} is unavailable`, + ); + } + const repoRoot = inferLocalRepoRoot(source.path); + return { + sourcePath: source.path, + repoRoot, + resolvedRevision: gitRevision(repoRoot), + cleanup: () => {}, + }; + } + const repoRoot = mkdtempSync(join(tmpdir(), "caplets-restore-")); + try { + execFileSync("git", ["clone", "--", source.repository, repoRoot], { + env: externalGitEnv(), + stdio: "ignore", + timeout: 60_000, + }); + if (useResolvedRevision && source.resolvedRevision) { + execFileSync("git", ["checkout", "--detach", source.resolvedRevision], { + cwd: repoRoot, + env: externalGitEnv(), + stdio: "ignore", + timeout: 60_000, + }); + } else if (!useResolvedRevision && source.trackedRef && source.trackedRef !== "HEAD") { + checkoutTrackedRef(repoRoot, source.trackedRef); + } + return { + sourcePath: join(repoRoot, source.path), + repoRoot, + resolvedRevision: gitRevision(repoRoot), + cleanup: () => removeInstallPath(repoRoot, `temporary restore source ${repoRoot}`, true), + }; + } catch (error) { + removeInstallPath(repoRoot, `temporary restore source ${repoRoot}`, true); + throw new CapletsError( + "CONFIG_NOT_FOUND", + `Could not restore locked source ${source.repository}`, + toSafeError(error), + ); + } +} + +function checkoutTrackedRef(repoRoot: string, trackedRef: string): void { + try { + execFileSync("git", ["checkout", "--detach", trackedRef], { + cwd: repoRoot, + env: externalGitEnv(), + stdio: "ignore", + timeout: 60_000, + }); + } catch { + execFileSync("git", ["checkout", "--detach", `origin/${trackedRef}`], { + cwd: repoRoot, + env: externalGitEnv(), + stdio: "ignore", + timeout: 60_000, + }); + } +} + +function riskSummaryForSourcePath(sourcePath: string): CapletsLockEntry["risk"] { + const frontmatter = readCapletFrontmatter(sourcePath); + const backendFamilies = capletBackendFamilies(frontmatter); + const auth = capletAuth(frontmatter); + const runtime = isRecord(frontmatter.runtime) ? frontmatter.runtime : undefined; + const projectBindingRequired = + isRecord(frontmatter.projectBinding) && frontmatter.projectBinding.required === true; + const runtimeFeatures = Array.isArray(runtime?.features) + ? runtime.features.filter((feature): feature is string => typeof feature === "string") + : undefined; + const mutating = capletCanMutate(frontmatter); + const destructive = capletCanDestroy(frontmatter); + return { + backendFamilies: backendFamilies.length > 0 ? backendFamilies : ["unknown"], + safety: derivedSafety({ + backendFamilies, + auth, + projectBindingRequired, + runtimeFeatures: runtimeFeatures ?? [], + mutating, + destructive, + frontmatter, + }), + projectBindingRequired, + authScopes: Array.isArray(auth?.scopes) + ? auth.scopes.filter((scope): scope is string => typeof scope === "string") + : undefined, + runtimeFeatures, + mutating, + destructive, + bodyHash: hashInstalledArtifact(sourcePath), + }; +} + +function riskIncrease(current: CapletsLockEntry["risk"], next: CapletsLockEntry["risk"]): boolean { + if (current.safety === "unknown" || next.safety === "unknown") return true; + if (riskRank(next.safety) > riskRank(current.safety)) return true; + if (!current.projectBindingRequired && next.projectBindingRequired) return true; + if (!current.mutating && next.mutating) return true; + if (!current.destructive && next.destructive) return true; + if (!isSubset(current.authScopes ?? [], next.authScopes ?? [])) return true; + if (!isSubset(current.runtimeFeatures ?? [], next.runtimeFeatures ?? [])) return true; + return false; +} + +function readCapletFrontmatter(sourcePath: string): Record { + const capletFile = lstatSync(sourcePath).isDirectory() + ? join(sourcePath, "CAPLET.md") + : sourcePath; + const text = readFileSync(capletFile, "utf8"); + const match = /^---\r?\n([\s\S]*?)\r?\n---/u.exec(text); + if (!match) return {}; + const yaml = match[1]; + if (yaml === undefined) return {}; + const parsed = parseYaml(yaml); + return isRecord(parsed) ? parsed : {}; +} + +function capletBackendFamilies(frontmatter: Record): string[] { + const families: Array = [ + ["mcp", "mcpServer"], + ["openapi", "openapiEndpoint"], + ["googleDiscovery", "googleDiscoveryApi"], + ["graphql", "graphqlEndpoint"], + ["http", "httpApi"], + ["cli", "cliTools"], + ["caplets", "capletSet"], + ]; + return families.flatMap(([family, key]) => (frontmatter[key] === undefined ? [] : [family])); +} + +function capletAuth(frontmatter: Record): Record | undefined { + for (const key of [ + "mcpServer", + "openapiEndpoint", + "googleDiscoveryApi", + "graphqlEndpoint", + "httpApi", + ]) { + const backend = frontmatter[key]; + if (isRecord(backend) && isRecord(backend.auth)) return backend.auth; + } + return undefined; +} + +function derivedSafety(input: { + backendFamilies: string[]; + auth: Record | undefined; + projectBindingRequired: boolean; + runtimeFeatures: string[]; + mutating: boolean; + destructive: boolean; + frontmatter: Record; +}): CapletsLockEntry["risk"]["safety"] { + if ( + input.projectBindingRequired || + input.runtimeFeatures.length > 0 || + input.backendFamilies.includes("cli") || + isLocalMcpServer(input.frontmatter) + ) { + return "local_control"; + } + if ( + input.destructive || + input.mutating || + input.auth !== undefined || + input.backendFamilies.some((family) => + ["openapi", "googleDiscovery", "graphql", "http"].includes(family), + ) + ) { + return "mutating_saas"; + } + return "standard"; +} + +function isLocalMcpServer(frontmatter: Record): boolean { + const mcpServer = frontmatter.mcpServer; + return isRecord(mcpServer) && typeof mcpServer.command === "string"; +} + +function capletCanMutate(frontmatter: Record): boolean { + if (frontmatter.graphqlEndpoint !== undefined) return true; + if (frontmatter.openapiEndpoint !== undefined || frontmatter.googleDiscoveryApi !== undefined) { + return true; + } + const httpApi = frontmatter.httpApi; + if (isRecord(httpApi) && isRecord(httpApi.actions)) { + return Object.values(httpApi.actions).some((action) => { + if (!isRecord(action)) return false; + return typeof action.method === "string" && action.method.toUpperCase() !== "GET"; + }); + } + if (isRecord(frontmatter.cliTools) && isRecord(frontmatter.cliTools.actions)) { + return Object.values(frontmatter.cliTools.actions).some((action) => { + if (!isRecord(action) || !isRecord(action.annotations)) return true; + return action.annotations.readOnlyHint !== true; + }); + } + return false; +} + +function capletCanDestroy(frontmatter: Record): boolean { + const httpApi = frontmatter.httpApi; + if (isRecord(httpApi) && isRecord(httpApi.actions)) { + return Object.values(httpApi.actions).some( + (action) => + isRecord(action) && + typeof action.method === "string" && + action.method.toUpperCase() === "DELETE", + ); + } + if (isRecord(frontmatter.cliTools) && isRecord(frontmatter.cliTools.actions)) { + return Object.values(frontmatter.cliTools.actions).some( + (action) => + isRecord(action) && + isRecord(action.annotations) && + action.annotations.destructiveHint === true, + ); + } + return false; +} + +function riskRank(value: CapletsLockEntry["risk"]["safety"]): number { + switch (value) { + case "standard": + return 0; + case "mutating_saas": + return 1; + case "local_control": + return 2; + case "unknown": + return 3; + } +} + +function isSubset(previous: string[], next: string[]): boolean { + const previousValues = new Set(previous); + return next.every((value) => previousValues.has(value)); +} + +function isRecord(value: unknown): value is Record { + return Boolean(value) && typeof value === "object" && !Array.isArray(value); +} + +function lockSourceDisplay(source: CapletsLockSource): string { + return source.type === "git" ? `${source.repository}#${source.path}` : `${source.path}`; +} + +function inferLocalRepoRoot(sourcePath: string): string { + const marker = `${sep}caplets${sep}`; + const index = sourcePath.lastIndexOf(marker); + return index === -1 ? dirname(sourcePath) : sourcePath.slice(0, index); +} + +function hashInstalledArtifact(path: string): string { + const hash = createHash("sha256"); + hashPath(path, "", hash); + return `sha256:${hash.digest("hex")}`; +} + +function hashDirectoryCapletInstallSource(path: string, sourceBoundary: string): string { + const hash = createHash("sha256"); + hashDirectoryCapletInstallPath(path, "", hash, sourceBoundary); + return `sha256:${hash.digest("hex")}`; +} + +function hashPath(path: string, relativePath: string, hash: ReturnType): void { + const stats = lstatSync(path); + const mode = stats.mode & 0o111 ? "executable" : "plain"; + if (stats.isDirectory()) { + hash.update(`dir\0${relativePath}\0`); + for (const entry of readdirSync(path).sort()) { + hashPath(join(path, entry), relativePath ? `${relativePath}/${entry}` : entry, hash); + } + return; + } + if (stats.isSymbolicLink()) { + hash.update(`symlink\0${relativePath}\0${readlinkSync(path)}\0`); + return; + } + hash.update(`file\0${relativePath}\0${mode}\0`); + hash.update(readFileSync(path)); + hash.update("\0"); +} + +function hashDirectoryCapletInstallPath( + path: string, + relativePath: string, + hash: ReturnType, + sourceBoundary: string, + seenDirectories = new Set(), +): void { + const lstat = lstatSync(path); + const resolvedPath = lstat.isSymbolicLink() + ? resolveDirectoryCapletSymlink(path, sourceBoundary) + : path; + const stats = statSync(resolvedPath); + if (stats.isDirectory()) { + const realDirectory = realpathSync(resolvedPath); + if (seenDirectories.has(realDirectory)) { + throw new CapletsError( + "CONFIG_INVALID", + `Directory Caplet symlink ${path} creates a copy cycle`, + ); + } + hash.update(`dir\0${relativePath}\0`); + const childSeenDirectories = new Set(seenDirectories); + childSeenDirectories.add(realDirectory); + for (const entry of readdirSync(resolvedPath).sort()) { + hashDirectoryCapletInstallPath( + join(resolvedPath, entry), + relativePath ? `${relativePath}/${entry}` : entry, + hash, + sourceBoundary, + childSeenDirectories, + ); + } + return; + } + const mode = stats.mode & 0o111 ? "executable" : "plain"; + hash.update(`file\0${relativePath}\0${mode}\0`); + hash.update(readFileSync(resolvedPath)); + hash.update("\0"); +} + +function gitRevision(repoRoot: string): string | undefined { + try { + return execFileSync("git", ["rev-parse", "HEAD"], { + cwd: repoRoot, + encoding: "utf8", + env: externalGitEnv(), + stdio: ["ignore", "pipe", "ignore"], + timeout: 10_000, + }).trim(); + } catch { + return undefined; + } +} + +function localGitInfo(repoRoot: string): Partial> { + const gitRevisionValue = gitRevision(repoRoot); + const dirty = gitDirty(repoRoot); + try { + const gitRepository = execFileSync("git", ["config", "--get", "remote.origin.url"], { + cwd: repoRoot, + encoding: "utf8", + env: externalGitEnv(), + stdio: ["ignore", "pipe", "ignore"], + timeout: 10_000, + }).trim(); + return { + ...(gitRepository ? { gitRepository } : {}), + ...(gitRevisionValue ? { gitRevision: gitRevisionValue } : {}), + ...(dirty === undefined ? {} : { dirty }), + }; + } catch { + return { + ...(gitRevisionValue ? { gitRevision: gitRevisionValue } : {}), + ...(dirty === undefined ? {} : { dirty }), + }; + } +} + +function gitDirty(repoRoot: string): boolean | undefined { + if (!gitRevision(repoRoot)) return undefined; + try { + const status = execFileSync("git", ["status", "--porcelain"], { + cwd: repoRoot, + encoding: "utf8", + env: externalGitEnv(), + stdio: ["ignore", "pipe", "ignore"], + timeout: 10_000, + }); + return status.trim().length > 0; + } catch { + return undefined; + } +} + +function externalGitEnv(): NodeJS.ProcessEnv { + const env = { ...process.env }; + delete env.GIT_DIR; + delete env.GIT_COMMON_DIR; + delete env.GIT_WORK_TREE; + return env; +} + export function normalizeGitRepo(repo: string): string { if (/^[A-Za-z0-9_.-]+\/[A-Za-z0-9_.-]+$/.test(repo)) { const normalized = repo.endsWith(".git") ? repo.slice(0, -4) : repo; @@ -329,10 +1122,9 @@ function installOneCaplet(plan: InstallPlan, options: { force: boolean }): Insta `Caplet ${plan.id} already exists at ${plan.destination}; pass --force to overwrite it`, ); } - removeInstallPath(plan.destination, `existing Caplet destination ${plan.destination}`, false); } - copyInstallPath(plan); + replaceInstallPath(plan, Boolean(stats)); return { id: plan.id, source: plan.source, @@ -341,6 +1133,55 @@ function installOneCaplet(plan: InstallPlan, options: { force: boolean }): Insta }; } +function replaceInstallPath(plan: InstallPlan, hasExistingDestination: boolean): void { + const stagedPath = uniqueSiblingPath(plan.destination, ".tmp"); + const backupPath = uniqueSiblingPath(plan.destination, ".old"); + try { + copyInstallPath(plan, stagedPath); + if (!hasExistingDestination) { + renameSync(stagedPath, plan.destination); + return; + } + + renameSync(plan.destination, backupPath); + try { + renameSync(stagedPath, plan.destination); + } catch (error) { + try { + renameSync(backupPath, plan.destination); + } catch (restoreError) { + throw new CapletsError( + "CONFIG_INVALID", + `Could not restore existing Caplet destination ${plan.destination}`, + toSafeError(restoreError), + ); + } + throw error; + } + removeInstallPath(backupPath, `previous Caplet destination ${backupPath}`, true); + } catch (error) { + removeInstallPath(stagedPath, `staged Caplet destination ${stagedPath}`, true); + if (error instanceof CapletsError) { + throw error; + } + throw new CapletsError( + "CONFIG_INVALID", + `Could not install Caplet ${plan.id} to ${plan.destination}`, + toSafeError(error), + ); + } +} + +function uniqueSiblingPath(path: string, suffix: string): string { + const parent = dirname(path); + const name = basename(path); + for (let attempt = 0; attempt < 100; attempt += 1) { + const candidate = join(parent, `.${name}${suffix}-${process.pid}-${Date.now()}-${attempt}`); + if (!existsSync(candidate)) return candidate; + } + throw new CapletsError("CONFIG_EXISTS", `Could not allocate staging path for ${path}`); +} + function rejectSymlinkDestination(id: string, path: string, stats: Stats | undefined): void { if (stats?.isSymbolicLink()) { throw new CapletsError( @@ -400,14 +1241,14 @@ function removeInstallPath(path: string, label: string, force: boolean): void { } } -function copyInstallPath(plan: InstallPlan): void { +function copyInstallPath(plan: InstallPlan, destination: string): void { try { if (plan.kind === "directory") { - copyDirectoryCaplet(plan.sourcePath, plan.destination, realpathSync(plan.sourceBoundary)); + copyDirectoryCaplet(plan.sourcePath, destination, realpathSync(plan.sourceBoundary)); return; } - cpSync(plan.sourcePath, plan.destination, { + cpSync(plan.sourcePath, destination, { recursive: false, force: false, errorOnExist: true, diff --git a/packages/core/src/cli/lockfile.ts b/packages/core/src/cli/lockfile.ts new file mode 100644 index 00000000..40f6db4a --- /dev/null +++ b/packages/core/src/cli/lockfile.ts @@ -0,0 +1,340 @@ +import { + existsSync, + lstatSync, + mkdirSync, + readFileSync, + realpathSync, + renameSync, + rmSync, + writeFileSync, +} from "node:fs"; +import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; +import { CapletsError, toSafeError } from "../errors"; + +export const CAPLETS_LOCKFILE_VERSION = 1; + +export type CapletsLockSource = + | { + type: "git"; + repository: string; + path: string; + trackedRef?: string | undefined; + resolvedRevision?: string | undefined; + portability: "portable" | "non_portable"; + } + | { + type: "local"; + path: string; + gitRepository?: string | undefined; + gitRevision?: string | undefined; + dirty?: boolean | undefined; + portability: "portable" | "non_portable"; + }; + +export type CapletsLockRiskSummary = { + backendFamilies: string[]; + safety: "standard" | "mutating_saas" | "local_control" | "unknown"; + projectBindingRequired: boolean; + authScopes?: string[] | undefined; + runtimeFeatures?: string[] | undefined; + mutating: boolean; + destructive: boolean; + bodyHash?: string | undefined; + referenceHash?: string | undefined; +}; + +export type CapletsLockEntry = { + id: string; + destination: string; + kind: "file" | "directory"; + source: CapletsLockSource; + installedHash: string; + installedAt: string; + updatedAt: string; + risk: CapletsLockRiskSummary; +}; + +export type CapletsLockfile = { + version: typeof CAPLETS_LOCKFILE_VERSION; + entries: CapletsLockEntry[]; +}; + +export function readCapletsLockfile(path: string): CapletsLockfile { + let contents: string; + try { + contents = readFileSync(path, "utf8"); + } catch (error) { + if (isErrorCode(error, "ENOENT")) { + throw new CapletsError("CONFIG_NOT_FOUND", `Caplets lockfile not found at ${path}`, { + cause: toSafeError(error), + }); + } + throw new CapletsError("CONFIG_INVALID", `Could not read Caplets lockfile at ${path}`, { + cause: toSafeError(error), + }); + } + + let parsed: unknown; + try { + parsed = JSON.parse(contents); + } catch (error) { + throw new CapletsError("CONFIG_INVALID", `Caplets lockfile at ${path} is not valid JSON`, { + cause: toSafeError(error), + }); + } + return parseCapletsLockfile(parsed, path); +} + +function isErrorCode(error: unknown, code: string): boolean { + return ( + typeof error === "object" && + error !== null && + "code" in error && + (error as { code?: unknown }).code === code + ); +} + +export function writeCapletsLockfile(path: string, lockfile: CapletsLockfile): void { + const validated = parseCapletsLockfile(lockfile, path); + const stable: CapletsLockfile = { + version: CAPLETS_LOCKFILE_VERSION, + entries: [...validated.entries].sort((left, right) => left.id.localeCompare(right.id)), + }; + const parent = dirname(path); + const basename = path.split(/[\\/]/).at(-1) ?? "caplets.lock.json"; + const temporary = join(parent, `.${basename}.tmp-${process.pid}-${Date.now()}`); + try { + mkdirSync(parent, { recursive: true, mode: 0o700 }); + writeFileSync(temporary, `${JSON.stringify(stable, null, 2)}\n`, { mode: 0o600 }); + renameSync(temporary, path); + } catch (error) { + try { + rmSync(temporary, { force: true }); + } catch { + // Best-effort cleanup; the final lockfile is protected by rename. + } + throw new CapletsError("CONFIG_INVALID", `Could not write Caplets lockfile at ${path}`, { + cause: toSafeError(error), + }); + } +} + +export function parseCapletsLockfile(value: unknown, path = "Caplets lockfile"): CapletsLockfile { + if (!isRecord(value)) { + throw new CapletsError("CONFIG_INVALID", `${path} must be a JSON object`); + } + if (value.version !== CAPLETS_LOCKFILE_VERSION) { + throw new CapletsError( + "CONFIG_INVALID", + `${path} uses unsupported Caplets lockfile version ${String(value.version)}`, + ); + } + if (!Array.isArray(value.entries)) { + throw new CapletsError("CONFIG_INVALID", `${path} entries must be an array`); + } + + const seen = new Set(); + const entries = value.entries.map((entry, index) => + parseLockEntry(entry, `${path} entry ${index}`), + ); + for (const entry of entries) { + if (seen.has(entry.id)) { + throw new CapletsError("CONFIG_EXISTS", `${path} contains duplicate Caplet ID ${entry.id}`); + } + seen.add(entry.id); + } + + return { version: CAPLETS_LOCKFILE_VERSION, entries }; +} + +export function validateLockfileDestination(capletsRoot: string, destination: string): string { + if (isAbsolute(destination)) { + throw new CapletsError( + "CONFIG_INVALID", + `Lockfile destination ${destination} must be relative to the selected Caplets root`, + ); + } + const normalizedRoot = resolve(capletsRoot); + const resolved = resolve(normalizedRoot, destination); + const relativeDestination = relative(normalizedRoot, resolved); + if ( + relativeDestination === "" || + relativeDestination.startsWith("..") || + isAbsolute(relativeDestination) + ) { + throw new CapletsError( + "CONFIG_INVALID", + `Lockfile destination ${destination} escapes the selected Caplets root`, + ); + } + rejectSymlinkedExistingPath(normalizedRoot, resolved); + return resolved; +} + +function parseLockEntry(value: unknown, label: string): CapletsLockEntry { + if (!isRecord(value)) throw new CapletsError("CONFIG_INVALID", `${label} must be an object`); + const id = requireString(value.id, `${label}.id`); + const destination = requireString(value.destination, `${label}.destination`); + const kind = parseEnum(value.kind, ["file", "directory"], `${label}.kind`); + const source = parseLockSource(value.source, `${label}.source`); + const installedHash = requireString(value.installedHash, `${label}.installedHash`); + const installedAt = requireString(value.installedAt, `${label}.installedAt`); + const updatedAt = requireString(value.updatedAt, `${label}.updatedAt`); + const risk = parseRiskSummary(value.risk, `${label}.risk`); + if (isAbsolute(destination) || destination.split(/[\\/]/).includes("..")) { + throw new CapletsError("CONFIG_INVALID", `${label}.destination must be a safe relative path`); + } + return { id, destination, kind, source, installedHash, installedAt, updatedAt, risk }; +} + +function parseLockSource(value: unknown, label: string): CapletsLockSource { + if (!isRecord(value)) throw new CapletsError("CONFIG_INVALID", `${label} must be an object`); + const type = parseEnum(value.type, ["git", "local"], `${label}.type`); + if (type === "git") { + const repository = requireCredentialFreeSource( + requireString(value.repository, `${label}.repository`), + label, + ); + const path = requireSafeRelativePath(requireString(value.path, `${label}.path`), label); + return { + type, + repository, + path, + trackedRef: optionalString(value.trackedRef, `${label}.trackedRef`), + resolvedRevision: optionalString(value.resolvedRevision, `${label}.resolvedRevision`), + portability: parseEnum( + value.portability, + ["portable", "non_portable"], + `${label}.portability`, + ), + }; + } + return { + type, + path: requireString(value.path, `${label}.path`), + gitRepository: + value.gitRepository === undefined + ? undefined + : requireCredentialFreeSource( + requireString(value.gitRepository, `${label}.gitRepository`), + label, + ), + gitRevision: optionalString(value.gitRevision, `${label}.gitRevision`), + dirty: value.dirty === undefined ? undefined : requireBoolean(value.dirty, `${label}.dirty`), + portability: parseEnum(value.portability, ["portable", "non_portable"], `${label}.portability`), + }; +} + +function parseRiskSummary(value: unknown, label: string): CapletsLockRiskSummary { + if (!isRecord(value)) throw new CapletsError("CONFIG_INVALID", `${label} must be an object`); + return { + backendFamilies: requireStringArray(value.backendFamilies, `${label}.backendFamilies`), + safety: parseEnum( + value.safety, + ["standard", "mutating_saas", "local_control", "unknown"], + `${label}.safety`, + ), + projectBindingRequired: requireBoolean( + value.projectBindingRequired, + `${label}.projectBindingRequired`, + ), + authScopes: + value.authScopes === undefined + ? undefined + : requireStringArray(value.authScopes, `${label}.authScopes`), + runtimeFeatures: + value.runtimeFeatures === undefined + ? undefined + : requireStringArray(value.runtimeFeatures, `${label}.runtimeFeatures`), + mutating: requireBoolean(value.mutating, `${label}.mutating`), + destructive: requireBoolean(value.destructive, `${label}.destructive`), + bodyHash: optionalString(value.bodyHash, `${label}.bodyHash`), + referenceHash: optionalString(value.referenceHash, `${label}.referenceHash`), + }; +} + +function requireCredentialFreeSource(source: string, label: string): string { + try { + const url = new URL(source); + if (url.username || url.password) { + throw new CapletsError("CONFIG_INVALID", `${label} must not contain credentials`); + } + } catch (error) { + if (error instanceof CapletsError) throw error; + // Local scp-like Git syntax is allowed, but credential-shaped URLs are not. + if (/^[^@\s]+:[^@\s]+@/.test(source) || /^[a-z][a-z0-9+.-]*:\/\/[^/\s]+@/i.test(source)) { + throw new CapletsError("CONFIG_INVALID", `${label} must not contain credentials`); + } + } + return source; +} + +function requireSafeRelativePath(path: string, label: string): string { + if (isAbsolute(path) || path.split(/[\\/]/).includes("..")) { + throw new CapletsError("CONFIG_INVALID", `${label}.path must be a safe relative path`); + } + return path; +} + +function rejectSymlinkedExistingPath(root: string, destination: string): void { + let current = root; + const relativePath = relative(root, destination); + for (const segment of relativePath.split(sep).filter(Boolean)) { + current = join(current, segment); + if (!existsSync(current)) return; + const stats = lstatSync(current); + if (stats.isSymbolicLink()) { + throw new CapletsError( + "CONFIG_EXISTS", + `Lockfile destination ${destination} resolves through symlink ${current}`, + ); + } + const real = realpathSync(current); + if (real !== root && !real.startsWith(`${root}${sep}`)) { + throw new CapletsError( + "CONFIG_INVALID", + `Lockfile destination ${destination} resolves outside the selected Caplets root`, + ); + } + } +} + +function requireString(value: unknown, label: string): string { + if (typeof value !== "string" || value.length === 0) { + throw new CapletsError("CONFIG_INVALID", `${label} must be a non-empty string`); + } + return value; +} + +function optionalString(value: unknown, label: string): string | undefined { + return value === undefined ? undefined : requireString(value, label); +} + +function requireBoolean(value: unknown, label: string): boolean { + if (typeof value !== "boolean") { + throw new CapletsError("CONFIG_INVALID", `${label} must be a boolean`); + } + return value; +} + +function requireStringArray(value: unknown, label: string): string[] { + if (!Array.isArray(value) || !value.every((item) => typeof item === "string")) { + throw new CapletsError("CONFIG_INVALID", `${label} must be an array of strings`); + } + return value; +} + +function parseEnum( + value: unknown, + allowed: T, + label: string, +): T[number] { + if (typeof value !== "string" || !allowed.includes(value)) { + throw new CapletsError("CONFIG_INVALID", `${label} must be one of ${allowed.join(", ")}`); + } + return value; +} + +function isRecord(value: unknown): value is Record { + return Boolean(value) && typeof value === "object" && !Array.isArray(value); +} diff --git a/packages/core/src/config.ts b/packages/core/src/config.ts index 85814518..1d05ddca 100644 --- a/packages/core/src/config.ts +++ b/packages/core/src/config.ts @@ -20,6 +20,7 @@ import { FileVaultStore, validateVaultKeyName, type VaultConfigOrigin } from "./ export { DEFAULT_AUTH_DIR, + DEFAULT_CAPLETS_LOCKFILE_PATH, DEFAULT_COMPLETION_CACHE_DIR, DEFAULT_CONFIG_PATH, DEFAULT_UPDATE_CHECK_CACHE_DIR, @@ -27,14 +28,17 @@ export { DEFAULT_TELEMETRY_STATE_DIR, PROJECT_CONFIG_FILE, defaultCacheBaseDir, + defaultCapletsLockfilePath, defaultCompletionCacheDir, defaultTelemetryStateDir, defaultUpdateCheckCacheDir, defaultUpdateCheckStateDir, + resolveCapletsLockfilePath, resolveCapletsRoot, resolveConfigPath, resolveProjectCapletsRoot, resolveProjectConfigPath, + resolveProjectLockfilePath, resolveTelemetryStateDir, resolveUpdateCheckCacheDir, resolveUpdateCheckStateDir, @@ -551,7 +555,6 @@ const runtimeRequirementsSchema = z }) .strict() .describe("Runtime feature and resource requirements for hosted execution."); - const agentSelectionHintSchema = z .string() .trim() diff --git a/packages/core/src/config/paths.ts b/packages/core/src/config/paths.ts index 6a399b95..85a59969 100644 --- a/packages/core/src/config/paths.ts +++ b/packages/core/src/config/paths.ts @@ -74,6 +74,15 @@ export function defaultAuthDir( return pathJoin(defaultStateBaseDir(env, home, platform), "caplets", "auth"); } +export function defaultCapletsLockfilePath( + env: PathEnv = process.env, + home = homedir(), + platform: Platform = process.platform, +): string { + const pathJoin = platform === "win32" ? win32.join : posix.join; + return pathJoin(defaultStateBaseDir(env, home, platform), "caplets", "caplets.lock.json"); +} + export function defaultTelemetryStateDir( env: PathEnv = process.env, home = homedir(), @@ -163,6 +172,7 @@ export function defaultObservedOutputShapeCacheDir( export const DEFAULT_CONFIG_PATH = defaultConfigPath(); export const DEFAULT_AUTH_DIR = defaultAuthDir(); +export const DEFAULT_CAPLETS_LOCKFILE_PATH = defaultCapletsLockfilePath(); export const DEFAULT_ARTIFACT_DIR = defaultArtifactDir(); export const DEFAULT_TELEMETRY_STATE_DIR = defaultTelemetryStateDir(); export const DEFAULT_TELEMETRY_IDENTITY_PATH = defaultTelemetryIdentityPath(); @@ -182,6 +192,14 @@ export function resolveProjectConfigPath(cwd = process.cwd()): string { return join(displayPath(cwd), PROJECT_CONFIG_FILE); } +export function resolveProjectLockfilePath(cwd = process.cwd()): string { + return join(displayPath(cwd), ".caplets.lock.json"); +} + +export function resolveCapletsLockfilePath(path?: string): string { + return path ?? DEFAULT_CAPLETS_LOCKFILE_PATH; +} + export function resolveCapletsRoot(configPath = resolveConfigPath()): string { return dirname(configPath); } diff --git a/packages/core/src/remote-control/dispatch.ts b/packages/core/src/remote-control/dispatch.ts index c84c0be4..aaa8a48e 100644 --- a/packages/core/src/remote-control/dispatch.ts +++ b/packages/core/src/remote-control/dispatch.ts @@ -16,11 +16,17 @@ import { } from "./../cli/auth"; import { completionShells, type CompletionShell } from "./../cli/completion"; import { initConfig } from "./../cli/init"; -import { installCaplets } from "./../cli/install"; +import { + installCaplets, + restoreCapletsFromLockfile, + updateCapletsFromLockfile, +} from "./../cli/install"; import { listCaplets } from "./../cli/inspection"; import { loadConfigWithSources, loadLocalOverlayConfigWithSources, + defaultCapletsLockfilePath, + resolveCapletsRoot, vaultBootstrapResolver, vaultResolverForAuthDir, vaultStoreForAuthDir, @@ -34,6 +40,8 @@ import type { RemoteCliRequest, RemoteCliResponse } from "./types"; export type RemoteControlDispatchContext = CapletsEngineOptions & { projectCapletsRoot: string; + globalCapletsRoot?: string | undefined; + globalLockfilePath?: string | undefined; authFlowStore?: RemoteAuthFlowStore; controlCallbackBaseUrl?: string; }; @@ -123,12 +131,39 @@ async function dispatch(request: RemoteCliRequest, context: RemoteControlDispatc return dispatchAdd(request.arguments, context); } + const globalCatalogTarget = () => ({ + destinationRoot: context.globalCapletsRoot ?? resolveCapletsRoot(context.configPath), + lockfilePath: context.globalLockfilePath ?? defaultCapletsLockfilePath(), + }); + if (request.command === "install") { + const repo = optionalString(request.arguments, "repo"); + if (!repo) { + return { + remote: true, + ...restoreCapletsFromLockfile({ + ...optionalProp("capletIds", optionalStringArray(request.arguments, "capletIds")), + ...globalCatalogTarget(), + ...optionalProp("force", optionalBoolean(request.arguments, "force")), + }), + }; + } + return { + remote: true, + ...installCaplets(repo, { + ...optionalProp("capletIds", optionalStringArray(request.arguments, "capletIds")), + ...globalCatalogTarget(), + ...optionalProp("force", optionalBoolean(request.arguments, "force")), + }), + }; + } + + if (request.command === "update") { return { remote: true, - ...installCaplets(requiredString(request.arguments, "repo"), { + ...updateCapletsFromLockfile({ ...optionalProp("capletIds", optionalStringArray(request.arguments, "capletIds")), - destinationRoot: context.projectCapletsRoot, + ...globalCatalogTarget(), ...optionalProp("force", optionalBoolean(request.arguments, "force")), }), }; diff --git a/packages/core/src/remote-control/types.ts b/packages/core/src/remote-control/types.ts index 48d446d0..12eac83b 100644 --- a/packages/core/src/remote-control/types.ts +++ b/packages/core/src/remote-control/types.ts @@ -19,6 +19,7 @@ export type RemoteCliCommand = | "init" | "add" | "install" + | "update" | "complete_cli" | "auth_login_start" | "auth_login_complete" diff --git a/packages/core/src/serve/http.ts b/packages/core/src/serve/http.ts index d6ec6ef3..0f657771 100644 --- a/packages/core/src/serve/http.ts +++ b/packages/core/src/serve/http.ts @@ -12,7 +12,11 @@ import { Hono, type MiddlewareHandler } from "hono"; import { logger } from "hono/logger"; import { WebSocketServer } from "ws"; import { fingerprintProjectRoot } from "../cloud/project-root"; -import { resolveProjectCapletsRoot } from "../config"; +import { + defaultCapletsLockfilePath, + resolveCapletsRoot, + resolveProjectCapletsRoot, +} from "../config"; import { CapletsEngine, type CapletsEngineOptions } from "../engine"; import { CapletsError, toSafeError } from "../errors"; import { @@ -1003,6 +1007,8 @@ function controlContext( return { ...io.control, projectCapletsRoot: io.control?.projectCapletsRoot ?? resolveProjectCapletsRoot(), + globalCapletsRoot: io.control?.globalCapletsRoot ?? resolveCapletsRoot(io.control?.configPath), + globalLockfilePath: io.control?.globalLockfilePath ?? defaultCapletsLockfilePath(), authFlowStore, controlCallbackBaseUrl: new URL( controlPath, @@ -1399,6 +1405,8 @@ export async function serveHttp( control: { ...resolvedEngineOptions, projectCapletsRoot: projectCapletsRootForEngineOptions(resolvedEngineOptions), + globalCapletsRoot: resolveCapletsRoot(resolvedEngineOptions.configPath), + globalLockfilePath: defaultCapletsLockfilePath(), }, }); const paths = servicePaths(options.path); @@ -1451,6 +1459,8 @@ export async function serveHttpWithSessionFactory( control: { ...resolvedEngineOptions, projectCapletsRoot: projectCapletsRootForEngineOptions(resolvedEngineOptions), + globalCapletsRoot: resolveCapletsRoot(resolvedEngineOptions.configPath), + globalLockfilePath: defaultCapletsLockfilePath(), }, }); const paths = servicePaths(options.path); diff --git a/packages/core/test/caplets-lockfile.test.ts b/packages/core/test/caplets-lockfile.test.ts new file mode 100644 index 00000000..3f57c9e9 --- /dev/null +++ b/packages/core/test/caplets-lockfile.test.ts @@ -0,0 +1,206 @@ +import { describe, expect, it } from "vitest"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + rmSync, + symlinkSync, + writeFileSync, +} from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { + readCapletsLockfile, + validateLockfileDestination, + writeCapletsLockfile, + type CapletsLockfile, +} from "../src/cli/lockfile"; +import type { CapletsError } from "../src/errors"; + +describe("caplets lockfile", () => { + it("writes and reads lockfiles atomically with stable entry ordering", () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-lockfile-")); + const lockPath = join(dir, "nested", "caplets.lock.json"); + try { + const lockfile: CapletsLockfile = { + version: 1, + entries: [ + lockEntry({ id: "github", destination: "github" }), + lockEntry({ id: "filesystem", destination: "filesystem.md", kind: "file" }), + ], + }; + + writeCapletsLockfile(lockPath, lockfile); + + expect(readFileSync(lockPath, "utf8")).toBe( + `${JSON.stringify( + { + version: 1, + entries: [ + lockEntry({ id: "filesystem", destination: "filesystem.md", kind: "file" }), + lockEntry({ id: "github", destination: "github" }), + ], + }, + null, + 2, + )}\n`, + ); + expect(readCapletsLockfile(lockPath)).toEqual({ + version: 1, + entries: [ + lockEntry({ id: "filesystem", destination: "filesystem.md", kind: "file" }), + lockEntry({ id: "github", destination: "github" }), + ], + }); + expect(existsSync(join(dir, "nested", "caplets.lock.json.tmp"))).toBe(false); + expect( + readdirSync(join(dir, "nested")).filter((entry) => + entry.startsWith(".caplets.lock.json.tmp-"), + ), + ).toEqual([]); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("rejects malformed, unsupported, duplicate, and credential-bearing lock entries", () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-lockfile-invalid-")); + const lockPath = join(dir, "caplets.lock.json"); + try { + writeFileSync(lockPath, "{ invalid"); + expect(() => readCapletsLockfile(lockPath)).toThrow( + expect.objectContaining({ code: "CONFIG_INVALID" }) as CapletsError, + ); + + writeFileSync(lockPath, `${JSON.stringify({ version: 2, entries: [] })}\n`); + expect(() => readCapletsLockfile(lockPath)).toThrow( + expect.objectContaining({ code: "CONFIG_INVALID" }) as CapletsError, + ); + + writeFileSync( + lockPath, + `${JSON.stringify({ + version: 1, + entries: [lockEntry({ id: "github" }), lockEntry({ id: "github" })], + })}\n`, + ); + expect(() => readCapletsLockfile(lockPath)).toThrow( + expect.objectContaining({ code: "CONFIG_EXISTS" }) as CapletsError, + ); + + writeFileSync( + lockPath, + `${JSON.stringify({ + version: 1, + entries: [ + lockEntry({ + source: { + type: "git", + repository: "https://token@example.com/private/repo.git", + path: "caplets/github", + trackedRef: "main", + resolvedRevision: "abc123", + portability: "portable", + }, + }), + ], + })}\n`, + ); + expect(() => readCapletsLockfile(lockPath)).toThrow( + expect.objectContaining({ code: "CONFIG_INVALID" }) as CapletsError, + ); + + writeFileSync( + lockPath, + `${JSON.stringify({ + version: 1, + entries: [ + lockEntry({ + source: { + type: "git", + repository: "https://github.com/spiritledsoftware/caplets.git", + path: "../outside", + trackedRef: "main", + resolvedRevision: "abc123", + portability: "portable", + }, + }), + ], + })}\n`, + ); + expect(() => readCapletsLockfile(lockPath)).toThrow( + expect.objectContaining({ code: "CONFIG_INVALID" }) as CapletsError, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("reports missing lockfiles as not found instead of invalid JSON", () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-lockfile-missing-")); + const lockPath = join(dir, "caplets.lock.json"); + try { + expect(() => readCapletsLockfile(lockPath)).toThrow( + expect.objectContaining({ + code: "CONFIG_NOT_FOUND", + message: `Caplets lockfile not found at ${lockPath}`, + }) as CapletsError, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("validates lockfile destinations against the selected Caplets root", () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-lockfile-destination-")); + const root = join(dir, "caplets"); + const outside = join(dir, "outside"); + try { + mkdirSync(root, { recursive: true }); + mkdirSync(outside, { recursive: true }); + symlinkSync(outside, join(root, "linked")); + + expect(validateLockfileDestination(root, "github")).toBe(join(root, "github")); + expect(() => validateLockfileDestination(root, "/tmp/github")).toThrow( + expect.objectContaining({ code: "CONFIG_INVALID" }) as CapletsError, + ); + expect(() => validateLockfileDestination(root, "../github")).toThrow( + expect.objectContaining({ code: "CONFIG_INVALID" }) as CapletsError, + ); + expect(() => validateLockfileDestination(root, "linked/github")).toThrow( + expect.objectContaining({ code: "CONFIG_EXISTS" }) as CapletsError, + ); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +function lockEntry(overrides: Partial = {}) { + return { + id: "github", + destination: "github", + kind: "directory" as const, + source: { + type: "git" as const, + repository: "https://github.com/spiritledsoftware/caplets.git", + path: "caplets/github", + trackedRef: "main", + resolvedRevision: "abc123", + portability: "portable" as const, + }, + installedHash: "sha256:abc", + installedAt: "2026-06-26T12:00:00.000Z", + updatedAt: "2026-06-26T12:00:00.000Z", + risk: { + backendFamilies: ["mcp"], + safety: "standard" as const, + projectBindingRequired: false, + mutating: false, + destructive: false, + }, + ...overrides, + }; +} diff --git a/packages/core/test/cli.test.ts b/packages/core/test/cli.test.ts index 9b9486cb..b5af67df 100644 --- a/packages/core/test/cli.test.ts +++ b/packages/core/test/cli.test.ts @@ -1,10 +1,12 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { execFileSync } from "node:child_process"; import { existsSync, lstatSync, mkdirSync, mkdtempSync, readFileSync, + readdirSync, rmSync, symlinkSync, writeFileSync, @@ -1327,6 +1329,26 @@ describe("cli init", () => { ); expect(out.join("")).toContain("Installed filesystem"); expect(out.join("")).toContain("Installed github"); + const lock = JSON.parse(readFileSync(join(projectRoot, ".caplets.lock.json"), "utf8")); + expect(lock).toMatchObject({ + version: 1, + entries: expect.arrayContaining([ + expect.objectContaining({ + id: "filesystem", + destination: "filesystem.md", + kind: "file", + source: expect.objectContaining({ type: "local", portability: "non_portable" }), + installedHash: expect.stringMatching(/^sha256:/), + }), + expect.objectContaining({ + id: "github", + destination: "github", + kind: "directory", + source: expect.objectContaining({ type: "local", portability: "non_portable" }), + installedHash: expect.stringMatching(/^sha256:/), + }), + ]), + }); } finally { process.chdir(cwd); rmSync(dir, { recursive: true, force: true }); @@ -1340,10 +1362,12 @@ describe("cli init", () => { const configPath = join(dir, "user", "config.json"); const out: string[] = []; const cwd = process.cwd(); + const originalXdgStateHome = process.env.XDG_STATE_HOME; try { writeInstallableRepo(repo); mkdirSync(projectRoot, { recursive: true }); process.env.CAPLETS_CONFIG = configPath; + process.env.XDG_STATE_HOME = join(dir, "state"); process.chdir(projectRoot); await runCli(["install", "--global", repo], { writeOut: (value) => out.push(value) }); @@ -1355,11 +1379,17 @@ describe("cli init", () => { "name: GitHub", ); expect(existsSync(join(projectRoot, ".caplets"))).toBe(false); + expect(existsSync(join(dir, "state", "caplets", "caplets.lock.json"))).toBe(true); expect(out.join("")).toContain( `Installed filesystem to ${join(dir, "user", "filesystem.md")}`, ); expect(out.join("")).toContain(`Installed github to ${join(dir, "user", "github")}`); } finally { + if (originalXdgStateHome === undefined) { + delete process.env.XDG_STATE_HOME; + } else { + process.env.XDG_STATE_HOME = originalXdgStateHome; + } process.chdir(cwd); rmSync(dir, { recursive: true, force: true }); } @@ -2034,6 +2064,961 @@ describe("cli init", () => { } }); + it("prints install results as JSON when requested", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-json-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + { + id: "github", + status: "installed", + destination: join(projectRoot, ".caplets", "github"), + lockfile: join(projectRoot, ".caplets.lock.json"), + hash: expect.stringMatching(/^sha256:/), + }, + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("records successful installs before a later install fails", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-partial-lockfile-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const capletsRoot = join(repo, "caplets"); + const out: string[] = []; + const cwd = process.cwd(); + try { + mkdirSync(join(capletsRoot, "alpha"), { recursive: true }); + writeFileSync(join(capletsRoot, "alpha", "CAPLET.md"), capletFixture("Alpha")); + mkdirSync(join(capletsRoot, "broken"), { recursive: true }); + writeFileSync(join(capletsRoot, "broken", "CAPLET.md"), capletFixture("Broken")); + writeFileSync(join(dir, "outside.txt"), "outside source boundary\n"); + symlinkSync(join(dir, "outside.txt"), join(capletsRoot, "broken", "outside-link.txt")); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await expect( + runCli(["install", repo, "alpha", "broken"], { writeOut: () => {} }), + ).rejects.toMatchObject({ + code: "CONFIG_INVALID", + } satisfies Partial); + + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + expect(lockfile.entries).toMatchObject([ + { + id: "alpha", + destination: "alpha", + installedHash: expect.stringMatching(/^sha256:/), + }, + ]); + expect(lockfile.entries.some((entry: { id: string }) => entry.id === "broken")).toBe(false); + + await runCli(["install", "--json"], { writeOut: (value) => out.push(value) }); + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + { + id: "alpha", + status: "noop", + lockfile: lockfilePath, + }, + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("reports a missing project lockfile when install has no repo argument", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-missing-lockfile-")); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await expect(runCli(["install"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "CONFIG_NOT_FOUND", + message: `Caplets lockfile not found at ${join(projectRoot, ".caplets.lock.json")}`, + } satisfies Partial); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("restores missing Caplets from the project lockfile when install has no repo argument", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-restore-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + rmSync(join(projectRoot, ".caplets", "github"), { recursive: true, force: true }); + + await runCli(["install", "--json"], { writeOut: (value) => out.push(value) }); + + expect(existsSync(join(projectRoot, ".caplets", "github", "CAPLET.md"))).toBe(true); + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + expect.objectContaining({ + id: "github", + status: "restored", + hash: expect.stringMatching(/^sha256:/), + }), + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("restores a selected Caplet ID from the project lockfile", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-restore-id-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + rmSync(join(projectRoot, ".caplets", "github"), { recursive: true, force: true }); + + await runCli(["install", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(existsSync(join(projectRoot, ".caplets", "github", "CAPLET.md"))).toBe(true); + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + expect.objectContaining({ + id: "github", + status: "restored", + }), + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("restores a selected Caplet ID when a same-named local directory exists", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-restore-id-directory-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + rmSync(join(projectRoot, ".caplets", "github"), { recursive: true, force: true }); + mkdirSync(join(projectRoot, "github"), { recursive: true }); + + await runCli(["install", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(existsSync(join(projectRoot, ".caplets", "github", "CAPLET.md"))).toBe(true); + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + expect.objectContaining({ + id: "github", + status: "restored", + }), + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("persists every changed lockfile entry after restoring multiple Caplets", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-restore-many-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo], { writeOut: () => {} }); + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + for (const entry of lockfile.entries) { + entry.installedHash = "sha256:stale"; + } + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + rmSync(join(projectRoot, ".caplets", "github"), { recursive: true, force: true }); + rmSync(join(projectRoot, ".caplets", "filesystem.md"), { force: true }); + + await runCli(["install"], { writeOut: () => {} }); + + const restored = JSON.parse(readFileSync(lockfilePath, "utf8")); + expect(restored.entries).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + id: "filesystem", + installedHash: expect.stringMatching(/^sha256:(?!stale$)/), + }), + expect.objectContaining({ + id: "github", + installedHash: expect.stringMatching(/^sha256:(?!stale$)/), + }), + ]), + ); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("persists restored lockfile hash changes before a later restore fails", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-restore-partial-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo], { writeOut: () => {} }); + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + lockfile.entries.sort((left: { id: string }, right: { id: string }) => + left.id === "filesystem" ? -1 : right.id === "filesystem" ? 1 : 0, + ); + const filesystem = lockfile.entries.find( + (entry: { id: string }) => entry.id === "filesystem", + ); + const github = lockfile.entries.find((entry: { id: string }) => entry.id === "github"); + filesystem.installedHash = "sha256:stale"; + github.source.path = join(repo, "caplets", "missing-github"); + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + rmSync(join(projectRoot, ".caplets", "filesystem.md"), { force: true }); + rmSync(join(projectRoot, ".caplets", "github"), { recursive: true, force: true }); + + await expect(runCli(["install"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "CONFIG_NOT_FOUND", + } satisfies Partial); + + const restored = JSON.parse(readFileSync(lockfilePath, "utf8")); + const restoredFilesystem = restored.entries.find( + (entry: { id: string }) => entry.id === "filesystem", + ); + expect(restoredFilesystem.installedHash).toMatch(/^sha256:/); + expect(restoredFilesystem.installedHash).not.toBe("sha256:stale"); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("cleans up temporary git clones after restoring from the project lockfile", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-git-restore-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + fixtureGit(repo, ["init"]); + fixtureGit(repo, ["add", "caplets/github/CAPLET.md", "caplets/github/README.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "add caplets", + ]); + const revision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + await runCli(["install", repo, "github"], { writeOut: () => {} }); + + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + lockfile.entries[0].source = { + type: "git", + repository: repo, + path: "caplets/github", + trackedRef: "HEAD", + resolvedRevision: revision, + portability: "portable", + }; + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + rmSync(join(projectRoot, ".caplets", "github"), { recursive: true, force: true }); + + const beforeRestore = capletsRestoreTempDirs(); + await runCli(["install"], { writeOut: () => {} }); + + expect(existsSync(join(projectRoot, ".caplets", "github", "CAPLET.md"))).toBe(true); + expect(capletsRestoreTempDirs()).toEqual(beforeRestore); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("accepts remote global lockfile restore through install --global --remote", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-remote-global-")); + const authDir = join(dir, "auth"); + const requests: unknown[] = []; + const out: string[] = []; + const fetchMock = vi.fn( + async (_url: Parameters[0], init?: Parameters[1]) => { + requests.push(JSON.parse(String(init?.body))); + return Response.json({ + ok: true, + result: { + installed: [ + { + id: "github", + destination: "/remote/global/github", + status: "restored", + }, + ], + }, + }); + }, + ); + try { + await new FileRemoteProfileStore({ + root: join(authDir, "remote-profiles"), + }).saveSelfHostedProfile({ + hostUrl: "http://127.0.0.1:5387", + clientId: "rcli_install_test", + clientLabel: "Remote Install Test", + credentials: { + accessToken: "remote-profile-access-token", + refreshToken: "remote-profile-refresh-token", + tokenType: "Bearer", + expiresAt: "2999-01-01T00:00:00.000Z", + }, + }); + + await runCli(["install", "--global", "--remote", "--json"], { + env: { + CAPLETS_MODE: "remote", + CAPLETS_REMOTE_URL: "http://127.0.0.1:5387", + CAPLETS_CONFIG: join(dir, "missing-user-config.json"), + CAPLETS_PROJECT_CONFIG: join(dir, "project", ".caplets", "config.json"), + }, + authDir, + fetch: fetchMock, + writeOut: (value) => out.push(value), + }); + + expect(requests).toEqual([ + { + command: "install", + arguments: { + capletIds: [], + force: false, + }, + }, + ]); + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + { + id: "github", + status: "restored", + destination: "/remote/global/github", + }, + ], + }); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("rejects project-scoped remote catalog lifecycle targets", async () => { + await expect( + runCli(["install", "--project", "--remote"], { writeOut: () => {} }), + ).rejects.toMatchObject({ + code: "REQUEST_INVALID", + message: "Cannot combine mutation target flags: --project, --remote", + } satisfies Partial); + + await expect( + runCli(["update", "--project", "--remote"], { writeOut: () => {} }), + ).rejects.toMatchObject({ + code: "REQUEST_INVALID", + message: "Cannot combine mutation target flags: --project, --remote", + } satisfies Partial); + }); + + it("refuses to restore over local modifications without force", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-restore-conflict-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + writeFileSync(join(projectRoot, ".caplets", "github", "README.md"), "local edit\n"); + + await expect(runCli(["install"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "CONFIG_EXISTS", + } satisfies Partial); + + await runCli(["install", "--force"], { writeOut: () => {} }); + expect(readFileSync(join(projectRoot, ".caplets", "github", "README.md"), "utf8")).toBe( + "Extra files are copied with directory Caplets.\n", + ); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("updates tracked Caplets from the project lockfile", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + writeFileSync(join(repo, "caplets", "github", "README.md"), "updated upstream\n"); + + await runCli(["update", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(readFileSync(join(projectRoot, ".caplets", "github", "README.md"), "utf8")).toBe( + "updated upstream\n", + ); + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + { + id: "github", + status: "updated", + destination: join(projectRoot, ".caplets", "github"), + lockfile: join(projectRoot, ".caplets.lock.json"), + hash: expect.stringMatching(/^sha256:/), + }, + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("reports symlink-materialized directory Caplets as current when unchanged", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-symlink-noop-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + writeFileSync(join(repo, "caplets", "shared.txt"), "shared source\n"); + symlinkSync("../shared.txt", join(repo, "caplets", "github", "shared-link.txt")); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + expect(lstatSync(join(projectRoot, ".caplets", "github", "shared-link.txt")).isFile()).toBe( + true, + ); + + await runCli(["update", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [ + { + id: "github", + status: "noop", + destination: join(projectRoot, ".caplets", "github"), + }, + ], + }); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("refreshes stale risk metadata when update is already current", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-noop-risk-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + const github = lockfile.entries.find((entry: { id: string }) => entry.id === "github"); + const originalBodyHash = github.risk.bodyHash; + github.risk.bodyHash = "sha256:stale-risk-metadata"; + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + + await runCli(["update", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [expect.objectContaining({ id: "github", status: "noop" })], + }); + const updatedLockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + const updatedGithub = updatedLockfile.entries.find( + (entry: { id: string }) => entry.id === "github", + ); + expect(updatedGithub.risk.bodyHash).toBe(originalBodyHash); + expect(updatedGithub.risk.bodyHash).not.toBe("sha256:stale-risk-metadata"); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("refreshes stale increased risk metadata when update is already current", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-noop-risk-increase-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + const github = lockfile.entries.find((entry: { id: string }) => entry.id === "github"); + const originalRisk = github.risk; + github.risk = { + ...github.risk, + safety: "unknown", + }; + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + + await runCli(["update", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [expect.objectContaining({ id: "github", status: "noop" })], + }); + const updatedLockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + const updatedGithub = updatedLockfile.entries.find( + (entry: { id: string }) => entry.id === "github", + ); + expect(updatedGithub.risk).toEqual(originalRisk); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("updates git-backed Caplets from the tracked ref and records the new revision", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-git-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + fixtureGit(repo, ["init"]); + fixtureGit(repo, ["add", "caplets/github/CAPLET.md", "caplets/github/README.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "add caplets", + ]); + const oldRevision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + await runCli(["install", repo, "github"], { writeOut: () => {} }); + + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + lockfile.entries[0].source = { + type: "git", + repository: repo, + path: "caplets/github", + trackedRef: "HEAD", + resolvedRevision: oldRevision, + portability: "portable", + }; + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + + writeFileSync(join(repo, "caplets", "github", "README.md"), "new git revision\n"); + fixtureGit(repo, ["add", "caplets/github/README.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "update caplet", + ]); + const newRevision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + await runCli(["update", "github"], { writeOut: () => {} }); + + expect(readFileSync(join(projectRoot, ".caplets", "github", "README.md"), "utf8")).toBe( + "new git revision\n", + ); + const updatedLockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + expect(updatedLockfile.entries[0].source.resolvedRevision).toBe(newRevision); + expect(updatedLockfile.entries[0].source.resolvedRevision).not.toBe(oldRevision); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("refreshes git source revisions when update is already current", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-git-noop-revision-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + fixtureGit(repo, ["init"]); + fixtureGit(repo, ["add", "caplets/github/CAPLET.md", "caplets/github/README.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "add caplets", + ]); + const oldRevision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + await runCli(["install", repo, "github"], { writeOut: () => {} }); + + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + lockfile.entries[0].source = { + type: "git", + repository: repo, + path: "caplets/github", + trackedRef: "HEAD", + resolvedRevision: oldRevision, + portability: "portable", + }; + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + + writeFileSync(join(repo, "NOTES.md"), "outside the caplet artifact\n"); + fixtureGit(repo, ["add", "NOTES.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "update repo metadata", + ]); + const newRevision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + await runCli(["update", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [expect.objectContaining({ id: "github", status: "noop" })], + }); + const updatedLockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + expect(updatedLockfile.entries[0].source.resolvedRevision).toBe(newRevision); + expect(updatedLockfile.entries[0].source.resolvedRevision).not.toBe(oldRevision); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("refreshes local source git metadata when update is already current", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-local-noop-git-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const out: string[] = []; + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + fixtureGit(repo, ["init"]); + fixtureGit(repo, ["add", "caplets/github/CAPLET.md", "caplets/github/README.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "add caplets", + ]); + const oldRevision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + await runCli(["install", repo, "github"], { writeOut: () => {} }); + + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + expect(lockfile.entries[0].source.gitRevision).toBe(oldRevision); + + writeFileSync(join(repo, "NOTES.md"), "outside the caplet artifact\n"); + fixtureGit(repo, ["add", "NOTES.md"]); + fixtureGit(repo, [ + "-c", + "user.email=caplets@example.com", + "-c", + "user.name=Caplets Test", + "commit", + "-m", + "update repo metadata", + ]); + const newRevision = execFileSync("git", ["-C", repo, "rev-parse", "HEAD"], { + encoding: "utf8", + env: gitFixtureEnv(), + }).trim(); + + await runCli(["update", "github", "--json"], { writeOut: (value) => out.push(value) }); + + expect(JSON.parse(out.join(""))).toMatchObject({ + entries: [expect.objectContaining({ id: "github", status: "noop" })], + }); + const updatedLockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + expect(updatedLockfile.entries[0].source.gitRevision).toBe(newRevision); + expect(updatedLockfile.entries[0].source.gitRevision).not.toBe(oldRevision); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("persists successful update entries before a later update fails", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-partial-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo], { writeOut: () => {} }); + const lockfilePath = join(projectRoot, ".caplets.lock.json"); + const lockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + const originalFilesystemHash = lockfile.entries.find( + (entry: { id: string }) => entry.id === "filesystem", + ).installedHash; + const github = lockfile.entries.find((entry: { id: string }) => entry.id === "github"); + github.source.path = join(repo, "caplets", "missing-github"); + writeFileSync(lockfilePath, `${JSON.stringify(lockfile, null, 2)}\n`); + writeFileSync(join(repo, "caplets", "filesystem.md"), capletFixture("Updated Files")); + + await expect(runCli(["update"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "CONFIG_NOT_FOUND", + } satisfies Partial); + + const updatedLockfile = JSON.parse(readFileSync(lockfilePath, "utf8")); + const updatedFilesystemHash = updatedLockfile.entries.find( + (entry: { id: string }) => entry.id === "filesystem", + ).installedHash; + expect(updatedFilesystemHash).toMatch(/^sha256:/); + expect(updatedFilesystemHash).not.toBe(originalFilesystemHash); + expect(readFileSync(join(projectRoot, ".caplets", "filesystem.md"), "utf8")).toContain( + "Updated Files", + ); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("blocks update over local modifications unless forced", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-conflict-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + writeFileSync(join(projectRoot, ".caplets", "github", "README.md"), "local edit\n"); + writeFileSync(join(repo, "caplets", "github", "README.md"), "updated upstream\n"); + + await expect(runCli(["update", "github"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "CONFIG_EXISTS", + } satisfies Partial); + + await runCli(["update", "github", "--force"], { writeOut: () => {} }); + expect(readFileSync(join(projectRoot, ".caplets", "github", "README.md"), "utf8")).toBe( + "updated upstream\n", + ); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("treats destination symlinks as local modifications during update", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-symlink-conflict-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + const installedReadme = join(projectRoot, ".caplets", "github", "README.md"); + const linkedReadme = join(projectRoot, "same-readme.txt"); + writeFileSync(linkedReadme, "Extra files are copied with directory Caplets.\n"); + rmSync(installedReadme); + symlinkSync(linkedReadme, installedReadme); + + await expect(runCli(["update", "github"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "CONFIG_EXISTS", + } satisfies Partial); + expect(lstatSync(installedReadme).isSymbolicLink()).toBe(true); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("requires force when update adds a required project binding", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-update-risk-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + writeFileSync( + join(repo, "caplets", "github", "CAPLET.md"), + [ + "---", + "name: GitHub", + "description: Work with GitHub repositories and pull requests.", + "projectBinding:", + " required: true", + " description: Requires the current repository to resolve issue references.", + "mcpServer:", + " command: npx", + " args:", + " - -y", + " - github-mcp-server", + "---", + "# GitHub", + ].join("\n"), + ); + + await expect(runCli(["update", "github"], { writeOut: () => {} })).rejects.toMatchObject({ + code: "REQUEST_INVALID", + } satisfies Partial); + + await runCli(["update", "github", "--force"], { writeOut: () => {} }); + expect(readFileSync(join(projectRoot, ".caplets", "github", "CAPLET.md"), "utf8")).toContain( + "projectBinding:", + ); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + + it("preserves the existing destination when a forced reinstall copy fails", async () => { + const dir = mkdtempSync(join(tmpdir(), "caplets-install-stage-failure-")); + const repo = join(dir, "repo"); + const projectRoot = join(dir, "project"); + const cwd = process.cwd(); + try { + writeInstallableRepo(repo); + mkdirSync(projectRoot, { recursive: true }); + process.chdir(projectRoot); + + await runCli(["install", repo, "github"], { writeOut: () => {} }); + writeFileSync(join(repo, "outside.txt"), "outside source boundary\n"); + symlinkSync(join(repo, "outside.txt"), join(repo, "caplets", "github", "outside-link")); + + await expect( + runCli(["install", repo, "github", "--force"], { writeOut: () => {} }), + ).rejects.toMatchObject({ + code: "CONFIG_INVALID", + } satisfies Partial); + expect(readFileSync(join(projectRoot, ".caplets", "github", "README.md"), "utf8")).toBe( + "Extra files are copied with directory Caplets.\n", + ); + } finally { + process.chdir(cwd); + rmSync(dir, { recursive: true, force: true }); + } + }); + it("installs a selected Caplet when an unrelated Caplet is invalid", async () => { const dir = mkdtempSync(join(tmpdir(), "caplets-install-")); const repo = join(dir, "repo"); @@ -3713,6 +4698,29 @@ function writeInstallableRepo(repo: string): void { ); } +function capletsRestoreTempDirs(): string[] { + return readdirSync(tmpdir()) + .filter((entry) => entry.startsWith("caplets-restore-")) + .sort(); +} + +function fixtureGit(repo: string, args: string[]): void { + execFileSync("git", ["-C", repo, ...args], { + env: gitFixtureEnv(), + stdio: "ignore", + }); +} + +function gitFixtureEnv(): NodeJS.ProcessEnv { + const env = { ...process.env }; + delete env.GIT_DIR; + delete env.GIT_COMMON_DIR; + delete env.GIT_WORK_TREE; + env.GIT_CONFIG_GLOBAL = "/dev/null"; + env.GIT_CONFIG_NOSYSTEM = "1"; + return env; +} + function capletFixture(name: string): string { return [ "---", diff --git a/packages/core/test/config-paths.test.ts b/packages/core/test/config-paths.test.ts index 7b1e213a..b568214f 100644 --- a/packages/core/test/config-paths.test.ts +++ b/packages/core/test/config-paths.test.ts @@ -2,11 +2,13 @@ import { describe, expect, it } from "vitest"; import { posix, win32 } from "node:path"; import { defaultAuthDir, + defaultCapletsLockfilePath, defaultConfigBaseDir, defaultConfigPath, defaultStateBaseDir, defaultUpdateCheckCacheDir, defaultUpdateCheckStateDir, + resolveProjectLockfilePath, } from "../src/config/paths"; describe("config paths", () => { @@ -22,6 +24,9 @@ describe("config paths", () => { expect(defaultAuthDir(env, home, "linux")).toBe( posix.join(home, ".local", "state", "caplets", "auth"), ); + expect(defaultCapletsLockfilePath(env, home, "linux")).toBe( + posix.join(home, ".local", "state", "caplets", "caplets.lock.json"), + ); }); it("uses absolute Unix XDG overrides", () => { @@ -36,6 +41,9 @@ describe("config paths", () => { expect(defaultAuthDir(env, "/home/alex", "darwin")).toBe( posix.join("/xdg/state", "caplets", "auth"), ); + expect(defaultCapletsLockfilePath(env, "/home/alex", "darwin")).toBe( + posix.join("/xdg/state", "caplets", "caplets.lock.json"), + ); }); it("ignores relative Unix XDG overrides", () => { @@ -65,6 +73,9 @@ describe("config paths", () => { expect(defaultAuthDir(env, "C:\\Users\\Alex", "win32")).toBe( win32.join(env.LOCALAPPDATA, "caplets", "auth"), ); + expect(defaultCapletsLockfilePath(env, "C:\\Users\\Alex", "win32")).toBe( + win32.join(env.LOCALAPPDATA, "caplets", "caplets.lock.json"), + ); }); it("falls back to Windows home app data directories when env vars are missing or relative", () => { @@ -83,6 +94,15 @@ describe("config paths", () => { expect(defaultAuthDir(env, home, "win32")).toBe( win32.join(home, "AppData", "Local", "caplets", "auth"), ); + expect(defaultCapletsLockfilePath(env, home, "win32")).toBe( + win32.join(home, "AppData", "Local", "caplets", "caplets.lock.json"), + ); + }); + + it("resolves project lockfiles next to the project root instead of inside .caplets", () => { + expect(resolveProjectLockfilePath("/workspace/project")).toBe( + posix.join("/workspace/project", ".caplets.lock.json"), + ); }); it("uses Caplets-owned update-check state and cache directories", () => { diff --git a/packages/core/test/remote-control-dispatch.test.ts b/packages/core/test/remote-control-dispatch.test.ts index 250fa92c..59321b6b 100644 --- a/packages/core/test/remote-control-dispatch.test.ts +++ b/packages/core/test/remote-control-dispatch.test.ts @@ -465,6 +465,48 @@ describe("dispatchRemoteCliRequest", () => { ).resolves.toMatchObject({ ok: true, result: { remote: true } }); }); + it("installs catalog Caplets into remote global state", async () => { + const context = testContext(); + const sourceRepo = join(context.tempRoot, "source-global"); + const sourceCaplets = join(sourceRepo, "caplets"); + const globalRoot = join(context.tempRoot, "remote-global"); + const globalLockfilePath = join(context.tempRoot, "remote-state", "caplets.lock.json"); + mkdirSync(sourceCaplets, { recursive: true }); + writeFileSync( + join(sourceCaplets, "sample.md"), + [ + "---", + "name: Sample", + "description: Sample Caplet.", + "httpApi:", + " baseUrl: http://127.0.0.1:1", + " auth:", + " type: none", + " actions:", + " check:", + " method: GET", + " path: /check", + "---", + "", + "# Sample", + "", + ].join("\n"), + ); + + await expect( + dispatchRemoteCliRequest( + { command: "install", arguments: { repo: sourceRepo, capletIds: ["sample"] } }, + { ...context, globalCapletsRoot: globalRoot, globalLockfilePath }, + ), + ).resolves.toMatchObject({ ok: true, result: { remote: true } }); + + expect(existsSync(join(globalRoot, "sample.md"))).toBe(true); + expect(existsSync(join(context.projectCapletsRoot, "sample.md"))).toBe(false); + expect(JSON.parse(readFileSync(globalLockfilePath, "utf8"))).toMatchObject({ + entries: [expect.objectContaining({ id: "sample" })], + }); + }); + it("dispatches complete_cli using server-owned config", async () => { const context = testContext(); writeFileSync( diff --git a/skills/writing-caplets/SKILL.md b/skills/writing-caplets/SKILL.md new file mode 100644 index 00000000..6137f022 --- /dev/null +++ b/skills/writing-caplets/SKILL.md @@ -0,0 +1,53 @@ +--- +name: writing-caplets +description: Use when creating, updating, reviewing, or validating Caplet files, prebuilt catalog entries, setup/auth metadata, Project Binding declarations, Vault references, runtime requirements, or public install-ready Caplets. +--- + +# Writing Caplets + +## Core Rule + +Write Caplets from the schema and nearby examples first, not from memory. Inspect `schemas/caplet.schema.json`, `apps/docs/src/content/docs/reference/caplet-files.mdx`, and similar files under `caplets/` before editing. + +Use `skills/caplets` for executing configured Caplets. This skill is for authoring and review. + +## Authoring Workflow + +1. Pick the backend family that matches the real integration: `mcpServer`, `openapiEndpoint`, `googleDiscoveryApi`, `graphqlEndpoint`, `httpApi`, `cliTools`, or `capletSet`. +2. Keep checked-in entries public-safe: no tokens, credential-bearing URLs, private provider IDs, browser profiles, user home paths, local absolute paths, or account-specific values. +3. Use `$vault:NAME` or `${vault:NAME}` for secrets that the runtime should resolve. Use `$env:NAME` only for non-secret machine-local paths or toggles. +4. Add `projectBinding.required: true` when the Caplet reads, searches, executes, or mutates project files. Explain in the body why the bound root is needed. +5. Add `setup.commands` and `setup.verify` when the backend needs local binaries, browser dependencies, generated specs, or provider setup. +6. Add `runtime.features` only for real runtime requirements such as `browser` or `docker`. +7. Keep the body useful to agents: when to use it, setup, safety, scope notes, and one or two concrete workflows. + +## Catalog-Grade Checklist + +- Description says what the Caplet lets an agent do, not how the backend is implemented. +- Tags are searchable and not noisy. +- Auth scopes are least-privilege and documented in the body. +- Mutating or destructive capabilities are called out in prose, auth scopes, CLI annotations, or narrowed operation filters. +- Local-control Caplets make risk obvious without adding install-time confirmation. +- Bundled reference files are relative, necessary, and parse under the schema. +- Project-bound entries do not hardcode local absolute paths. + +## Validation + +Run the narrowest relevant checks after edits: + +```sh +pnpm --filter @caplets/core test -- test/caplet-files.test.ts test/catalog-vault.test.ts test/exposure-discovery.test.ts test/config.test.ts test/config-validation.test.ts +pnpm schema:check +pnpm docs:check +``` + +When schema source changes, run `pnpm schema:generate` and `pnpm docs:generate` before checks. + +## Common Mistakes + +| Mistake | Fix | +| --------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- | +| Copying a personal config directly | Replace paths, profile names, tokens, and account IDs with Vault/env placeholders or public defaults. | +| Marking a provider ready without a public validation path | Keep it out of the prebuilt catalog until the endpoint, auth model, and validation command are confirmed. | +| Omitting Project Binding for repo tools | Add `projectBinding.required: true` and explain the bound-root dependency. | +| Hiding risky local control in a toolkit | Keep high-risk local-control entries explicit unless the grouping clearly promises that capability. |