diff --git a/packages/agents/README.md b/packages/agents/README.md index e32fae7a..82aaee0f 100644 --- a/packages/agents/README.md +++ b/packages/agents/README.md @@ -155,6 +155,8 @@ A declared subagent is deployed into each detected harness's project-local subag `rulebooks`, `skills`, `subagents`, and `collections` are all deployed. +Two further top-level keys name where artifacts come from rather than which to adopt: `sources` (see [Sources](#sources)) and `packages` (see [Packages](#packages)). `packages` takes the same `use`/`drop` shape as a type block, so the semantics above carry over to it unchanged. + ### Collections A collection is a traversal-only aggregate: It deploys no file of its own, but declaring it pulls in its members' transitive closure, which `sync` then deploys. Declare one like any other type: @@ -218,10 +220,69 @@ rulebooks: Each source is a `{ name, path }` pair (both required). A relative `path` resolves against the declaring file's `.agents/` directory; `~` expands to the home directory, and absolute paths are used as-is. Declaration entries stay bare slugs — resolution is transparent, so `team-standards` resolves from whichever source (or the library) provides it, with no per-entry `from:` syntax. -**Precedence.** A later-declared source shadows an earlier one, and any source shadows the library, so a source can override a same-slug library artifact. Repeating a source `name` remaps its path and moves it ahead of the sources declared before it. Because paths are `.agents/`-relative, commit only repo-relative source paths in `codeassembly.yaml`; confine machine-specific and absolute paths to `codeassembly.local.yaml`. A higher-precedence tier's `root: true` discards previously-declared sources exactly as it discards `rulebooks`, `skills`, `subagents`, and `collections`. +**Precedence.** A later-declared source shadows an earlier one, and any source shadows the library, so a source can override a same-slug library artifact. A package adopted via [`packages`](#packages) is a source too, ranked below every hand-declared one. Repeating a source `name` remaps its path and moves it ahead of the sources declared before it. Because paths are `.agents/`-relative, commit only repo-relative source paths in `codeassembly.yaml`; confine machine-specific and absolute paths to `codeassembly.local.yaml`. A higher-precedence tier's `root: true` discards previously-declared sources exactly as it discards `rulebooks`, `skills`, `subagents`, and `collections`. Every artifact type resolves through sources: An artifact's body and its closure edges (`dependencies:`, or `members:` for a collection) resolve from the source that owns it, with ownership and retraction semantics identical to a library artifact's. A source-resolved skill or subagent expands its `` directives against its own source root — it can reuse partials within its own source tree, but a target that resolves outside that root fails. A source-resolved **collection** expands its members through the resolver like any other type, and its `'@library'` token is source-scoped: It enumerates that source's own catalog rather than the built-in library. A declared source that is missing or not a directory fails the run — dry-run included — before any file is written, and a slug found in no source or the library fails with an error naming every location searched. +### Packages + +A dependency can ship the guidance for using it, and a project adopts it by naming the package — no filesystem path, no generated file to keep in sync: + +```yaml +packages: + use: + - '@williamthorsen/nmr' +``` + +That one line does two things: the package's content directory joins the source search order, and every rulebook, skill, and subagent the package ships is deployed. Nothing else is needed, because a package's whole catalog is its declaration — which is also why granularity is all-or-nothing. Adopting a package takes every artifact in its catalog; an individual one cannot be dropped, matching the existing limitation on collection members. + +`packages:` is an ordinary declaration block, so `use`, `drop`, and `root: true` behave exactly as they do for an artifact type. A project-local tier can therefore decline a package the committed tier adopted: + +```yaml +# .agents/codeassembly.local.yaml +packages: + drop: + - '@williamthorsen/nmr' +``` + +**Precedence.** Every `sources` entry, from any tier, outranks every package, and every package outranks the built-in library — a directory you pointed at by hand should win over a dependency's. Among packages the ordinary rule applies: the highest tier wins, and within a tier the last declared wins. A package that masks a library slug is reported by the same shadow warning a declared source triggers; two packages that ship the same slug resolve by precedence with no warning, and `sync --dry-run` names the source each artifact resolved from. + +**Resolution.** A declared package resolves through the module resolver, walking the `node_modules` chain Node itself searches, so it holds under pnpm's hoisting and symlinked layouts. It also holds under a `workspace:*` link, which means a repo that produces a guidance-shipping package consumes its own guidance through the same declaration a third party writes, resolved against the live source tree rather than a packed copy. A declared package that is not installed, declares no content directory, or points at a missing one fails the run — dry-run included — before any file is written, naming what was searched. + +**Discovery.** `sync` reports any direct dependency that ships content the project has not declared, printing the `packages:` block that would adopt it. That is advice, not action: an undeclared dependency contributes nothing, so installing one changes nothing about what an agent reads, and `drop` silences the advice for a package the project has turned down. + +Upgrading an already-declared package is the other case. Its catalog is read from the filesystem, so a version that adds an artifact deploys it with no declaration change — the freshness property that makes the rendered guidance a function of what is installed. `sync --dry-run` prints the resolution report naming every artifact and the source it came from, which is where that change is visible. + +#### Shipping guidance from a package + +A package declares where its content lives with a `codeassembly` key in its `package.json`, pointing at a directory structured like the library's `content/`: + +```json +{ + "name": "@williamthorsen/nmr", + "codeassembly": { "content": "content/agents" }, + "files": ["bin", "content", "dist"] +} +``` + +``` +content/agents/ + collections/ + guidance/rulebooks/ + skills/ + subagents/ +``` + +The key is required and has no default location. That is deliberate: a default would claim a directory name in every producer's package root, so instead a producer says where its content lives and can nest it under a directory it already owns — including build output, if a build step puts it there. + +A package's catalog is its rulebooks, skills, and subagents; a `collections/` entry is resolvable but not adopted on its own, so a collection reaches a consumer only when that consumer declares it by name. Its members are already in the catalog anyway, so the way to pull in an artifact from outside the package — a library rulebook, say — is a `dependencies:` edge on an artifact the catalog does contain. + +**Include the content directory in `files`.** This is the one thing most likely to go wrong, because a `workspace:*` self-link resolves the live source tree and so never exercises packing. A producer that omits the entry sees its own guidance work perfectly and every consumer's install fail. `pnpm pack` and inspecting the tarball is the check that catches it. + +Authoring the artifacts themselves is no different from authoring library content; see the content specification for frontmatter fields, `dependencies:`, `members:`, and invocation tokens. + +One shape cannot consume its own guidance: a single-package repo whose package is the repo root has no `workspace:*` self-link to resolve through. Such a repo declares a `sources:` entry pointing at the directory instead. + ### Scopes The declaration resolves in two independent **domains**, each with its own base and local tiers and its own deployment target. The tiers within a domain run lowest to highest precedence. diff --git a/packages/agents/src/commands/__tests__/init.test.ts b/packages/agents/src/commands/__tests__/init.test.ts index 5f8ae71a..d828f5e6 100644 --- a/packages/agents/src/commands/__tests__/init.test.ts +++ b/packages/agents/src/commands/__tests__/init.test.ts @@ -43,6 +43,8 @@ describe(initCommand, () => { skills: [], subagents: [], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -98,6 +100,8 @@ describe(initGlobalCommand, () => { skills: [], subagents: [], collections: ['all'], + packages: [], + declinedPackages: [], sources: [], }); }); diff --git a/packages/agents/src/commands/__tests__/sync-packages.test.ts b/packages/agents/src/commands/__tests__/sync-packages.test.ts new file mode 100644 index 00000000..ec77116b --- /dev/null +++ b/packages/agents/src/commands/__tests__/sync-packages.test.ts @@ -0,0 +1,307 @@ +import { existsSync } from 'node:fs'; +import { mkdir, readFile, rm, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; + +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; + +import type { InstallOptions } from '../../lib/types.ts'; +import { syncCommand } from '../sync.ts'; + +// Exercises the `packages:` declaration: a package's content dir joins the source search order and its catalog seeds +// the closure, so naming the package is the whole declaration. Fixture packages live under the temp project's own +// `node_modules`, which is the first directory Node's resolver searches from there — no real install involved. +describe('sync with a declared package', () => { + const PACKAGE_NAME = '@ca-fixture/guide'; + + let projectRoot: string; + let contentDir: string; + let packageDir: string; + + beforeEach(async () => { + const stamp = `${Date.now()}-${Math.random().toString(36).slice(2)}`; + projectRoot = path.join(tmpdir(), `agents-test-sync-pkg-proj-${stamp}`); + contentDir = path.join(tmpdir(), `agents-test-sync-pkg-content-${stamp}`); + packageDir = path.join(projectRoot, 'node_modules', PACKAGE_NAME); + await mkdir(path.join(projectRoot, '.agents'), { recursive: true }); + await mkdir(path.join(contentDir, 'guidance', 'rulebooks'), { recursive: true }); + await writeOverlays(); + await installPackage(PACKAGE_NAME, 'codeassembly'); + }); + + afterEach(async () => { + vi.restoreAllMocks(); + await rm(projectRoot, { recursive: true, force: true }); + await rm(contentDir, { recursive: true, force: true }); + }); + + function makeOptions(overrides: Partial = {}): InstallOptions { + return { harness: 'claude', link: false, force: false, dryRun: false, ...overrides }; + } + + const skillPath = (slug: string): string => path.join(projectRoot, '.claude', 'skills', slug, 'SKILL.md'); + const subagentPath = (slug: string): string => path.join(projectRoot, '.claude', 'agents', `${slug}.md`); + const localHostPath = (): string => path.join(projectRoot, 'CLAUDE.local.md'); + + /** Installs a fixture package under the project's `node_modules`, declaring `content` as its content directory. */ + async function installPackage(name: string, content: string): Promise { + const dir = path.join(projectRoot, 'node_modules', name); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, 'package.json'), JSON.stringify({ name, codeassembly: { content } }), 'utf8'); + } + + /** Writes the project-scope codeassembly.yaml verbatim. */ + async function declare(body: string): Promise { + await writeFile(path.join(projectRoot, '.agents', 'codeassembly.yaml'), body, 'utf8'); + } + + /** Writes a rulebook into a content root, which may be the package's, a plain source's, or the library's. */ + async function writeRulebook(root: string, slug: string, frontmatter: string, body: string): Promise { + const dir = path.join(root, 'guidance', 'rulebooks'); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, `${slug}.md`), `---\nslug: ${slug}\n${frontmatter}\n---\n\n${body}\n`, 'utf8'); + } + + /** Writes a skill into a content root, with an optional `dependencies:` block. */ + async function writeSkill(root: string, slug: string, frontmatter = ''): Promise { + const dir = path.join(root, 'skills', slug); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, 'SKILL.md'), `---\nname: ${slug}\n${frontmatter}---\n\n# ${slug}\n`, 'utf8'); + } + + /** Writes a subagent into a content root. */ + async function writeSubagent(root: string, slug: string): Promise { + const dir = path.join(root, 'subagents'); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, `${slug}.md`), `---\nname: ${slug}\n---\n\n# ${slug}\n\nUse {tool:Read}.\n`, 'utf8'); + } + + /** Writes a members-based collection into a content root. */ + async function writeCollection(root: string, slug: string, skills: ReadonlyArray): Promise { + const dir = path.join(root, 'collections'); + await mkdir(dir, { recursive: true }); + const members = `members:\n skills:\n${skills.map((member) => ` - ${member}`).join('\n')}\n`; + await writeFile(path.join(dir, `${slug}.md`), `---\nname: ${slug}\n${members}---\n\n# ${slug}\n`, 'utf8'); + } + + /** Writes the Claude harness overlay into the library, which is where the subagent transform reads it from. */ + async function writeOverlays(): Promise { + const dataDir = path.join(contentDir, 'subagents', '_data'); + await mkdir(dataDir, { recursive: true }); + await writeFile( + path.join(dataDir, 'claude.yaml'), + '_tools:\n Read: Read\n\n_defaults:\n model: sonnet\n', + 'utf8', + ); + } + + /** The package's content root, as resolved from its declared `codeassembly.content`. */ + const packageContent = (): string => path.join(packageDir, 'codeassembly'); + + it('deploys every deployable artifact the package ships, from the package name alone', async () => { + await writeRulebook(packageContent(), 'pkg-rules', 'delivery: skill\ndescription: From the package.', 'Pkg rules.'); + await writeSkill(packageContent(), 'pkg-skill'); + await writeSubagent(packageContent(), 'pkg-agent'); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(await readFile(skillPath('consult-pkg-rules'), 'utf8')).toContain('Pkg rules.'); + expect(existsSync(skillPath('pkg-skill'))).toBe(true); + expect(existsSync(subagentPath('pkg-agent'))).toBe(true); + }); + + it('delivers an ambient rulebook the package ships to the local host', async () => { + await writeRulebook(packageContent(), 'pkg-ambient', 'delivery: ambient', 'Ambient package rules.'); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(await readFile(localHostPath(), 'utf8')).toContain('Ambient package rules.'); + }); + + it('resolves a collection the package ships when the project declares it', async () => { + await writeSkill(packageContent(), 'member-skill'); + await writeCollection(packageContent(), 'pkg-bundle', ['member-skill']); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\ncollections:\n use:\n - pkg-bundle\n`); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(existsSync(skillPath('member-skill'))).toBe(true); + }); + + it('pulls in a library artifact a package artifact depends on', async () => { + await writeRulebook(contentDir, 'library-dep', 'delivery: skill', 'Library dependency.'); + await writeSkill(packageContent(), 'pkg-skill', 'dependencies:\n rulebooks:\n - library-dep\n'); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(await readFile(skillPath('consult-library-dep'), 'utf8')).toContain('Library dependency.'); + }); + + it('lets a hand-declared source outrank a package source on a shared slug', async () => { + const sourceDir = path.join(projectRoot, 'local-guidance'); + await writeRulebook(sourceDir, 'contested', 'delivery: ambient', 'Source body.'); + await writeRulebook(packageContent(), 'contested', 'delivery: ambient', 'Package body.'); + await declare( + `sources:\n - name: org\n path: ${sourceDir}\npackages:\n use:\n - '${PACKAGE_NAME}'\nrulebooks:\n use:\n - contested\n`, + ); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + const localHost = await readFile(localHostPath(), 'utf8'); + expect(localHost).toContain('Source body.'); + expect(localHost).not.toContain('Package body.'); + }); + + it('warns when a package source shadows a same-slug library artifact', async () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => undefined); + await writeRulebook(contentDir, 'shadowed', 'delivery: ambient', 'Library body.'); + await writeRulebook(packageContent(), 'shadowed', 'delivery: ambient', 'Package body.'); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(await readFile(localHostPath(), 'utf8')).toContain('Package body.'); + expect(warn.mock.calls.flat().join('\n')).toMatch(/shadow.*shadowed/s); + }); + + it('keeps a hand-declared source resolving alongside a declared package', async () => { + const sourceDir = path.join(projectRoot, 'local-guidance'); + await writeRulebook(sourceDir, 'from-source', 'delivery: ambient', 'Source rules.'); + await writeRulebook(packageContent(), 'from-package', 'delivery: ambient', 'Package rules.'); + await declare( + `sources:\n - name: org\n path: ${sourceDir}\npackages:\n use:\n - '${PACKAGE_NAME}'\nrulebooks:\n use:\n - from-source\n`, + ); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + const localHost = await readFile(localHostPath(), 'utf8'); + expect(localHost).toContain('Source rules.'); + expect(localHost).toContain('Package rules.'); + }); + + it('retracts a package artifact once a higher tier drops the package', async () => { + await writeRulebook(packageContent(), 'pkg-ambient', 'delivery: ambient', 'Ambient package rules.'); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + await syncCommand(makeOptions(), projectRoot, contentDir); + expect(await readFile(localHostPath(), 'utf8')).toContain('Ambient package rules.'); + + await writeFile( + path.join(projectRoot, '.agents', 'codeassembly.local.yaml'), + `packages:\n drop:\n - '${PACKAGE_NAME}'\n`, + 'utf8', + ); + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(await readFile(localHostPath(), 'utf8')).not.toContain('Ambient package rules.'); + }); + + it('lets a higher-tier package outrank a committed-tier one on a shared slug', async () => { + await installPackage('@ca-fixture/base', 'codeassembly'); + await installPackage('@ca-fixture/local', 'codeassembly'); + const baseContent = path.join(projectRoot, 'node_modules', '@ca-fixture/base', 'codeassembly'); + const localContent = path.join(projectRoot, 'node_modules', '@ca-fixture/local', 'codeassembly'); + await writeRulebook(baseContent, 'contested', 'delivery: ambient', 'Base body.'); + await writeRulebook(localContent, 'contested', 'delivery: ambient', 'Local body.'); + await declare("packages:\n use:\n - '@ca-fixture/base'\n"); + await writeFile( + path.join(projectRoot, '.agents', 'codeassembly.local.yaml'), + "packages:\n use:\n - '@ca-fixture/local'\n", + 'utf8', + ); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + const localHost = await readFile(localHostPath(), 'utf8'); + expect(localHost).toContain('Local body.'); + expect(localHost).not.toContain('Base body.'); + }); + + it('lets the last package declared in a tier outrank an earlier one on a shared slug', async () => { + await installPackage('@ca-fixture/first', 'codeassembly'); + await installPackage('@ca-fixture/second', 'codeassembly'); + const firstContent = path.join(projectRoot, 'node_modules', '@ca-fixture/first', 'codeassembly'); + const secondContent = path.join(projectRoot, 'node_modules', '@ca-fixture/second', 'codeassembly'); + await writeRulebook(firstContent, 'contested', 'delivery: ambient', 'First body.'); + await writeRulebook(secondContent, 'contested', 'delivery: ambient', 'Second body.'); + await declare("packages:\n use:\n - '@ca-fixture/first'\n - '@ca-fixture/second'\n"); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + const localHost = await readFile(localHostPath(), 'utf8'); + expect(localHost).toContain('Second body.'); + expect(localHost).not.toContain('First body.'); + }); + + it('stops advising a package the project declined with drop', async () => { + const info = vi.spyOn(console, 'info').mockImplementation(() => undefined); + await writeRulebook(packageContent(), 'pkg-ambient', 'delivery: ambient', 'Ambient package rules.'); + await writeFile( + path.join(projectRoot, 'package.json'), + JSON.stringify({ name: 'consumer', devDependencies: { [PACKAGE_NAME]: '1.0.0' } }), + 'utf8', + ); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + await writeFile( + path.join(projectRoot, '.agents', 'codeassembly.local.yaml'), + `packages:\n drop:\n - '${PACKAGE_NAME}'\n`, + 'utf8', + ); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(info.mock.calls.flat().join('\n')).not.toContain(PACKAGE_NAME); + }); + + it('advises adopting an installed dependency that ships guidance the project has not declared', async () => { + const info = vi.spyOn(console, 'info').mockImplementation(() => undefined); + await writeFile( + path.join(projectRoot, 'package.json'), + JSON.stringify({ name: 'consumer', devDependencies: { [PACKAGE_NAME]: '1.0.0' } }), + 'utf8', + ); + await declare('rulebooks:\n use: []\n'); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + const advice = info.mock.calls.flat().join('\n'); + expect(advice).toContain(PACKAGE_NAME); + expect(advice).toMatch(/packages:\n {2}use:/); + }); + + it('stops advising once the dependency is declared', async () => { + const info = vi.spyOn(console, 'info').mockImplementation(() => undefined); + await writeFile( + path.join(projectRoot, 'package.json'), + JSON.stringify({ name: 'consumer', devDependencies: { [PACKAGE_NAME]: '1.0.0' } }), + 'utf8', + ); + await writeRulebook(packageContent(), 'pkg-ambient', 'delivery: ambient', 'Ambient package rules.'); + await declare(`packages:\n use:\n - '${PACKAGE_NAME}'\n`); + + await syncCommand(makeOptions(), projectRoot, contentDir); + + expect(info.mock.calls.flat().join('\n')).not.toContain('has not declared'); + }); + + it('fails the run when a declared package is not installed, writing nothing', async () => { + await declare("packages:\n use:\n - '@ca-fixture/absent'\n"); + + await expect(syncCommand(makeOptions(), projectRoot, contentDir)).rejects.toThrow( + /"@ca-fixture\/absent" is not installed/, + ); + expect(existsSync(path.join(projectRoot, '.agents', 'rulebooks'))).toBe(false); + }); + + it('fails the run when a declared package ships no content directory, writing nothing', async () => { + await installPackage('@ca-fixture/empty', 'missing-dir'); + await declare("packages:\n use:\n - '@ca-fixture/empty'\n"); + + await expect(syncCommand(makeOptions(), projectRoot, contentDir)).rejects.toThrow( + /Invalid declared source.*@ca-fixture\/empty/s, + ); + expect(existsSync(path.join(projectRoot, '.agents', 'rulebooks'))).toBe(false); + }); +}); diff --git a/packages/agents/src/commands/sync.ts b/packages/agents/src/commands/sync.ts index fe7879af..c8937c71 100644 --- a/packages/agents/src/commands/sync.ts +++ b/packages/agents/src/commands/sync.ts @@ -6,7 +6,7 @@ import process from 'node:process'; import { appendAmbientRegion, classifyAmbientRegion, injectAmbientRegion } from '../lib/ambient-region.ts'; import { makeArtifactMarker } from '../lib/artifact-marker.ts'; -import { artifactFrontmatterPath, type ArtifactType } from '../lib/artifact-types.ts'; +import { ARTIFACT_TYPE_VALUES, artifactFrontmatterPath, type ArtifactType } from '../lib/artifact-types.ts'; import { resolveDeclaration } from '../lib/codeassembly-manifest.ts'; import { resolveContentDir } from '../lib/content-resolver.ts'; import { @@ -15,11 +15,13 @@ import { hasLibraryArtifact, type SourceResolver, } from '../lib/content-sources.ts'; -import { resolveClosure } from '../lib/dependency-resolver.ts'; +import { type DirectArtifacts, resolveClosure } from '../lib/dependency-resolver.ts'; import { readDirEntries, readFileOrEmpty, writeIfChanged } from '../lib/fs-helpers.ts'; import { checkGitIgnored } from '../lib/git-ignore.ts'; import { HARNESSES, resolveAmbientHostPath, resolveHarnessIds, resolveHarnessPaths } from '../lib/harness.ts'; import { loadHarnessOverlay } from '../lib/harness-overlay.ts'; +import { enumerateCatalogSlugs } from '../lib/library-catalog.ts'; +import { findUndeclaredGuidancePackages, resolvePackageSources } from '../lib/package-sources.ts'; import { collectPromptEntries, renderPromptEntries } from '../lib/prompts-yml.ts'; import { hasPromptsRegion, injectPromptsRegion, removePromptsRegion } from '../lib/prompts-yml-region.ts'; import { parseRulebookFile } from '../lib/rulebook-schema.ts'; @@ -163,20 +165,32 @@ async function reconcileDomain( const contentDir = contentDirOverride ?? resolveContentDir(); + // A declared package contributes both a source and a set of seeds: Its content dir joins the search order below the + // hand-declared sources, so a hand-pointed local directory outranks a dependency, and everything it ships seeds the + // closure — which is what makes naming the package the whole declaration. + const packageSources = await resolvePackageSources(declaration.packages, domain.baseDir); + const sources = [...declaration.sources, ...packageSources]; + // Resolution searches declared sources (highest precedence first) then the built-in library. Validate each declared // source up front so a missing or non-directory source fails the whole run — dry-run included — before any write. - const resolver = createSourceResolver(declaration.sources, contentDir); - await assertValidSources(declaration.sources); + const resolver = createSourceResolver(sources, contentDir); + await assertValidSources(sources); + + // Enumerated after validation, so a package whose content dir is missing has already failed the run. + const packageCatalogs = await Promise.all(packageSources.map((source) => enumerateCatalogSlugs(source.dir))); // Expand declared collections — and any artifact's own dependencies — into the deployable per-type sets before // resolving against the sources and library, so a declared collection deploys exactly its transitive closure. const closure = await resolveClosure( - { - rulebook: declaration.rulebooks, - skill: declaration.skills, - subagent: declaration.subagents, - collection: declaration.collections, - }, + mergeSeeds([ + { + rulebook: declaration.rulebooks, + skill: declaration.skills, + subagent: declaration.subagents, + collection: declaration.collections, + }, + ...packageCatalogs, + ]), resolver, ); const declaredRulebooks = closure.rulebooks; @@ -197,7 +211,7 @@ async function reconcileDomain( // Rulebook skills and declared skills share the project-local skills dirs. A directory name claimed by both // delivery namespaces would clobber, so reject the overlap before any write. - assertNoCrossNamespaceCollisions([...desiredSkillDirs.values()], declaredSkillSet); + assertNoCrossNamespaceCollisions(desiredSkillDirs.values().toArray(), declaredSkillSet); // Skill delivery targets project-local harness skills dirs, gated by detection (or `--harness`). Passing // `projectRoot` as the base is what keeps the skills project-scoped, and keeps tests out of the real home dir. Each @@ -248,7 +262,7 @@ async function reconcileDomain( }; }), ); - // Declared subagents reconcile file-based (flat `.md` files, not directories): an owned subagent file is one whose + // Declared subagents reconcile file-based (flat `.md` files, not directories): An owned subagent file is one whose // content carries the `codeassembly-subagent:` marker. It is an orphan once its slug is no longer declared. A // marker-less hand-authored file is never claimed, so it survives untouched. const subagentOrphansByDir = await Promise.all( @@ -323,11 +337,11 @@ async function reconcileDomain( } } - // Reconcile declared skills per targeted harness, independently of the rulebook-skill pass above: retract owned + // Reconcile declared skills per targeted harness, independently of the rulebook-skill pass above: Retract owned // declared-skill dirs no longer declared, then deploy each declared skill into `//`. await reconcileDeclaredSkills(harnessSkillTargets, declaredSkillOrphansByDir, resolvedSkills); - // Reconcile declared subagents per targeted harness, independently of the skill passes: retract owned subagent + // Reconcile declared subagents per targeted harness, independently of the skill passes: Retract owned subagent // files no longer declared, then deploy each declared subagent as `/.md` with the harness // transform applied and the ownership marker stamped. await reconcileDeclaredSubagents(harnessSubagentTargets, subagentOrphansByDir, resolvedSubagents); @@ -361,18 +375,28 @@ async function reconcileDomain( if (shadows.length > 0) { console.warn(renderShadowWarning(shadows)); } + + // Otherwise a consumer has to learn a third party's catalog by hand to discover there is anything to adopt. This is + // advice, not action: Nothing is deployed until the project declares the package. + const undeclared = await findUndeclaredGuidancePackages( + [...declaration.packages, ...declaration.declinedPackages], + domain.baseDir, + ); + if (undeclared.length > 0) { + console.info(renderPackageAdvice(undeclared)); + } } // region | Helpers -/** True when a skill targets the given harness — either it names no harnesses (so all) or lists this one. */ +/** True when a skill targets the given harness; either it names no harnesses (so all) or lists this one. */ function skillTargetsHarness(skill: ResolvedSkill, harnessId: HarnessId): boolean { return skill.targetHarnesses === undefined || skill.targetHarnesses.includes(harnessId); } /** * Throws when any declared source path is missing, not a directory, or unreadable, so a bad source fails the whole - * run — dry-run included — before any file is touched. The error names each offending source and what is wrong with it. + * run (dry-run included) before any file is touched. The error names each offending source and what is wrong with it. */ async function assertValidSources(sources: ReadonlyArray<{ name: string; dir: string }>): Promise { const invalid: Array = []; @@ -390,10 +414,10 @@ async function assertValidSources(sources: ReadonlyArray<{ name: string; dir: st } /** - * Reports what disqualifies `dir` as a source — that it is missing, not a directory, or unreadable — or `undefined` + * Reports what disqualifies `dir` as a source (that it is missing, not a directory, or unreadable) or `undefined` * when valid. Validity requires both that `dir` is a directory and that the process can read and traverse it, because * `stat` alone passes a directory that is itself unreadable (`stat` needs only search permission on the parent chain, - * not on `dir`). Any permission failure — from the `stat` or the read-and-traverse access probe — folds into the + * not on `dir`). Any permission failure (from the `stat` or the read-and-traverse access probe) folds into the * "unreadable" case so it surfaces through the attributed `Invalid declared source(s)` error naming `dir`. */ async function describeSourceProblem(dir: string): Promise { @@ -414,6 +438,21 @@ async function describeSourceProblem(dir: string): Promise { } } +/** + * Concatenates per-type seed sets into the one set that seeds closure resolution. Deduping is deliberately left out: + * `resolveClosure` already dedupes by slug as it walks, so a slug both declared directly and enumerated from a + * package's catalog is visited once. + */ +function mergeSeeds(sets: ReadonlyArray): DirectArtifacts { + const merged: Record> = { rulebook: [], skill: [], subagent: [], collection: [] }; + for (const set of sets) { + for (const type of ARTIFACT_TYPE_VALUES) { + merged[type].push(...(set[type] ?? [])); + } + } + return merged; +} + /** A planned write whose destination must be sync-owned (or absent) before the write proceeds. */ interface OwnedTarget { readonly filePath: string; @@ -500,10 +539,10 @@ function assertNoCrossNamespaceCollisions( rulebookSkillDirs: ReadonlyArray, declaredSkillSlugs: Set, ): void { - const collisions = [...new Set(rulebookSkillDirs)].filter((dir) => declaredSkillSlugs.has(dir)); - if (collisions.length > 0) { + const collisions = new Set(rulebookSkillDirs).intersection(declaredSkillSlugs); + if (collisions.size > 0) { throw new Error( - `Skill directory name collision across delivery namespaces: ${collisions.join(', ')} ` + + `Skill directory name collision across delivery namespaces: ${Array.from(collisions).join(', ')} ` + 'is delivered as both a rulebook skill and a declared skill. Rename one so they no longer share a directory.', ); } @@ -1153,6 +1192,19 @@ function renderResolutionReport(entries: ReadonlyArray): string return ['[dry-run] sync would resolve:', ...lines].join('\n'); } +/** + * Renders the advice naming each dependency that ships content the project has not declared, as the `packages:` block + * that adopts them. Emitted as the block rather than as prose so it can be pasted rather than transcribed. + */ +function renderPackageAdvice(names: ReadonlyArray): string { + const subject = names.length === 1 ? 'dependency ships' : 'dependencies ship'; + const entries = names.map((name) => ` - '${name}'`).join('\n'); + return ( + `💡 ${names.length} ${subject} CodeAssembly guidance this project has not declared. ` + + `To adopt, add to .agents/codeassembly.yaml:\n\npackages:\n use:\n${entries}\n` + ); +} + /** Renders the real-run warning naming each deployed artifact that shadows a same-slug library artifact. */ function renderShadowWarning(shadows: ReadonlyArray): string { const details = shadows diff --git a/packages/agents/src/lib/__tests__/codeassembly-manifest.test.ts b/packages/agents/src/lib/__tests__/codeassembly-manifest.test.ts index d970d21b..be981a14 100644 --- a/packages/agents/src/lib/__tests__/codeassembly-manifest.test.ts +++ b/packages/agents/src/lib/__tests__/codeassembly-manifest.test.ts @@ -47,6 +47,8 @@ describe(resolveDeclaration, () => { skills: [], subagents: [], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -58,6 +60,8 @@ describe(resolveDeclaration, () => { skills: [], subagents: [], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -69,6 +73,8 @@ describe(resolveDeclaration, () => { skills: ['one', 'two'], subagents: [], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -80,6 +86,8 @@ describe(resolveDeclaration, () => { skills: [], subagents: ['canary', 'other'], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -91,6 +99,8 @@ describe(resolveDeclaration, () => { skills: [], subagents: [], collections: ['recommended', 'other'], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -104,6 +114,8 @@ describe(resolveDeclaration, () => { skills: ['one'], subagents: ['canary'], collections: ['recommended'], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -118,6 +130,8 @@ describe(resolveDeclaration, () => { skills: ['one', 'two'], subagents: ['canary', 'other'], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -130,6 +144,8 @@ describe(resolveDeclaration, () => { skills: [], subagents: [], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); @@ -142,22 +158,83 @@ describe(resolveDeclaration, () => { skills: [], subagents: [], collections: ['other'], + packages: [], + declinedPackages: [], sources: [], }); }); + it('resolves additive package use, deduplicating', async () => { + await writeProject("packages:\n use:\n - '@williamthorsen/nmr'\n - readyup\n - '@williamthorsen/nmr'\n"); + expect(await resolveDeclaration({ cwd })).toEqual({ + rulebooks: [], + skills: [], + subagents: [], + collections: [], + packages: ['@williamthorsen/nmr', 'readyup'], + declinedPackages: [], + sources: [], + }); + }); + + it('lets a higher tier drop a package inherited from a lower tier, recording it as declined', async () => { + await writeProject("packages:\n use:\n - '@williamthorsen/nmr'\n - readyup\n"); + await writeLocal("packages:\n drop:\n - '@williamthorsen/nmr'\n"); + expect(await resolveDeclaration({ cwd })).toEqual({ + rulebooks: [], + skills: [], + subagents: [], + collections: [], + packages: ['readyup'], + declinedPackages: ['@williamthorsen/nmr'], + sources: [], + }); + }); + + it('orders packages highest tier first, matching how sources are ordered', async () => { + await writeProject("packages:\n use:\n - '@acme/base'\n"); + await writeLocal("packages:\n use:\n - '@acme/local'\n"); + + expect((await resolveDeclaration({ cwd }))?.packages).toEqual(['@acme/local', '@acme/base']); + }); + + it('orders packages declared in one tier last-declared first', async () => { + await writeProject("packages:\n use:\n - '@acme/first'\n - '@acme/second'\n"); + + expect((await resolveDeclaration({ cwd }))?.packages).toEqual(['@acme/second', '@acme/first']); + }); + + it('treats a package re-adopted by a higher tier as adopted rather than declined', async () => { + await writeProject("packages:\n drop:\n - '@acme/pkg'\n"); + await writeLocal("packages:\n use:\n - '@acme/pkg'\n"); + + const declaration = await resolveDeclaration({ cwd }); + + expect(declaration?.packages).toEqual(['@acme/pkg']); + expect(declaration?.declinedPackages).toEqual([]); + }); + + it('clears declined packages when a higher tier declares root: true', async () => { + await writeProject("packages:\n drop:\n - '@acme/pkg'\n"); + await writeLocal('root: true\n'); + + expect((await resolveDeclaration({ cwd }))?.declinedPackages).toEqual([]); + }); + it('discards every type from lower tiers when a higher tier declares root: true', async () => { await writeProject( - 'rulebooks:\n use:\n - alpha\nskills:\n use:\n - one\nsubagents:\n use:\n - canary\ncollections:\n use:\n - recommended\n', + "rulebooks:\n use:\n - alpha\nskills:\n use:\n - one\nsubagents:\n use:\n - canary\ncollections:\n use:\n - recommended\npackages:\n use:\n - '@acme/old'\n", ); await writeLocal( - 'root: true\nrulebooks:\n use:\n - beta\nskills:\n use:\n - two\nsubagents:\n use:\n - other\ncollections:\n use:\n - fresh\n', + "root: true\nrulebooks:\n use:\n - beta\nskills:\n use:\n - two\nsubagents:\n use:\n - other\ncollections:\n use:\n - fresh\npackages:\n use:\n - '@acme/new'\n", ); expect(await resolveDeclaration({ cwd })).toEqual({ rulebooks: ['beta'], skills: ['two'], subagents: ['other'], collections: ['fresh'], + packages: ['@acme/new'], + declinedPackages: [], sources: [], }); }); @@ -169,6 +246,8 @@ describe(resolveDeclaration, () => { skills: ['gamma'], subagents: [], collections: [], + packages: [], + declinedPackages: [], sources: [], }); }); diff --git a/packages/agents/src/lib/__tests__/codeassembly-schema.test.ts b/packages/agents/src/lib/__tests__/codeassembly-schema.test.ts index 9128ee98..1d6e7652 100644 --- a/packages/agents/src/lib/__tests__/codeassembly-schema.test.ts +++ b/packages/agents/src/lib/__tests__/codeassembly-schema.test.ts @@ -47,6 +47,26 @@ describe(parseCodeAssemblyFile, () => { expect(declaration.collections?.use).toEqual([{ name: 'gamma' }]); }); + it('parses a packages declaration of scoped and unscoped names', () => { + const declaration = parseCodeAssemblyFile( + "packages:\n use:\n - '@williamthorsen/nmr'\n - readyup\n drop:\n - '@acme/legacy'\n", + ); + + expect(declaration.packages?.use).toEqual([{ name: '@williamthorsen/nmr' }, { name: 'readyup' }]); + expect(declaration.packages?.drop).toEqual([{ name: '@acme/legacy' }]); + }); + + it('tolerates a packages key whose value is null (all entries commented out)', () => { + const declaration = parseCodeAssemblyFile('packages:\n'); + + expect(declaration.packages).toBeUndefined(); + expect(declaration.root).toBe(false); + }); + + it('throws on an unknown key inside the packages block', () => { + expect(() => parseCodeAssemblyFile('packages:\n install:\n - alpha\n')).toThrow(/install/); + }); + it('throws on an unknown top-level key (typo protection)', () => { expect(() => parseCodeAssemblyFile('rulebookz:\n use:\n - alpha\n')).toThrow(/rulebookz/); }); diff --git a/packages/agents/src/lib/__tests__/package-sources.test.ts b/packages/agents/src/lib/__tests__/package-sources.test.ts new file mode 100644 index 00000000..2b9bf77b --- /dev/null +++ b/packages/agents/src/lib/__tests__/package-sources.test.ts @@ -0,0 +1,234 @@ +import { mkdir, rm, symlink, writeFile } from 'node:fs/promises'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; + +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; + +import { findUndeclaredGuidancePackages, resolvePackageSources } from '../package-sources.ts'; + +describe(resolvePackageSources, () => { + let baseDir: string; + + beforeEach(async () => { + baseDir = await makeBaseDir('pkgsrc'); + }); + + afterEach(async () => { + await rm(baseDir, { recursive: true, force: true }); + }); + + it('resolves an unscoped package to its declared content directory', async () => { + const dir = await installPackage(baseDir, 'ca-fixture-unscoped', { codeassembly: { content: 'codeassembly' } }); + + expect(await resolvePackageSources(['ca-fixture-unscoped'], baseDir)).toEqual([ + { name: 'ca-fixture-unscoped', dir: path.join(dir, 'codeassembly') }, + ]); + }); + + it('resolves a scoped package to a content directory nested under one it already owns', async () => { + const dir = await installPackage(baseDir, '@ca-fixture/nmr', { codeassembly: { content: 'content/agents' } }); + + expect(await resolvePackageSources(['@ca-fixture/nmr'], baseDir)).toEqual([ + { name: '@ca-fixture/nmr', dir: path.join(dir, 'content', 'agents') }, + ]); + }); + + it('resolves a package that ships no JavaScript at all', async () => { + const dir = await installPackage(baseDir, '@ca-fixture/guidance-only', { codeassembly: { content: 'guidance' } }); + + expect(await resolvePackageSources(['@ca-fixture/guidance-only'], baseDir)).toEqual([ + { name: '@ca-fixture/guidance-only', dir: path.join(dir, 'guidance') }, + ]); + }); + + // Mirrors how pnpm lays out both an external dependency (a link into `.pnpm/`) and a `workspace:*` sibling: the + // `node_modules` entry is a symlink, and the content resolves through it to the real directory. + it('resolves a package whose node_modules entry is a symlink', async () => { + const realDir = path.join(baseDir, 'workspace-packages', 'linked'); + await mkdir(realDir, { recursive: true }); + await writeFile( + path.join(realDir, 'package.json'), + JSON.stringify({ name: '@ca-fixture/linked', codeassembly: { content: 'guidance' } }), + 'utf8', + ); + await mkdir(path.join(baseDir, 'node_modules', '@ca-fixture'), { recursive: true }); + await symlink(realDir, path.join(baseDir, 'node_modules', '@ca-fixture', 'linked'), 'dir'); + + expect(await resolvePackageSources(['@ca-fixture/linked'], baseDir)).toEqual([ + { name: '@ca-fixture/linked', dir: path.join(baseDir, 'node_modules', '@ca-fixture', 'linked', 'guidance') }, + ]); + }); + + it('preserves declaration order rather than sorting', async () => { + await installPackage(baseDir, '@ca-fixture/beta', { codeassembly: { content: 'c' } }); + await installPackage(baseDir, '@ca-fixture/alpha', { codeassembly: { content: 'c' } }); + + const resolved = await resolvePackageSources(['@ca-fixture/beta', '@ca-fixture/alpha'], baseDir); + + expect(resolved.map((source) => source.name)).toEqual(['@ca-fixture/beta', '@ca-fixture/alpha']); + }); + + it('throws when a declared package is not installed, naming the directories searched', async () => { + await expect(resolvePackageSources(['@ca-fixture/absent'], baseDir)).rejects.toThrow( + /not installed.*node_modules[/\\]@ca-fixture[/\\]absent/s, + ); + }); + + it('throws when a declared package name could never resolve from node_modules', async () => { + await expect(resolvePackageSources(['node:fs'], baseDir)).rejects.toThrow(/"node:fs" is not installed/); + }); + + // Node answers a relative specifier with the anchor directory itself, so a path here names a directory rather than + // resolving through `node_modules`: `./guidance` sits under `baseDir`, and `../sibling` outside it. + it.each(['./guidance', '../sibling', '.', path.join(path.sep, 'abs', 'path')])( + 'throws when a declared name is the filesystem path %s', + async (name) => { + await expect(resolvePackageSources([name], baseDir)).rejects.toThrow(/is a filesystem path, not a package name/); + }, + ); + + it('resolves a path-shaped name to nothing rather than a directory that happens to sit there', async () => { + const stray = path.join(baseDir, 'guidance'); + await mkdir(stray, { recursive: true }); + await writeFile( + path.join(stray, 'package.json'), + JSON.stringify({ name: 'stray', codeassembly: { content: 'c' } }), + 'utf8', + ); + + await expect(resolvePackageSources(['./guidance'], baseDir)).rejects.toThrow(/is a filesystem path/); + }); + + it('throws when a package declares no codeassembly content', async () => { + await installPackage(baseDir, '@ca-fixture/plain', {}); + + await expect(resolvePackageSources(['@ca-fixture/plain'], baseDir)).rejects.toThrow( + /"@ca-fixture\/plain" declares no CodeAssembly content/, + ); + }); + + it('throws when a package declares a malformed codeassembly content value', async () => { + await installPackage(baseDir, '@ca-fixture/broken', { codeassembly: { content: 42 } }); + + await expect(resolvePackageSources(['@ca-fixture/broken'], baseDir)).rejects.toThrow( + /"@ca-fixture\/broken" declares an invalid "codeassembly" key/, + ); + }); + + it('throws when a package.json is not valid JSON, naming the package', async () => { + const dir = path.join(baseDir, 'node_modules', '@ca-fixture', 'malformed'); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, 'package.json'), '{ not json', 'utf8'); + + await expect(resolvePackageSources(['@ca-fixture/malformed'], baseDir)).rejects.toThrow( + /"@ca-fixture\/malformed" has an unreadable package\.json/, + ); + }); +}); + +describe(findUndeclaredGuidancePackages, () => { + let baseDir: string; + + beforeEach(async () => { + baseDir = await makeBaseDir('pkgscan'); + }); + + afterEach(async () => { + await rm(baseDir, { recursive: true, force: true }); + }); + + it('reports a guidance-shipping dependency the project has not declared', async () => { + await installPackage(baseDir, '@ca-fixture/ships', { codeassembly: { content: 'c' } }); + await writeProjectManifest(baseDir, { dependencies: { '@ca-fixture/ships': '1.0.0' } }); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual(['@ca-fixture/ships']); + }); + + it('reports a guidance-shipping devDependency', async () => { + await installPackage(baseDir, '@ca-fixture/tooling', { codeassembly: { content: 'c' } }); + await writeProjectManifest(baseDir, { devDependencies: { '@ca-fixture/tooling': '1.0.0' } }); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual(['@ca-fixture/tooling']); + }); + + it('stays silent once the package is declared', async () => { + await installPackage(baseDir, '@ca-fixture/ships', { codeassembly: { content: 'c' } }); + await writeProjectManifest(baseDir, { dependencies: { '@ca-fixture/ships': '1.0.0' } }); + + expect(await findUndeclaredGuidancePackages(['@ca-fixture/ships'], baseDir)).toEqual([]); + }); + + it('never reports a dependency that declares no content', async () => { + await installPackage(baseDir, '@ca-fixture/plain', {}); + await writeProjectManifest(baseDir, { dependencies: { '@ca-fixture/plain': '1.0.0' } }); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual([]); + }); + + it('never reports a dependency that is declared but not installed', async () => { + await writeProjectManifest(baseDir, { dependencies: { '@ca-fixture/absent': '1.0.0' } }); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual([]); + }); + + it('sorts the reported names', async () => { + await installPackage(baseDir, '@ca-fixture/zulu', { codeassembly: { content: 'c' } }); + await installPackage(baseDir, '@ca-fixture/alpha', { codeassembly: { content: 'c' } }); + await writeProjectManifest(baseDir, { + dependencies: { '@ca-fixture/zulu': '1.0.0' }, + devDependencies: { '@ca-fixture/alpha': '1.0.0' }, + }); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual(['@ca-fixture/alpha', '@ca-fixture/zulu']); + }); + + it('reports nothing when the project has no package.json', async () => { + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual([]); + }); + + it('reports nothing when the project package.json will not parse', async () => { + await writeFile(path.join(baseDir, 'package.json'), '{ not json', 'utf8'); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual([]); + }); + + it('skips a dependency whose own package.json will not parse rather than failing', async () => { + const dir = path.join(baseDir, 'node_modules', '@ca-fixture', 'malformed'); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, 'package.json'), '{ not json', 'utf8'); + await installPackage(baseDir, '@ca-fixture/ships', { codeassembly: { content: 'c' } }); + await writeProjectManifest(baseDir, { + dependencies: { '@ca-fixture/malformed': '1.0.0', '@ca-fixture/ships': '1.0.0' }, + }); + + expect(await findUndeclaredGuidancePackages([], baseDir)).toEqual(['@ca-fixture/ships']); + }); +}); + +// region | Helpers + +/** + * Installs a fixture package under `baseDir`'s `node_modules` and returns its directory. Fixture names are + * deliberately distinctive: Resolution searches every ancestor `node_modules`, so a common name could resolve against + * a real package outside the temp tree. + */ +async function installPackage(baseDir: string, name: string, manifest: Record): Promise { + const dir = path.join(baseDir, 'node_modules', name); + await mkdir(dir, { recursive: true }); + await writeFile(path.join(dir, 'package.json'), JSON.stringify({ name, ...manifest }), 'utf8'); + return dir; +} + +/** Creates a uniquely named temp directory to act as a project root. */ +async function makeBaseDir(label: string): Promise { + const dir = path.join(tmpdir(), `agents-test-${label}-${Date.now()}-${Math.random().toString(36).slice(2)}`); + await mkdir(dir, { recursive: true }); + return dir; +} + +/** Writes the consuming project's own `package.json` at `baseDir`. */ +async function writeProjectManifest(baseDir: string, manifest: Record): Promise { + await writeFile(path.join(baseDir, 'package.json'), JSON.stringify(manifest), 'utf8'); +} + +// endregion | Helpers diff --git a/packages/agents/src/lib/codeassembly-manifest.ts b/packages/agents/src/lib/codeassembly-manifest.ts index 6ae58a73..41c77e39 100644 --- a/packages/agents/src/lib/codeassembly-manifest.ts +++ b/packages/agents/src/lib/codeassembly-manifest.ts @@ -10,7 +10,9 @@ import { resolveSourcePath } from './source-path.ts'; * The effective slugs a project declares per artifact type, after combining the scope chain. `rulebooks`, `skills`, * and `subagents` are deployable; `collections` are dependency-only aggregates the caller expands into the others. * `sources` are the declared content sources, each resolved to an absolute directory, in precedence order (highest - * first). + * first). `packages` are the declared package names in that same precedence order, left unresolved: locating one probes + * `node_modules`, which is filesystem work this parser deliberately leaves to its caller. `declinedPackages` are the + * names a tier dropped and no higher tier re-adopted, which distinguishes "declined" from "never mentioned". */ export interface ResolvedDeclaration { readonly rulebooks: ReadonlyArray; @@ -18,6 +20,8 @@ export interface ResolvedDeclaration { readonly subagents: ReadonlyArray; readonly collections: ReadonlyArray; readonly sources: ReadonlyArray<{ name: string; dir: string }>; + readonly packages: ReadonlyArray; + readonly declinedPackages: ReadonlyArray; } /** @@ -44,6 +48,8 @@ export async function resolveDeclaration(options: { cwd: string }): Promise(); const subagents = new Set(); const collections = new Set(); + const packages = new Set(); + const declinedPackages = new Set(); // Sources key on `name` so a repeated name remaps its path; the value is the resolved absolute dir. const sources = new Map(); for (const filePath of chain) { @@ -54,12 +60,15 @@ export async function resolveDeclaration(options: { cwd: string }): Promise ({ name, dir })), }; } // region | Helpers +/** + * Applies one `packages` block's `use` and `drop` entries to the adopted and declined accumulators, keeping them + * disjoint: adopting a name clears any earlier decline, and declining one removes it from the adopted set. Tracking + * declines is what separates a package a project turned down from one it has never mentioned. + */ +function accumulatePackages(adopted: Set, declined: Set, block: TypeDeclaration | undefined): void { + const adoptions = block?.use ?? []; + const declines = block?.drop ?? []; + for (const entry of adoptions) { + // Re-inserting after a delete moves a repeated name to the end, so its latest declaration sets its precedence. + adopted.delete(entry.name); + adopted.add(entry.name); + declined.delete(entry.name); + } + for (const entry of declines) { + adopted.delete(entry.name); + declined.add(entry.name); + } +} + /** * Resolves each declared source's `path` against `fileDir` and accumulates it by `name`. Re-inserting after a delete * moves a repeated name to the end of the map, so a later (higher-precedence) declaration wins both the path and the @@ -93,10 +125,12 @@ function accumulateSources( /** Applies one type's `use` (add) and `drop` (subtract) entries to its accumulator, in declaration order. */ function accumulateType(effective: Set, block: TypeDeclaration | undefined): void { - for (const entry of block?.use ?? []) { + const additions = block?.use ?? []; + const subtractions = block?.drop ?? []; + for (const entry of additions) { effective.add(entry.name); } - for (const entry of block?.drop ?? []) { + for (const entry of subtractions) { effective.delete(entry.name); } } diff --git a/packages/agents/src/lib/codeassembly-schema.ts b/packages/agents/src/lib/codeassembly-schema.ts index a83f0b69..27a0ab26 100644 --- a/packages/agents/src/lib/codeassembly-schema.ts +++ b/packages/agents/src/lib/codeassembly-schema.ts @@ -15,14 +15,17 @@ export const SourceSchema = z.object({ name: z.string().min(1), path: z.string() /** * Schema for a single grouped `codeassembly.yaml` declaration: a top-level `root` flag, an optional `sources` list, - * plus one optional block per artifact type (`rulebooks`, `skills`, `subagents`, `collections`). The top level is - * closed (an unrecognized key triggers an error); entries are open (unknown keys pass through). Each type's block - * resolves to `{ use, drop }` lists; an absent or null block is omitted. + * an optional `packages` block naming installed packages that ship content, plus one optional block per artifact type + * (`rulebooks`, `skills`, `subagents`, `collections`). The top level is closed (an unrecognized key triggers an + * error); entries are open (unknown keys pass through). Each type's block resolves to `{ use, drop }` lists; an absent + * or null block is omitted. `packages` reuses that same block shape, so `use`, `drop`, and `root` apply to a package + * name exactly as they do to an artifact slug. */ const CodeAssemblySchema = z .object({ root: z.boolean().default(false), sources: optionalSourceList(), + packages: optionalTypeDeclaration(), rulebooks: optionalTypeDeclaration(), skills: optionalTypeDeclaration(), subagents: optionalTypeDeclaration(), @@ -55,7 +58,7 @@ export function parseCodeAssemblyFile(raw: string, sourceLabel?: string): CodeAs parsed = parseYaml(raw); } catch (error: unknown) { const message = error instanceof Error ? error.message : String(error); - throw new Error(`Invalid codeassembly.yaml${where}: malformed YAML — ${message}`); + throw new Error(`Invalid codeassembly.yaml${where}: malformed YAML — ${message}`, { cause: error }); } // An empty or comment-only document parses to nullish; treat it as "nothing declared". diff --git a/packages/agents/src/lib/package-sources.ts b/packages/agents/src/lib/package-sources.ts new file mode 100644 index 00000000..96b50316 --- /dev/null +++ b/packages/agents/src/lib/package-sources.ts @@ -0,0 +1,229 @@ +import { readFile } from 'node:fs/promises'; +import { createRequire } from 'node:module'; +import path from 'node:path'; + +import { z } from 'zod'; + +import { isMissingFile } from './type-guards.ts'; + +/** + * The only part of a dependency's `package.json` that CodeAssembly reads: the content directory it declares. Both + * levels are `.loose()` so every other field passes through and a later cut can add per-package config without a + * breaking change. + */ +const PackageManifestSchema = z + .object({ + codeassembly: z + .object({ content: z.string().min(1) }) + .loose() + .optional(), + }) + .loose(); + +/** The dependency fields of the consuming project's own `package.json`, read for the advisory scan alone. */ +const ProjectManifestSchema = z + .object({ + dependencies: z.record(z.string(), z.string()).optional(), + devDependencies: z.record(z.string(), z.string()).optional(), + }) + .loose(); + +/** A declared package resolved to the content directory it ships, named by the package it came from. */ +export interface PackageSource { + readonly name: string; + readonly dir: string; +} + +/** + * Reports the direct dependencies of the project at `baseDir` that ship CodeAssembly content and are absent from + * `addressed`, sorted by name, so a caller can name the declaration that would adopt each. `addressed` carries every + * name the project has spoken about, adopted and declined alike, so a package a project turned down stays quiet. + * Purely advisory: it contributes no source and cannot fail a run, so an + * unreadable or absent `package.json` yields nothing — which is also what lets the home domain share this path with no + * carve-out. Because a package must declare its content directory to ship any, detection reads that declaration and + * cannot report a false positive. + */ +export async function findUndeclaredGuidancePackages( + addressed: ReadonlyArray, + baseDir: string, +): Promise> { + try { + const addressedNames = new Set(addressed); + const candidates = (await readDirectDependencies(baseDir)).filter((name) => !addressedNames.has(name)); + const shipping = await Promise.all( + candidates.map(async (name) => ((await shipsGuidance(name, baseDir)) ? name : undefined)), + ); + return shipping.filter((name): name is string => name !== undefined).toSorted((a, b) => a.localeCompare(b)); + } catch { + // A suggestion is never worth failing a run for, so any surprise here yields no advice rather than an error. + return []; + } +} + +/** + * Resolves each declared package name to the content directory it ships, in declaration order, for use as a content + * source. Resolution walks the `node_modules` chain Node itself would search from `baseDir`, so it holds under pnpm's + * symlinked layout and under `workspace:*` links — which is what lets a producing repo consume its own guidance + * through the same declaration a third party writes. Throws when a declared name is a filesystem path rather than a + * package name, when a declared package is not installed, or when it declares no content directory; whether that + * directory exists is left to the caller's source validation, so a package source and a hand-declared one fail through + * one path. + */ +export async function resolvePackageSources( + names: ReadonlyArray, + baseDir: string, +): Promise> { + const resolved: Array = []; + for (const name of names) { + assertPackageName(name); + const installed = await findInstalledPackage(name, baseDir); + if (installed === undefined) { + throw new Error( + `Declared package "${name}" is not installed. Searched: ${listCandidateDirs(name, baseDir).join(', ')}.`, + ); + } + resolved.push({ name, dir: path.join(installed.dir, readContentPath(name, installed.manifest)) }); + } + return resolved; +} + +// region | Helpers + +/** + * Throws when `name` is a filesystem path rather than a package name. Node's resolver answers a relative specifier + * with the anchor directory itself, so `./guidance` would otherwise resolve to `/guidance` and `../sibling` + * would escape `baseDir` entirely — a second, undocumented path-source route beside `sources`. + */ +function assertPackageName(name: string): void { + if (name.startsWith('.') || path.isAbsolute(name)) { + throw new Error( + `Declared package "${name}" is a filesystem path, not a package name. Point at a directory with a \`sources\` entry instead.`, + ); + } +} + +/** + * Locates the installed directory of `name`, with its parsed `package.json`, by probing each candidate directory + * Node's resolver would search. Probing the filesystem rather than resolving a package subpath is deliberate: a + * modern `exports` map does not expose `./package.json`, so `require.resolve` cannot reach it, and a guidance-only + * package has no importable entry to resolve instead. + */ +async function findInstalledPackage( + name: string, + baseDir: string, +): Promise<{ dir: string; manifest: unknown } | undefined> { + for (const dir of listCandidateDirs(name, baseDir)) { + const raw = await readFileIfPresent(path.join(dir, 'package.json')); + if (raw !== undefined) { + return { dir, manifest: parsePackageManifest(name, raw) }; + } + } + return; +} + +/** Lists the candidate installed directories for `name`, in the order Node's resolver searches them from `baseDir`. */ +function listCandidateDirs(name: string, baseDir: string): ReadonlyArray { + // `createRequire` needs only a path to anchor resolution; the file itself need not exist. + const requireFromBase = createRequire(path.join(baseDir, 'package.json')); + // `resolve.paths` returns null for a core module, which a garbage declaration can produce; an empty candidate list + // reports it as not installed. + return (requireFromBase.resolve.paths(name) ?? []).map((nodeModules) => path.join(nodeModules, name)); +} + +/** Parses JSON, resolving to `undefined` rather than throwing, for the advisory scan that must not fail a run. */ +function parseJsonOrUndefined(raw: string): unknown { + try { + return JSON.parse(raw); + } catch { + return undefined; + } +} + +/** Parses a package's `package.json` text, naming the package so a syntax error is attributable. */ +function parsePackageManifest(name: string, raw: string): unknown { + try { + return JSON.parse(raw); + } catch (error: unknown) { + const message = error instanceof Error ? error.message : String(error); + throw new Error(`Package "${name}" has an unreadable package.json: ${message}`, { cause: error }); + } +} + +/** + * Reads the content directory a package declares under `codeassembly.content`. The key is required and has no default: + * a default location would claim a directory name in every producer's package root, so a producer states where its + * content lives and can nest it under a directory it already owns. Throws when the key is malformed or absent, naming + * the package either way. + */ +function readContentPath(name: string, manifest: unknown): string { + const result = PackageManifestSchema.safeParse(manifest); + if (!result.success) { + const detail = result.error.issues + .map((issue) => `${issue.path.join('.') || '(root)'}: ${issue.message}`) + .join('; '); + throw new Error(`Package "${name}" declares an invalid "codeassembly" key: ${detail}.`); + } + + const content = result.data.codeassembly?.content; + if (content === undefined) { + throw new Error( + `Package "${name}" declares no CodeAssembly content. A package that ships content sets "codeassembly": { "content": "" } in its package.json, and includes that directory in its published "files".`, + ); + } + return content; +} + +/** + * Reads the direct dependency names declared by the project at `baseDir`, or nothing when it has no readable manifest. + * Direct dependencies only: guidance is something a project opts into by depending on the package that ships it, and + * pnpm's strict layout would not surface a transitive package at the probed paths anyway. + */ +async function readDirectDependencies(baseDir: string): Promise> { + const raw = await readFileIfPresent(path.join(baseDir, 'package.json')); + if (raw === undefined) { + return []; + } + + const result = ProjectManifestSchema.safeParse(parseJsonOrUndefined(raw)); + if (!result.success) { + return []; + } + return [...Object.keys(result.data.dependencies ?? {}), ...Object.keys(result.data.devDependencies ?? {})]; +} + +/** + * Reads `filePath`, resolving to `undefined` when it is absent. Any other failure — e.g. `EACCES` on an unreadable + * `node_modules` directory — rethrows, so a permission problem surfaces instead of reading as a bare absence and + * sending resolution on to the next candidate. + */ +async function readFileIfPresent(filePath: string): Promise { + try { + return await readFile(filePath, 'utf8'); + } catch (error: unknown) { + if (isMissingFile(error)) { + return undefined; + } + throw error; + } +} + +/** + * Reports whether `name` resolves to an installed package that declares a content directory. Advisory: a package that + * is absent, or whose `package.json` will not parse, answers `false` rather than throwing, because the scan consuming + * this must never be the thing that fails a run. + */ +async function shipsGuidance(name: string, baseDir: string): Promise { + try { + const installed = await findInstalledPackage(name, baseDir); + if (installed === undefined) { + return false; + } + + const result = PackageManifestSchema.safeParse(installed.manifest); + return result.success && result.data.codeassembly?.content !== undefined; + } catch { + return false; + } +} + +// endregion | Helpers