Skip to content

docs(skills): seller-skill behavioral coverage gaps — 5 patterns from matrix v4 (#1120)#1122

Merged
bokelley merged 5 commits into
mainfrom
claude/issue-1120-seller-skill-behavioral-coverage
May 1, 2026
Merged

docs(skills): seller-skill behavioral coverage gaps — 5 patterns from matrix v4 (#1120)#1122
bokelley merged 5 commits into
mainfrom
claude/issue-1120-seller-skill-behavioral-coverage

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Closes #1120

Docs-only changes to skills/build-seller-agent/SKILL.md and skills/build-seller-agent/specialisms/sales-guaranteed.md. No changeset needed — skills/ markdown is excluded from the changeset requirement per CLAUDE.md.

Matrix v4 surfaced 5 behavioral patterns where LLM-built sales_guaranteed agents failed storyboard scenarios despite framework-correct scaffolding. This PR adds the missing teaching signal in the skill files:

  • Pattern 1 (sync_accounts omitted): Added a sales-guaranteed minimum-tool-surface callout immediately after the tools table. The required surface is 10 tools (note: list_creatives is optional — only required if the seller hosts its own creative library; omitted from the mandatory list per SalesPlatform interface classification).
  • Patterns 2+5 (TERMS_REJECTED, PACKAGE_NOT_FOUND missing): Added an error-code matrix covering all spec-defined rejections on create_media_buy and update_media_buy. Fixed a contradictory comment at the old line 523 that said "not adcpError" when adcpError is the correct mechanism throughout.
  • Pattern 3 (pending_creatives never exits): Rewrote the updateMediaBuy example to advance pending_creatives → pending_start (future start_time) or active (past/now) when creative_assignments arrive. Added key-point Fix Fly.io deployment host/port configuration and add regression tests #8 to the key-points list.
  • Pattern 4 (property_list/collection_list field path wrong): Fixed sales-guaranteed.md's example from pkg.property_list (no such field on PackageRequest) to pkg.targeting_overlay (correct per generated types — TargetingOverlay contains property_list/collection_list; neither Package nor PackageRequest has them as direct fields). Added a storyboard-discrepancy note: the inventory_list_targeting grader may check packages[].property_list (flat) rather than the spec-correct packages[].targeting_overlay.property_list; the upstream storyboard fix belongs at adcontextprotocol/adcp.

Nits surfaced (not fixed — see pre-PR review):

  • The minimum-tool callout is partly redundant with the table above it; kept for LLM-scanning salience.
  • The storyboard-discrepancy note's exact grader path is unconfirmed against the storyboard YAML; "may check" hedging added.

What was tested

  • npm run format:check — ✅ all files pass Prettier
  • npm run typecheck — pre-existing TS2688/TS5107 errors on main confirmed prior to this branch; no new errors introduced
  • npx tsx scripts/typecheck-skill-examples.ts — ✅ no new errors (229 known baselined, 33 blocks compiled)
  • Pre-push validation hook — ✅ passed (~6s)

Pre-PR review

  • code-reviewer: approved — paused-guard fix correct (patch.paused === false), list_creatives removal correct per SalesPlatform interface, BUDGET_TOO_LOW addition correct, storyboard-path nit addressed in note
  • ad-tech-protocol-expert: approved — TERMS_REJECTED bundling of measurement_terms+performance_standards confirmed in docs/llms.txt line 913; BUDGET_TOO_LOW vs INVALID_REQUEST distinction confirmed per error taxonomy; targeting_overlay.property_list nesting confirmed in generated types; pending_creatives trigger via update_media_buy aligns with storyboard flow at docs/llms.txt line 916–917

Triage-managed PR. This bot does not currently iterate on
review comments or PR conversation threads (only on the source
issue). To unblock:

  • Push fixup commits directly: gh pr checkout <num>
    fix → push.
  • Or re-trigger: comment /triage execute on the source
    issue.

See adcp#3121
for context.

Session: https://claude.ai/code/session_01LYWfxv9GxrAwXKYVZ1cXGJ


Generated by Claude Code

claude added 2 commits April 30, 2026 21:03
…1120)

Five patterns surfaced by matrix v4 where LLM-built sales_guaranteed agents
failed storyboard scenarios. All fixes are to skills/build-seller-agent/ only
(docs-only, no changeset needed).

- Pattern 1: add minimum-tool-surface callout after the tools table so LLMs
  can't omit sync_accounts and cascade-skip 3 storyboard scenarios
- Patterns 2+5: add error-code matrix for create_media_buy / update_media_buy —
  TERMS_REJECTED (unacceptable performance_standards / measurement_terms),
  PACKAGE_NOT_FOUND (bogus package_id on update); fix misleading prose at
  line 523 that said "not adcpError" when adcpError is correct
- Pattern 3: rewrite updateMediaBuy example to advance pending_creatives →
  pending_start (start_time future) or active (start_time now/past) when
  creative_assignments arrive; add key-point #8
- Pattern 4: fix property_list / collection_list field path from
  pkg.property_list (wrong — no such field on PackageRequest) to
  pkg.targeting_overlay.property_list (correct per generated types); add
  storyboard-discrepancy note for adcontextprotocol/adcp upstream fix

https://claude.ai/code/session_01LYWfxv9GxrAwXKYVZ1cXGJ
…guard, list_creatives, BUDGET_TOO_LOW

- sales-guaranteed (not sales_guaranteed) in the minimum-tool callout — spec
  uses kebab-case specialism IDs per CLAUDE.md
- remove list_creatives from required surface (it is optional per SalesPlatform
  interface; only required when seller hosts its own creative library)
- add BUDGET_TOO_LOW as distinct row in error matrix (below-floor budget maps
  to BUDGET_TOO_LOW, not INVALID_REQUEST per error taxonomy)
- fix paused → active guard to patch.paused === false (absent paused field
  means no state change, not resume)
- fix status cast from concrete union to string (covers all reachable states)
- tighten storyboard-discrepancy note path to packages[].property_list

https://claude.ai/code/session_01LYWfxv9GxrAwXKYVZ1cXGJ
bokelley and others added 3 commits April 30, 2026 21:24
Skills bundle with the npm package (files: ["skills/**/*"]), so the
behavioral-coverage updates need a published patch release. Documents
the four concrete deltas: minimum tool surface, error-code matrix,
update_media_buy state machine, targeting_overlay nesting.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Addressed parallel expert review on PR #1122:

- Error-code matrix: added PRODUCT_UNAVAILABLE row (canonical sold-out /
  no-longer-available rejection per the spec — `schemas/cache/3.0.0/
  enums/error-code.json`). Product expert flagged it missing from the
  guaranteed-rejection paths.
- updateMediaBuy state-machine: typed the `status` local + return cast
  as `MediaBuyStatus` instead of `string`. Imported `MediaBuyStatus`
  from `@adcp/sdk`. Code reviewer flagged `as string` as type-loose.
- sales-guaranteed.md: dropped the "may check the flat path / storyboard
  is the bug" footnote. Without verified grader logic the hedge reads as
  a permission slip ("ship broken code, blame the storyboard"). Skill
  now teaches the spec-correct nested path only — if the grader actually
  checks a wrong path, that surfaces organically and we file an upstream
  issue then with evidence.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley marked this pull request as ready for review May 1, 2026 01:39
@bokelley bokelley merged commit 3770916 into main May 1, 2026
10 checks passed
@bokelley bokelley deleted the claude/issue-1120-seller-skill-behavioral-coverage branch May 1, 2026 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

v6.1: seller-skill behavioral coverage gaps surfaced by matrix v4

2 participants