feat(validation): schema-driven validation with client hooks and opt-in server middleware

.changeset/schema-driven-validation.md

@@ -0,0 +1,34 @@
+---
+'@adcp/client': minor
+---
+
+Add schema-driven validation against the bundled AdCP JSON schemas on both
+the client and the server (closes adcp-client#688).
+
+**Client hooks** (on the `AdcpClient` / `SingleAgentClient` `validation`
+config, applied automatically via `TaskExecutor`):
+
+- `validation.requests: 'strict' | 'warn' | 'off'` — validate outgoing
+  payloads before dispatch. `strict` throws `ValidationError`
+  (`code: 'VALIDATION_ERROR'`) with a JSON Pointer to the offending field;
+  `warn` logs to debug logs and continues. Default: `warn`.
+- `validation.responses: 'strict' | 'warn' | 'off'` — validate incoming
+  payloads on receive. `strict` fails the task; `warn` logs and continues.
+  Default: strict in dev/test, warn in production. Overrides the legacy
+  `strictSchemaValidation` flag when set.
+
+**Server middleware** (opt-in on `createAdcpServer`'s `validation` config):
+
+- `validation.requests: 'strict'` — dispatcher returns
+  `adcpError('VALIDATION_ERROR', …)` before the handler runs.
+- `validation.responses: 'strict'` — handler-returned drift surfaces as a
+  `VALIDATION_ERROR` envelope; `warn` logs to the configured logger and
+  returns the response unchanged.
+
+Validation uses the bundled JSON schemas shipped at
+`dist/lib/schemas-data/<adcp_version>/` — async response variants
+(`-submitted`, `-working`, `-input-required`) are selected by payload shape
+(`status` field), matching issue #688's spec. `additionalProperties` is
+left permissive so vendor extensions don't trip the validator. The
+`VALIDATION_ERROR` envelope carries the full issue list (pointer, message,
+keyword, schema path) under `details.issues` for programmatic indexing.

docs/guides/BUILD-AN-AGENT.md

@@ -201,6 +201,26 @@ Scoping is per-principal — `resolveSessionKey` doubles as the idempotency prin
 
 **Only successful responses are cached.** Handler errors re-execute on retry rather than replaying — so a transient 5xx doesn't lock a failure into the cache.
 
+### Schema-Driven Validation (opt-in)
+
+`createAdcpServer` can validate every inbound request and handler response against the bundled AdCP JSON schemas for the SDK's declared version. Catches field-name drift (e.g. a handler emits `targeting_overlay` where the spec expects `targeting`) before the response leaves your agent.
+
+```typescript
+createAdcpServer({
+  name: 'my-seller',
+  version: '1.0.0',
+  validation: {
+    requests: 'strict',   // reject malformed requests with VALIDATION_ERROR
+    responses: 'warn',    // log handler drift, return response unchanged
+  },
+  mediaBuy: { /* … */ },
+});
+```
+
+Modes per side: `'strict' | 'warn' | 'off'`. Default is `'off'` — enable explicitly. `VALIDATION_ERROR` envelopes carry the full issue list (pointer, message, keyword, schema path) under `details.issues` so buyers can surface each offending field.
+
+The same validator runs on the `AdcpClient` side — storyboards and third-party clients configure it via `validation: { requests, responses }` on the client config. Request default is `warn` (so existing callers that send partial payloads still work); response default is `strict` in dev/test, `warn` in production. Set either side to `'off'` for zero overhead.
+
 ### createTaskCapableServer (Low-Level)
 
 For advanced cases where you need direct control over MCP tool registration, schema wiring, and response formatting. `createAdcpServer` uses this internally.

docs/llms.txt

@@ -643,7 +643,7 @@ Flow: `get_adcp_capabilities → sync_accounts → list_creative_formats → get
 Flow: `get_adcp_capabilities → get_products → create_media_buy → get_media_buys → list_creative_formats → sync_creatives → expect_webhook → get_media_buy_delivery`
 
 **Catalog-driven creative and conversion tracking** — Seller that renders dynamic ads from product catalogs, tracks conversions, and optimizes delivery based on performance feedback.
-Flow: `get_adcp_capabilities → sync_accounts → list_creative_formats → sync_catalogs → get_products → create_media_buy → sync_event_sources → log_event → provide_performance_feedback → get_media_buy_delivery`
+Flow: `get_adcp_capabilities → sync_accounts → list_creative_formats → sync_catalogs → build_creative → expect_substitution_safe → get_products → create_media_buy → sync_event_sources → log_event → provide_performance_feedback → get_media_buy_delivery`
 
 **Guaranteed media buy with human IO approval** — Seller agent that requires human-in-the-loop IO signing before guaranteed media buys go live.
 Flow: `get_adcp_capabilities → sync_accounts → get_products → create_media_buy → get_media_buys → sync_creatives → get_media_buy_delivery`
@@ -684,7 +684,7 @@ Flow: `get_adcp_capabilities → si_get_offering → si_initiate_session → si_
 Flow: `get_adcp_capabilities → list_accounts → sync_audiences`
 
 **Social platform** — Social media platform that accepts audience segments, native creatives, and conversion events from buyer agents.
-Flow: `get_adcp_capabilities → sync_accounts → list_accounts → sync_audiences → sync_creatives → log_event → get_account_financials`
+Flow: `get_adcp_capabilities → sync_accounts → list_accounts → sync_audiences → sync_creatives → sync_catalogs → sync_creatives → log_event → get_account_financials`
 
 ### Core
 
@@ -722,7 +722,7 @@ Flow: `get_adcp_capabilities → create_property_list → list_property_lists 
 ### 
 
 **Generative creative agent** — Agent that takes a brief and generates finished creatives from scratch — no input assets required.
-Flow: `get_adcp_capabilities → list_creative_formats → build_creative`
+Flow: `get_adcp_capabilities → list_creative_formats → build_creative → sync_catalogs → build_creative`
 
 **Runner output contract** — Required failure-detail shape that AdCP storyboard runners MUST emit so implementors can self-diagnose validation failures.
 
@@ -731,6 +731,7 @@ Flow: `get_adcp_capabilities → list_creative_formats → build_creative`
 **Programmatic exchange sales** — Programmatic exchange or SSP — auction-based inventory access with real-time bidding semantics and multi-seller inventory pooling.
 
 **Retail media sales** — Retail media network seller — onsite, offsite, and in-store inventory tied to commerce signals and SKU-level reporting.
+Flow: `sync_catalogs`
 
 **Streaming TV sales** — Connected TV and streaming sellers — ad-supported video inventory, audience-based buying, and frequency management across CTV apps.
 

package.json

@@ -126,7 +126,7 @@
     "prepare": "node scripts/install-hooks.js 2>/dev/null || true",
     "prepublishOnly": "npm run clean && npm run sync-schemas && (test -f src/lib/types/tools.generated.ts || npm run generate-types) && npm run build:lib && node --test-timeout=60000 --test test/lib/adcp-client.test.js test/lib/validation.test.js test/lib/zod-schemas.test.js",
     "build": "npm run build:lib",
-    "build:lib": "npm run sync-version && tsc --project tsconfig.lib.json",
+    "build:lib": "npm run sync-version && tsc --project tsconfig.lib.json && tsx scripts/copy-schemas-to-dist.ts",
     "build:test-agents": "npm run build:lib && tsc -p test-agents/tsconfig.json --rootDir test-agents",
     "test": "node --test-timeout=60000 --test-force-exit --test test/*.test.js test/lib/*.test.js",
     "test:lib": "node --test-timeout=60000 --test-force-exit --test test/lib/*.test.js",

scripts/copy-schemas-to-dist.ts

@@ -0,0 +1,61 @@
+#!/usr/bin/env tsx
+/**
+ * Copy the bundled + core JSON schemas into the built package so the
+ * runtime validator (src/lib/validation/schema-loader.ts) can read them
+ * without a dependency on the source tree.
+ *
+ * Source: schemas/cache/<ver>/{bundled,core,<domain>}/
+ * Dest:   dist/lib/schemas-data/<ver>/{bundled,core,<domain>}/
+ *
+ * Invoked by the `build:lib` npm script after tsc emits JS.
+ */
+
+import { cpSync, existsSync, readFileSync, mkdirSync } from 'fs';
+import path from 'path';
+
+function main(): void {
+  const repoRoot = path.resolve(__dirname, '..');
+  const versionFile = path.join(repoRoot, 'ADCP_VERSION');
+  if (!existsSync(versionFile)) {
+    throw new Error(`ADCP_VERSION file not found at ${versionFile}`);
+  }
+  const version = readFileSync(versionFile, 'utf-8').trim();
+  if (!version) throw new Error('ADCP_VERSION file is empty.');
+
+  const srcRoot = path.join(repoRoot, 'schemas', 'cache', version);
+  if (!existsSync(srcRoot)) {
+    // The schema cache is fetched by `sync-schemas`. CI jobs that don't run
+    // the full toolchain (e.g., the code-quality integrity check that does
+    // `npm clean && build:lib` without a prior sync-schemas) would otherwise
+    // break here. Skip quietly — the loader falls back to the same source
+    // path, and any job that actually needs the schemas at runtime will get
+    // a clear error at first use.
+    console.warn(
+      `[copy-schemas-to-dist] schema cache missing at ${srcRoot}; skipping. ` +
+        `Run \`npm run sync-schemas\` to populate it before shipping.`
+    );
+    return;
+  }
+
+  const destRoot = path.join(repoRoot, 'dist', 'lib', 'schemas-data', version);
+  mkdirSync(destRoot, { recursive: true });
+
+  // Copy bundled/ and every per-domain directory except registry extras.
+  // The async response variants live in the flat per-domain tree; we need
+  // them plus bundled/ (sync) and core/ (ref targets for async).
+  cpSync(srcRoot, destRoot, {
+    recursive: true,
+    // Skip heavy subtrees that the runtime validator doesn't read.
+    filter: src => {
+      const rel = path.relative(srcRoot, src);
+      if (!rel) return true;
+      const top = rel.split(path.sep)[0];
+      if (top === 'tmp' || top === 'compliance') return false;
+      return true;
+    },
+  });
+
+  console.log(`[copy-schemas-to-dist] copied ${srcRoot} → ${destRoot}`);
+}
+
+main();

src/lib/core/SingleAgentClient.ts

@@ -254,10 +254,33 @@ export interface SingleAgentClientConfig extends ConversationConfig {
    */
   validation?: {
     /**
-     * Fail tasks when response schema validation fails (default: true)
+     * Validate outgoing requests against the bundled AdCP JSON schema before
+     * dispatch. Catches field-name drift at call-time instead of at
+     * storyboard-time.
      *
-     * When true: Invalid responses cause task to fail with error
-     * When false: Schema violations are logged but task continues
+     * - `strict`: throw `ValidationError` with a JSON Pointer to the bad field
+     * - `warn`: log to debug logs and continue
+     * - `off`: skip the validator entirely (no overhead)
+     *
+     * @default `strict` in dev/test, `warn` in production
+     */
+    requests?: import('../validation/client-hooks').ValidationMode;
+    /**
+     * Validate incoming responses against the bundled AdCP JSON schema.
+     *
+     * - `strict`: fail the task with `VALIDATION_ERROR`
+     * - `warn`: log to debug logs and surface the task as successful
+     * - `off`: skip the validator entirely
+     *
+     * Overrides `strictSchemaValidation` when set.
+     *
+     * @default `strict` in dev/test, `warn` in production
+     */
+    responses?: import('../validation/client-hooks').ValidationMode;
+    /**
+     * Legacy: fail tasks when response schema validation fails (default: true).
+     * Superseded by `responses` above — retained for backward compat.
+     * `false` maps to `responses: 'warn'` when `responses` isn't set.
      *
      * @default true
      */
@@ -336,6 +359,10 @@ export class SingleAgentClient {
       strictSchemaValidation: config.validation?.strictSchemaValidation !== false, // Default: true
       logSchemaViolations: config.validation?.logSchemaViolations !== false, // Default: true
       filterInvalidProducts: config.validation?.filterInvalidProducts === true, // Default: false
+      validation: {
+        ...(config.validation?.requests != null && { requests: config.validation.requests }),
+        ...(config.validation?.responses != null && { responses: config.validation.responses }),
+      },
       onActivity: config.onActivity,
       governance: config.governance,
     });

src/lib/core/TaskExecutor.ts

@@ -9,7 +9,14 @@ import { getAuthToken } from '../auth';
 import { is401Error, adcpErrorToTypedError } from '../errors';
 import type { ADCPError } from '../errors';
 import type { Storage } from '../storage/interfaces';
-import { responseValidator } from './ResponseValidator';
+import {
+  validateOutgoingRequest,
+  validateIncomingResponse,
+  resolveValidationModes,
+  type ValidationHookConfig,
+  type ValidationMode,
+} from '../validation/client-hooks';
+import { formatIssues } from '../validation/schema-validator';
 import { unwrapProtocolResponse, isAdcpError } from '../utils/response-unwrapper';
 import { extractAdcpErrorInfo, extractCorrelationId } from '../utils/error-extraction';
 import { generateIdempotencyKey, isMutatingTask, redactIdempotencyKeyInArgs } from '../utils/idempotency';
@@ -110,6 +117,8 @@ export class TaskExecutor {
   private conversationStorage?: Map<string, Message[]>;
   private governanceMiddleware?: GovernanceMiddleware;
   private lastKnownServerVersion?: 'v2' | 'v3';
+  private requestValidationMode!: ValidationMode;
+  private responseValidationMode!: ValidationMode;
 
   constructor(
     private config: {
@@ -135,6 +144,13 @@ export class TaskExecutor {
       strictSchemaValidation?: boolean;
       /** Log all schema validation violations to debug logs (default: true) */
       logSchemaViolations?: boolean;
+      /**
+       * Schema-driven validation using the bundled AdCP JSON schemas.
+       * Controls outgoing request and incoming response checks independently.
+       * Defaults: strict in dev/test, warn in prod. Set a mode to `off` to
+       * skip the validator entirely on that side (zero overhead).
+       */
+      validation?: ValidationHookConfig;
       /** Filter out invalid products from get_products responses instead of rejecting the entire response (default: false) */
       filterInvalidProducts?: boolean;
       /** Global activity callback for observability */
@@ -150,6 +166,18 @@ export class TaskExecutor {
     if (config.governance) {
       this.governanceMiddleware = new GovernanceMiddleware(config.governance, config.onActivity);
     }
+    const modes = resolveValidationModes(config.validation);
+    this.requestValidationMode = modes.requests;
+    // Legacy `strictSchemaValidation: false` flips response-side enforcement
+    // to warn mode — preserve that behaviour unless `validation.responses`
+    // was set explicitly.
+    if (config.validation?.responses !== undefined) {
+      this.responseValidationMode = config.validation.responses;
+    } else if (config.strictSchemaValidation === false) {
+      this.responseValidationMode = 'warn';
+    } else {
+      this.responseValidationMode = modes.responses;
+    }
   }
 
   /**
@@ -342,6 +370,9 @@ export class TaskExecutor {
         metadata: { toolName: taskName, type: 'request' },
       };
 
+      // Pre-send schema check — throws in strict, logs in warn, skips in off.
+      validateOutgoingRequest(taskName, effectiveParams, this.requestValidationMode, debugLogs);
+
       // Send initial request and get streaming response with webhook URL
       const response = await ProtocolClient.callTool(
         agent,
@@ -1485,77 +1516,52 @@ export class TaskExecutor {
   }
 
   /**
-   * Validate response against AdCP schema and log any violations
-   *
-   * Respects config.strictSchemaValidation (default: true):
-   * - true: Validation failures cause task to fail
-   * - false: Validation failures are logged only
+   * Validate an incoming response against the bundled AdCP JSON schema for
+   * `taskName`. Strict mode fails the task; warn mode logs and returns
+   * valid so the caller can continue. Off mode short-circuits before
+   * AJV runs.
    */
   private validateResponseSchema(
     response: any,
     taskName: string,
     debugLogs: any[]
   ): { valid: boolean; errors: string[] } {
-    const strictMode = this.config.strictSchemaValidation !== false; // Default: true
-    const logViolations = this.config.logSchemaViolations !== false; // Default: true
+    const mode = this.responseValidationMode;
+    const logViolations = this.config.logSchemaViolations !== false;
 
     try {
-      // Normalize response to v3 format before validation
-      // This ensures v2 server responses pass validation against v3 schemas
       const normalizedResponse = this.normalizeResponseForValidation(response, taskName);
+      const outcome = validateIncomingResponse(taskName, normalizedResponse, mode, debugLogs);
+      if (outcome.valid) return { valid: true, errors: [] };
+
+      const errorStrings = outcome.issues.map(i => `${i.pointer}: ${i.message}`);
+
+      if (logViolations) {
+        debugLogs.push({
+          timestamp: new Date().toISOString(),
+          type: mode === 'strict' ? 'error' : 'warning',
+          message: `Schema validation ${mode === 'strict' ? 'failed' : 'warning'} for ${taskName}: ${formatIssues(
+            outcome.issues
+          )}`,
+          errors: errorStrings,
+          schemaVariant: outcome.variant,
+          mode,
+        });
+      }
 
-      const validationResult = responseValidator.validate(normalizedResponse, taskName, {
-        validateSchema: true,
-        strict: false,
-      });
-
-      if (!validationResult.valid) {
-        // Log to debug logs if enabled
-        if (logViolations) {
-          const errorSummary = validationResult.errors.slice(0, 3).join('; ');
-          const moreErrors =
-            validationResult.errors.length > 3 ? ` (and ${validationResult.errors.length - 3} more)` : '';
-
-          debugLogs.push({
-            timestamp: new Date().toISOString(),
-            type: strictMode ? 'error' : 'warning',
-            message: `Schema validation ${
-              strictMode ? 'failed' : 'warning'
-            } for ${taskName}: ${errorSummary}${moreErrors}`,
-            errors: validationResult.errors,
-            schemaErrors: validationResult.schemaErrors,
-            strictMode,
-          });
-        }
-
-        // Console output based on strict mode
-        if (strictMode) {
-          console.error(`Schema validation failed for ${taskName}:`, validationResult.errors);
-        } else {
-          console.warn(`Schema validation failed for ${taskName} (non-blocking):`, validationResult.errors);
-        }
-
-        // In strict mode, validation failures are treated as invalid
-        if (strictMode) {
-          return {
-            valid: false,
-            errors: validationResult.errors,
-          };
-        }
+      if (mode === 'warn') {
+        console.warn(`Schema validation failed for ${taskName} (non-blocking):`, errorStrings);
+        return { valid: true, errors: [] };
       }
 
-      // Non-strict mode or validation passed
-      return {
-        valid: true,
-        errors: [],
-      };
+      console.error(`Schema validation failed for ${taskName}:`, errorStrings);
+      return { valid: false, errors: errorStrings };
     } catch (error) {
       const errorMessage = error instanceof Error ? error.message : 'unknown error';
       console.error('Error during schema validation:', errorMessage);
-      // On validation error, fail safe based on strict mode
       return {
-        valid: !strictMode, // In strict mode, treat validation errors as failures
-        errors: strictMode ? [`Validation error: ${errorMessage}`] : [],
+        valid: mode !== 'strict',
+        errors: mode === 'strict' ? [`Validation error: ${errorMessage}`] : [],
       };
     }
   }