feat: add optional reporting webhooks with metrics and aggregation support#81
Merged
Conversation
…pport Implements automated reporting data delivery via webhooks for media buys: **Reporting Capabilities** (on Products): - `available_reporting_frequencies`: hourly, daily, monthly options - `expected_delay_minutes`: delay before data is available (e.g., 240 for 4hrs) - `timezone`: reporting period boundaries (UTC or IANA timezone) - `supports_webhooks`: webhook support flag - `available_metrics`: metrics available beyond impressions/spend (clicks, conversions, etc.) **Reporting Webhook** (on create_media_buy): - Optional webhook configuration with URL, auth, and frequency - `requested_metrics`: buyer can request subset of available metrics - Publisher commits to (duration / frequency) + 1 notifications **Webhook Payload**: - `notification_type`: scheduled, final, or delayed - `sequence_number`: sequential notification tracking - `next_expected_at`: timestamp for next notification - `media_buy_deliveries`: ALWAYS an array (enables publisher aggregation) **Aggregation**: - Publishers SHOULD aggregate webhooks by URL/frequency/period - Reduces webhook volume for buyers with many campaigns - Buyers iterate array, calculate totals if needed - No `aggregated_totals` in webhooks (arbitrary grouping) All tests pass, docs updated, schema synchronized. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
bokelley
added a commit
that referenced
this pull request
Oct 4, 2025
Address Yahoo's production readiness feedback on PR #81 with enterprise-grade webhook infrastructure requirements. Security & Authentication: - HMAC-SHA256 signature verification with X-ADCP-Signature header - Replay attack prevention with timestamp validation - Constant-time comparison for timing attack protection - Secret rotation guidance Reliability & Circuit Breaking: - Exponential backoff with jitter for retries - Circuit breaker pattern (CLOSED/OPEN/HALF_OPEN states) - Bounded queues (1000 webhooks per endpoint) to prevent memory exhaustion - Per-endpoint circuit breakers with automatic recovery Privacy & Compliance: - PII scrubbing requirements for GDPR/CCPA - Explicit guidance on what to scrub vs. aggregate - Minimum aggregation thresholds to prevent re-identification - Publisher and buyer responsibilities documented Operational Monitoring: - Webhook health metrics in get_media_buy_delivery responses - Circuit breaker status visibility for debugging - Success/failure tracking and alerting guidance - Last attempt timestamps and failure reasons Data Integrity: - get_media_buy_delivery API defined as authoritative source of truth - Reconciliation process with discrepancy thresholds - Late-arriving impression handling (adjustments array and correction webhooks) - Attribution window guidance (7-90 days depending on ad type) Partial Failure Handling: - Best-effort delivery with per-campaign status indicators - partial_data flag and unavailable_count field - reporting_delayed status for temporarily unavailable data - Guidance on when to use partial vs. delayed notifications Timezone & DST: - Explicit handling of 23-hour and 25-hour DST transition days - IANA timezone identifier requirements - ISO 8601 with timezone offset requirements - Complete implementation examples for period calculation and billing - Common pitfalls and test date guidance Schema Updates: - Add notification_type: "correction" for late-arriving data - Add partial_data and unavailable_count fields - Add webhook_health object with circuit breaker status - Add reporting_period DST fields (duration_hours, is_dst_transition) - Add status: "reporting_delayed" for partial failures - Add adjustments array for incremental corrections This addresses all critical gaps identified by Yahoo for production deployment at scale, enabling Phase 1 rollout with proper security, monitoring, and operational safeguards. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
bokelley
added a commit
that referenced
this pull request
Oct 9, 2025
…#86) * feat: add comprehensive webhook security and reliability specifications Address Yahoo's production readiness feedback on PR #81 with enterprise-grade webhook infrastructure requirements. Security & Authentication: - HMAC-SHA256 signature verification with X-ADCP-Signature header - Replay attack prevention with timestamp validation - Constant-time comparison for timing attack protection - Secret rotation guidance Reliability & Circuit Breaking: - Exponential backoff with jitter for retries - Circuit breaker pattern (CLOSED/OPEN/HALF_OPEN states) - Bounded queues (1000 webhooks per endpoint) to prevent memory exhaustion - Per-endpoint circuit breakers with automatic recovery Privacy & Compliance: - PII scrubbing requirements for GDPR/CCPA - Explicit guidance on what to scrub vs. aggregate - Minimum aggregation thresholds to prevent re-identification - Publisher and buyer responsibilities documented Operational Monitoring: - Webhook health metrics in get_media_buy_delivery responses - Circuit breaker status visibility for debugging - Success/failure tracking and alerting guidance - Last attempt timestamps and failure reasons Data Integrity: - get_media_buy_delivery API defined as authoritative source of truth - Reconciliation process with discrepancy thresholds - Late-arriving impression handling (adjustments array and correction webhooks) - Attribution window guidance (7-90 days depending on ad type) Partial Failure Handling: - Best-effort delivery with per-campaign status indicators - partial_data flag and unavailable_count field - reporting_delayed status for temporarily unavailable data - Guidance on when to use partial vs. delayed notifications Timezone & DST: - Explicit handling of 23-hour and 25-hour DST transition days - IANA timezone identifier requirements - ISO 8601 with timezone offset requirements - Complete implementation examples for period calculation and billing - Common pitfalls and test date guidance Schema Updates: - Add notification_type: "correction" for late-arriving data - Add partial_data and unavailable_count fields - Add webhook_health object with circuit breaker status - Add reporting_period DST fields (duration_hours, is_dst_transition) - Add status: "reporting_delayed" for partial failures - Add adjustments array for incremental corrections This addresses all critical gaps identified by Yahoo for production deployment at scale, enabling Phase 1 rollout with proper security, monitoring, and operational safeguards. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: simplify webhook specifications based on implementation feedback Simplify and clarify webhook specifications based on Yahoo's feedback and team discussion. Focus on practical implementation guidance rather than exhaustive edge cases. Key Simplifications: 1. **Recommend UTC for Reporting** - Eliminates DST complexity for most use cases - Simplifies reconciliation across systems - Reduced DST documentation to brief "if you must use local time" section - Made DST payload fields optional (duration_hours, is_dst_transition) 2. **Clarify Hourly Webhooks Are Optional** - Publishers not required to support all frequencies - Recommend daily for Phase 1 deployments - Note cost considerations (hourly = 24x traffic) 3. **Add Offline Reporting Option** - Cloud storage bucket push for large buyer-seller pairs - 10-100x cost reduction vs. webhooks for high volume - Supports S3, GCS, Azure Blob Storage - Same payload format as webhooks (JSON Lines, CSV, Parquet) - Best for >100 campaigns or hourly reporting needs 4. **Move Reconciliation to General Reporting** - Not webhook-specific - applies to all delivery methods - Webhooks, offline files, and polling all need reconciliation - Late-arriving data affects all methods - API remains authoritative source of truth 5. **Clarify Webhook Correction Advantage** - Webhooks enable "overwrite this period" with notification_type: correction - Polling-only implementations must detect via reconciliation - Makes late-arriving impression handling easier for buyers These changes address the feedback: - "Is it possible / realistic to ask for all reporting in UTC?" → Yes, recommended - "Publishers don't have to support hourly" → Clarified as optional - "Offline reporting mechanism makes sense" → Added cloud storage option - "Reconciliation isn't a webhook issue" → Moved to general reporting section - "Webhooks could fire same range again with 'overwrite'" → Emphasized this advantage No schema changes needed - existing fields support both simplified (UTC, daily) and complex (local timezone, hourly) implementations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: simplify webhook specs to UTC-only with minimal adjustment model Further simplify webhook specifications based on team consensus: 1. **UTC-Only Reporting (MUST)** - Removed local timezone option entirely - All reporting periods use UTC (no DST complexity) - Daily: 00:00:00Z to 23:59:59Z (always 24 hours) - Removed duration_hours, is_dst_transition, dst_transition_type fields 2. **Simplified Late-Arrival Handling** - Single approach: Resend period with is_adjusted: true - Removed adjustments array (incremental model) - Removed is_correction, correction_reason fields - notification_type: "adjusted" (replaced "correction") - Buyer simply replaces entire period when is_adjusted flag present 3. **Webhook Health via Task Management** - Removed webhook_health object from delivery response - Webhook delivery status tracked through global task management - Consistent with other AdCP operations - No need for media-buy-specific health endpoints Schema Changes: - notification_type: "correction" → "adjusted" - Removed reporting_period.{duration_hours, is_dst_transition, dst_transition_type} - Removed media_buy_deliveries[].{adjustments, is_correction, correction_reason} - Added media_buy_deliveries[].is_adjusted (boolean) - Removed media_buy_deliveries[].webhook_health (use task management) Documentation simplified by ~200 lines while maintaining all critical security and reliability features (HMAC, circuit breakers, PII scrubbing, etc.). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: clarify webhook behavior for async operations ## Summary Documents webhook notification format and trigger behavior for asynchronous AdCP operations, addressing confusion about when webhooks are called and what payload format is sent. ## Changes ### Webhook Trigger Behavior - **Key rule**: Webhooks are ONLY called when initial response is `submitted` - `completed`/`failed` responses are synchronous - no webhook needed - `working` responses complete within ~120s - no webhook, just wait - `submitted` responses are long-running (hours/days) - webhooks used ### Webhook Payload Format - Webhooks POST the complete task response object (not just status) - Each webhook matches the task's response schema exactly - For `create_media_buy`: full create-media-buy-response.json structure - For `activate_signal`: full activate-signal-response.json structure ### Webhook URL Pattern - Use task name in URL: `/webhooks/adcp/{task_name}/{agent_id}/{op_id}` - Examples: `/create_media_buy/...`, `/activate_signal/...` - Enables clear routing based on task type ### Status Change Notifications For `submitted` operations, webhooks called on: - `input-required` → Human needs to respond/approve - `completed` → Operation finished successfully - `failed` → Operation failed with error details - `canceled` → Operation was canceled ## Files Modified - docs/protocols/core-concepts.md - Added comprehensive webhook section - docs/protocols/task-management.md - Updated webhook integration docs - docs/media-buy/task-reference/create_media_buy.md - Clarified webhook flow - docs/signals/tasks/activate_signal.md - Added webhook examples ## Examples Added - Human-in-the-loop approval flow with webhooks - Synchronous vs async operation handling - Complete webhook payload structures - Webhook handler implementation patterns 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: clarify webhook behavior for async operations Clarify that webhook security (HMAC-SHA256) is a core AdCP protocol requirement, not an optional implementation detail. **Key Change: Secure by Default** The problem with "recommended security": - Every integration becomes a negotiation - Different publishers implement different auth schemes - Default becomes "no security" because it's easier - Buyers must handle N different authentication methods - Result: Insecure by default The benefit of required security: - One standardized approach that always works - Publishers know exactly what to implement - Buyers know exactly what to expect - Secure by default across all implementations - Exception is "opt out of security" (conscious choice) **Schema Changes:** In `create-media-buy-request.json`: - Changed `auth_type` + `auth_token` → `secret` (required) - `secret` is shared HMAC-SHA256 key (min 32 chars) - Exchanged out-of-band during publisher onboarding - Used to generate X-ADCP-Signature header **Documentation Changes:** Added "Why Required" rationale to webhook security section: - Explains default-secure philosophy - Clarifies this is protocol requirement, not implementation guidance - Updates examples to use `secret` field instead of auth_type/token **Compliance:** Implementations without HMAC signature verification are non-compliant with AdCP specification. This applies to: - Reporting webhooks (media buy domain) - Task status webhooks (all domains) - Any future webhook functionality 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: standardize webhook security across all AdCP webhooks Apply required HMAC-SHA256 security to ALL webhook configurations in AdCP, not just reporting webhooks. **Universal Webhook Security Model:** All webhook configurations now use: - `webhook_secret` (required, min 32 chars) - NOT `webhook_auth` - `X-ADCP-Signature` header with HMAC-SHA256 signature - `X-ADCP-Timestamp` header for replay protection - Constant-time signature verification **Changes Apply To:** 1. **Protocol-level task webhooks** (MCP, A2A) - `webhook_url` + `webhook_secret` parameters - Used for task status notifications (submitted → completed) - Applies to: create_media_buy, update_media_buy, sync_creatives, etc. 2. **Domain-specific webhooks** (e.g., reporting webhooks) - `reporting_webhook.secret` field - Used for scheduled reporting delivery - Applies to: media buy reporting **Before (inconsistent):** ```javascript // MCP task webhook { webhook_url: "...", webhook_auth: { type: "bearer", ... } } // Reporting webhook { url: "...", auth_type: "bearer", auth_token: "..." } // A2A push notification { webhook_url: "...", auth: { type: "bearer", ... } } ``` **After (standardized):** ```javascript // ALL webhooks use same model { webhook_url: "...", webhook_secret: "min_32_chars..." } { url: "...", secret: "min_32_chars..." } { webhook_url: "...", secret: "min_32_chars..." } ``` **Documentation Updates:** - core-concepts.md: Protocol-level webhook examples - mcp-guide.md: MCP-specific task webhook examples - protocol-comparison.md: Cross-protocol webhook examples - task-management.md: Task webhook configuration - optimization-reporting.md: Reporting webhook examples **Why This Matters:** As Brian said: "If it's clearly laid out in the spec, do it this way, then okay, maybe once in a while people will make an exception, but there's no exception. The default would be secure and best practice." One security model across ALL AdCP webhooks = secure by default everywhere. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: align A2A webhook config with A2A protocol spec Clarify that A2A's push_notification_config structure is defined by the A2A protocol specification (not under our control), and document how AdCP's HMAC security requirement maps to A2A's native auth structure. **Key Insight:** A2A is an external protocol - we adapt to it, not the other way around. **A2A Protocol Structure (Fixed):** ```javascript push_notification_config: { webhook_url: "...", auth: { type: "bearer" | "basic" | "custom", data: { /* auth-specific data */ } } } ``` **AdCP's Approach:** Use A2A's `auth.type: "custom"` extensibility to pass HMAC secret: ```javascript push_notification_config: { webhook_url: "...", auth: { type: "custom", data: { hmac_secret: "..." } // AdCP security requirement } } ``` **Server Implementation:** Regardless of protocol (MCP or A2A), AdCP servers MUST: - Extract HMAC secret from protocol-specific location - Send X-ADCP-Signature and X-ADCP-Timestamp headers - Implement HMAC-SHA256 signature generation **Documentation Added:** New "Protocol Alignment" section explains: - MCP: AdCP adds webhook_secret parameter - A2A: AdCP maps to auth.type: "custom" - Both: Servers always send AdCP security headers This maintains AdCP's security requirements while respecting that A2A's structure is defined by an external specification we cannot change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: simplify webhook security to Bearer token authentication Replace complex HMAC-SHA256 signature verification with simple Bearer token authentication. This simplification is appropriate because: 1. We control the AdCP protocol specification 2. TLS already provides transport security 3. Bearer tokens are simpler to implement and debug 4. Uniform auth structure across all webhook types Changes: - Unified webhook_config structure with auth.type and auth.token - Replace X-ADCP-Signature header with Authorization: Bearer <token> - Standardize across protocol-level webhooks, reporting webhooks, and A2A - Update schema to require auth object instead of secret field - Update all documentation examples to use Bearer token pattern Net change: -123 lines (reduced complexity) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: correct A2A authentication structure to match protocol spec Fix A2A push_notification_config to use the correct authentication structure according to A2A v0.2.5 specification: - Change from `auth: { type, credentials }` to `authentication: { schemes, credentials }` - Use `schemes: ["Bearer"]` (array format per A2A spec) - Update all A2A examples in core-concepts.md and protocol-comparison.md This ensures AdCP correctly maps to A2A's PushNotificationAuthenticationInfo which uses: ```typescript interface PushNotificationAuthenticationInfo { schemes: string[]; // Array of authentication scheme names credentials?: string; // Optional credentials string } ``` Reference: https://a2a-protocol.org/v0.2.5/specification/#69-pushnotificationauthenticationinfo-object 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: adopt A2A PushNotificationConfig as universal webhook structure Replace custom webhook_config with A2A's PushNotificationConfig structure across all AdCP webhooks. This provides: 1. **Industry-standard structure**: Uses A2A v0.2.5 specification 2. **Zero protocol mapping**: Same structure works for MCP and A2A 3. **Multi-auth support**: schemes array handles Bearer, HMAC-SHA256, etc. 4. **Simpler implementation**: One webhook config format to learn Structure changes: - FROM: webhook_config { url, auth: { type, token } } - TO: push_notification_config { url, authentication: { schemes, credentials } } Supported authentication schemes: - Bearer: Simple token auth for development - HMAC-SHA256: Signature verification for production (prevents replay attacks) Benefits: - Consistent developer experience across protocols - Natural multi-auth support via schemes array - Future-proof for JWT, mTLS, or other auth types - Eliminates protocol-specific webhook configuration Reference: https://a2a-protocol.org/v0.2.5/specification/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * refactor: extract reusable push-notification-config schema Create shared core schema for webhook configuration that can be reused across any task-specific webhooks (reporting, creative sync, etc.). Changes: - New: static/schemas/v1/core/push-notification-config.json - Updated: create-media-buy-request.json uses allOf + $ref pattern - reporting_webhook extends base push_notification_config with reporting-specific fields (reporting_frequency, requested_metrics) Benefits: - Single source of truth for webhook authentication structure - Easy to add webhooks to other async tasks (sync_creatives, etc.) - Maintains A2A PushNotificationConfig compatibility - DRY principle for webhook configuration Note: Protocol-level push_notification_config (for task status notifications) is handled at MCP/A2A wrapper level, not in task schemas. This schema is for task-specific business webhooks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: add webhook support to async task requests Add optional push_notification_config to update_media_buy and sync_creatives task requests for task-specific async notifications. Schema changes: - update-media-buy-request.json: Add push_notification_config field - sync-creatives-request.json: Add push_notification_config field - Both reference core/push-notification-config.json for consistency Documentation updates: - update_media_buy.md: Add webhook configuration section - sync_creatives.md: Add webhook configuration section - Clarify when webhooks are triggered (working→completed, submitted→completed) Task-specific vs Protocol-level webhooks: - Task-specific: Passed in task params (this commit) - update_media_buy, sync_creatives, reporting_webhook - Notify about specific task completion - Protocol-level: Passed in MCP/A2A wrapper options - Generic task status notifications - Works for ALL async tasks Benefits: - Consistent webhook structure across all async tasks - Easy to add webhooks to future long-running operations - Clear documentation for when/why to use task-specific webhooks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * docs: clarify webhook triggers include HITL approval Update sync_creatives push_notification_config description to mention manual approval/HITL as a webhook trigger, not just large bulk operations. Webhooks are sent when: - Large bulk operations take longer than ~120 seconds - Manual approval required (Human-in-the-Loop workflows) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements automated reporting data delivery via webhooks for media buys, enabling publishers to push reporting data to buyers at configurable frequencies.
Key Features
Reporting Capabilities (on Products via
get_products):available_reporting_frequencies: hourly, daily, monthly optionsexpected_delay_minutes: expected delay before data becomes available (e.g., 240 for 4 hours)timezone: timezone for reporting period boundaries (UTC or IANA timezone)supports_webhooks: boolean flag indicating webhook supportavailable_metrics: array of available metrics beyond impressions/spend (clicks, conversions, video_completions, etc.)Reporting Webhook Configuration (on
create_media_buy):reporting_webhookobject with URL, auth, and desired frequencyrequested_metrics: buyers can optionally request a subset of available metrics to reduce payload sizeWebhook Payload Structure:
notification_type:scheduled(regular update),final(campaign complete), ordelayed(data not yet available)sequence_number: sequential notification tracking (starts at 1)next_expected_at: ISO 8601 timestamp for next expected notificationmedia_buy_deliveries: ALWAYS an array to enable publisher aggregationPublisher Aggregation:
aggregated_totalsfield is NOT included in webhook payloads (only in API responses)Changes
New Schema:
static/schemas/v1/core/reporting-capabilities.json: Defines reporting capabilities structureModified Schemas:
static/schemas/v1/core/product.json: Added optionalreporting_capabilitiesreferencestatic/schemas/v1/media-buy/create-media-buy-request.json: Added optionalreporting_webhookconfigurationstatic/schemas/v1/media-buy/get-media-buy-delivery-response.json:notification_type,sequence_number,next_expected_at)deliveries→media_buy_deliveriesto clarify array semanticsaggregated_totalsas API-only (not in webhooks)Documentation Updates:
docs/media-buy/task-reference/get_products.md: Documentedreporting_capabilitiesfielddocs/media-buy/task-reference/create_media_buy.md: Documentedreporting_webhookconfigurationdocs/media-buy/media-buys/optimization-reporting.md: Added comprehensive "Webhook-Based Reporting" sectiondocs/media-buy/task-reference/get_media_buy_delivery.md: Updated to usemedia_buy_deliveriesdocs/protocols/core-concepts.md: Added "Reporting Webhooks" section with implementation guidanceTest Plan
Notes
🤖 Generated with Claude Code