From 0146d4a6f3fa750fbd24987edbd5833490ca6b7a Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Fri, 3 Oct 2025 04:51:37 -0400 Subject: [PATCH 01/13] feat: add comprehensive webhook security and reliability specifications MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Address Yahoo's production readiness feedback on PR #81 with enterprise-grade webhook infrastructure requirements. Security & Authentication: - HMAC-SHA256 signature verification with X-ADCP-Signature header - Replay attack prevention with timestamp validation - Constant-time comparison for timing attack protection - Secret rotation guidance Reliability & Circuit Breaking: - Exponential backoff with jitter for retries - Circuit breaker pattern (CLOSED/OPEN/HALF_OPEN states) - Bounded queues (1000 webhooks per endpoint) to prevent memory exhaustion - Per-endpoint circuit breakers with automatic recovery Privacy & Compliance: - PII scrubbing requirements for GDPR/CCPA - Explicit guidance on what to scrub vs. aggregate - Minimum aggregation thresholds to prevent re-identification - Publisher and buyer responsibilities documented Operational Monitoring: - Webhook health metrics in get_media_buy_delivery responses - Circuit breaker status visibility for debugging - Success/failure tracking and alerting guidance - Last attempt timestamps and failure reasons Data Integrity: - get_media_buy_delivery API defined as authoritative source of truth - Reconciliation process with discrepancy thresholds - Late-arriving impression handling (adjustments array and correction webhooks) - Attribution window guidance (7-90 days depending on ad type) Partial Failure Handling: - Best-effort delivery with per-campaign status indicators - partial_data flag and unavailable_count field - reporting_delayed status for temporarily unavailable data - Guidance on when to use partial vs. delayed notifications Timezone & DST: - Explicit handling of 23-hour and 25-hour DST transition days - IANA timezone identifier requirements - ISO 8601 with timezone offset requirements - Complete implementation examples for period calculation and billing - Common pitfalls and test date guidance Schema Updates: - Add notification_type: "correction" for late-arriving data - Add partial_data and unavailable_count fields - Add webhook_health object with circuit breaker status - Add reporting_period DST fields (duration_hours, is_dst_transition) - Add status: "reporting_delayed" for partial failures - Add adjustments array for incremental corrections This addresses all critical gaps identified by Yahoo for production deployment at scale, enabling Phase 1 rollout with proper security, monitoring, and operational safeguards. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../media-buys/optimization-reporting.md | 621 +++++++++++++++++- docs/protocols/core-concepts.md | 339 ++++++++++ .../get-media-buy-delivery-response.json | 141 +++- 3 files changed, 1092 insertions(+), 9 deletions(-) diff --git a/docs/media-buy/media-buys/optimization-reporting.md b/docs/media-buy/media-buys/optimization-reporting.md index df5924f954..cc32c009c3 100644 --- a/docs/media-buy/media-buys/optimization-reporting.md +++ b/docs/media-buy/media-buys/optimization-reporting.md @@ -145,16 +145,170 @@ Reporting webhooks use the same payload structure as [`get_media_buy_delivery`]( - **`next_expected_at`**: ISO 8601 timestamp for next notification (omitted for final notifications) - **`media_buy_deliveries`**: Array of media buy delivery data (may contain multiple media buys aggregated by publisher) -### Timezone Considerations +### Timezone and DST Handling -For daily and monthly frequencies, the publisher's reporting timezone (from `reporting_capabilities.timezone`) determines period boundaries: +For daily and monthly frequencies, the publisher's reporting timezone (from `reporting_capabilities.timezone`) determines period boundaries. Publishers MUST handle Daylight Saving Time (DST) transitions correctly to prevent billing disputes. + +#### Timezone Rules - **Daily**: Reporting day starts/ends at midnight in publisher's timezone - **Monthly**: Reporting month starts on 1st and ends on last day of month in publisher's timezone -- **Hourly**: Uses UTC unless otherwise specified +- **Hourly**: Uses UTC unless otherwise specified in product capabilities **Example**: Publisher with `"timezone": "America/New_York"` and daily frequency sends notifications at ~8:00 UTC (midnight ET + expected delay). +#### DST Transition Handling + +**Critical Rule**: DST transitions can create 23-hour or 25-hour reporting days. Publishers and buyers MUST account for this in calculations. + +**Spring Forward (Clocks move ahead):** +``` +March 10, 2024 - DST Starts in America/New_York +- 2:00 AM becomes 3:00 AM (clocks skip forward) +- This creates a 23-hour reporting day +- Period: 2024-03-10T00:00:00-05:00 to 2024-03-10T23:59:59-04:00 +- Duration: 23 hours (82800 seconds) +``` + +**Fall Back (Clocks move back):** +``` +November 3, 2024 - DST Ends in America/New_York +- 2:00 AM becomes 1:00 AM (clocks repeat an hour) +- This creates a 25-hour reporting day +- Period: 2024-11-03T00:00:00-04:00 to 2024-11-03T23:59:59-05:00 +- Duration: 25 hours (90000 seconds) +``` + +#### Implementation Requirements + +**Publishers MUST:** +1. Use IANA timezone identifiers (e.g., `America/New_York`, not `EST` or `EDT`) +2. Include timezone offset in all ISO 8601 timestamps +3. Calculate reporting periods using wall-clock time, not duration +4. Document DST behavior in product documentation +5. Test webhook delivery during DST transitions + +**Buyers MUST:** +1. Parse timezone offsets from ISO 8601 timestamps +2. Expect 23/25-hour days during DST transitions +3. Calculate durations using actual timestamps, not day counts +4. Validate period boundaries account for DST + +**Correct Period Calculation (Publisher):** +```javascript +const moment = require('moment-timezone'); + +function getReportingPeriod(date, timezone) { + // Use wall-clock midnight, not duration + const start = moment.tz(date, timezone).startOf('day'); + const end = moment.tz(date, timezone).endOf('day'); + + return { + start: start.toISOString(), // Includes timezone offset + end: end.toISOString(), + duration_hours: end.diff(start, 'hours', true) // Will be 23, 24, or 25 + }; +} + +// Example: DST transition day +const period = getReportingPeriod('2024-03-10', 'America/New_York'); +// Returns: +// { +// start: "2024-03-10T00:00:00-05:00", +// end: "2024-03-10T23:59:59-04:00", +// duration_hours: 23 +// } +``` + +**Correct Billing Calculation (Buyer):** +```javascript +function calculateExpectedSpend(period, dailyBudget) { + const start = new Date(period.start); + const end = new Date(period.end); + + // Calculate actual duration in hours + const durationHours = (end - start) / (1000 * 60 * 60); + + // Pro-rate budget based on actual duration + const expectedSpend = (dailyBudget / 24) * durationHours; + + return { + expected_spend: expectedSpend, + duration_hours: durationHours, + is_dst_transition: durationHours !== 24 + }; +} + +// Example: $1000 daily budget on 23-hour DST day +const spend = calculateExpectedSpend( + { start: "2024-03-10T00:00:00-05:00", end: "2024-03-10T23:59:59-04:00" }, + 1000 +); +// Returns: { expected_spend: 958.33, duration_hours: 23, is_dst_transition: true } +``` + +**Webhook Payload Format:** +```json +{ + "reporting_period": { + "start": "2024-03-10T00:00:00-05:00", + "end": "2024-03-10T23:59:59-04:00", + "duration_hours": 23, + "is_dst_transition": true, + "dst_transition_type": "spring_forward" + }, + "totals": { + "impressions": 95833, + "spend": 958.33 + } +} +``` + +#### Common Pitfalls + +**❌ WRONG - Assuming 24-hour days:** +```javascript +// This will over-bill on spring forward days +const dailySpend = impressions * cpm / 1000; +``` + +**✅ CORRECT - Pro-rating by actual duration:** +```javascript +const hourlyRate = dailyCPM / 24; +const actualSpend = (impressions * hourlyRate * durationHours) / 1000; +``` + +**❌ WRONG - Using date arithmetic without timezone:** +```javascript +// This loses DST information +const start = new Date('2024-03-10').toISOString(); +// Returns: "2024-03-10T00:00:00.000Z" (UTC, not local time) +``` + +**✅ CORRECT - Using timezone-aware library:** +```javascript +const start = moment.tz('2024-03-10', 'America/New_York').toISOString(); +// Returns: "2024-03-10T00:00:00-05:00" (preserves timezone) +``` + +#### Testing DST Transitions + +**Key Test Dates (US):** +- **Spring Forward**: Second Sunday of March (23-hour day) +- **Fall Back**: First Sunday of November (25-hour day) + +**Publishers SHOULD test:** +1. Webhook delivery timing during DST transitions +2. Period boundary calculations across DST changes +3. Spend calculations for 23/25-hour days +4. `next_expected_at` calculation across DST transitions + +**Buyers SHOULD test:** +1. Period duration calculation during DST +2. Budget pacing with non-24-hour days +3. Timezone parsing from ISO 8601 strings +4. Reconciliation across DST boundaries + ### Delayed Reporting If reporting data is not available within the product's `expected_delay_minutes`, publishers send a notification with `notification_type: "delayed"`: @@ -202,6 +356,168 @@ The `media_buy_deliveries` array may contain 1 to N media buys per webhook. Buye Buyers should iterate through the array and process each media buy independently. If aggregated totals are needed, calculate them from the individual media buy totals. +#### Partial Failure Handling + +When aggregating multiple media buys into a single webhook, publishers must handle cases where some campaigns have data available while others don't. + +**Approach: Best-Effort Delivery with Status Indicators** + +Publishers SHOULD send aggregated webhooks containing all available data, using status fields to indicate partial availability: + +```json +{ + "notification_type": "scheduled", + "sequence_number": 5, + "reporting_period": { + "start": "2024-02-05T00:00:00Z", + "end": "2024-02-05T23:59:59Z" + }, + "currency": "USD", + "media_buy_deliveries": [ + { + "media_buy_id": "mb_001", + "status": "active", + "totals": { + "impressions": 50000, + "spend": 1750 + } + }, + { + "media_buy_id": "mb_002", + "status": "active", + "totals": { + "impressions": 48500, + "spend": 1695 + } + }, + { + "media_buy_id": "mb_003", + "status": "reporting_delayed", + "message": "Reporting data temporarily unavailable for this campaign", + "expected_availability": "2024-02-06T02:00:00Z" + } + ], + "partial_data": true, + "unavailable_count": 1 +} +``` + +**Key Fields for Partial Failures:** +- `partial_data`: Boolean indicating if any campaigns are missing data +- `unavailable_count`: Number of campaigns with delayed/missing data +- `status`: Per-campaign status (`"active"`, `"reporting_delayed"`, `"failed"`) +- `expected_availability`: When delayed data is expected (if known) + +**When to Use Partial Delivery:** +1. **Upstream delays**: Some data sources are slower than others +2. **System degradation**: Partial system outage affects subset of campaigns +3. **Data quality issues**: Specific campaigns fail validation, others proceed +4. **Rate limiting**: API limits prevent fetching all campaign data + +**When NOT to Use Partial Delivery:** +1. **Complete system outage**: Send `"delayed"` notification instead +2. **All campaigns affected**: Use `notification_type: "delayed"` +3. **Buyer endpoint issues**: Circuit breaker handles this (don't send at all) + +**Buyer Processing Logic:** +```javascript +function processAggregatedWebhook(webhook) { + if (webhook.partial_data) { + console.warn(`Partial data: ${webhook.unavailable_count} campaigns delayed`); + } + + for (const delivery of webhook.media_buy_deliveries) { + if (delivery.status === 'reporting_delayed') { + // Mark campaign as pending, retry via polling or wait for next webhook + markCampaignPending(delivery.media_buy_id, delivery.expected_availability); + } else if (delivery.status === 'active') { + // Process normal delivery data + processCampaignMetrics(delivery); + } else { + console.error(`Unexpected status for ${delivery.media_buy_id}: ${delivery.status}`); + } + } +} +``` + +**Best Practices:** +- Always include all campaigns in array, even if data unavailable (with status indicator) +- Set `partial_data: true` flag when any campaigns are delayed/failed +- Provide `expected_availability` timestamp if known +- Don't retry the entire webhook - buyers can poll individual campaigns if needed +- Track partial delivery rates in monitoring to detect systemic issues + +### Privacy and Compliance + +#### PII Scrubbing for GDPR/CCPA + +Publishers MUST scrub personally identifiable information (PII) from all webhook payloads to ensure GDPR and CCPA compliance. Reporting webhooks should contain only aggregated, anonymized metrics. + +**What to Scrub:** +- User IDs, device IDs, IP addresses +- Email addresses, phone numbers +- Precise geolocation data (latitude/longitude) +- Cookie IDs, advertising IDs (unless aggregated) +- Any custom dimensions containing PII + +**What to Keep:** +- Aggregated metrics (impressions, spend, clicks, etc.) +- Coarse geography (city, state, country - not street address) +- Device type categories (mobile, desktop, tablet) +- Browser/OS categories +- Time-based aggregations + +**Example - Before PII Scrubbing (❌ DO NOT SEND):** +```json +{ + "media_buy_id": "mb_001", + "user_events": [ + { + "user_id": "user_12345", + "ip_address": "192.168.1.100", + "device_id": "abc-def-ghi", + "impressions": 1, + "lat": 40.7128, + "lon": -74.0060 + } + ] +} +``` + +**Example - After PII Scrubbing (✅ CORRECT):** +```json +{ + "media_buy_id": "mb_001", + "totals": { + "impressions": 125000, + "spend": 5625.00, + "clicks": 250 + }, + "by_geography": [ + { + "city": "New York", + "state": "NY", + "country": "US", + "impressions": 45000, + "spend": 2025.00 + } + ] +} +``` + +**Publisher Responsibilities:** +- Implement PII scrubbing at the data collection layer, not at webhook delivery +- Ensure aggregation thresholds prevent re-identification (e.g., minimum 10 users per segment) +- Document what data is collected vs. what is shared in webhooks +- Provide data processing agreements (DPAs) for GDPR compliance +- Support GDPR/CCPA data deletion requests + +**Buyer Responsibilities:** +- Do not request PII in `requested_metrics` or custom dimensions +- Understand that webhook data is aggregated and anonymized +- Implement proper data retention policies +- Include webhook data in privacy policies and user disclosures + ### Implementation Best Practices 1. **Handle Arrays**: Always process `media_buy_deliveries` as an array, even if it contains one element @@ -211,6 +527,305 @@ Buyers should iterate through the array and process each media buy independently 5. **Timezone Awareness**: Store publisher's reporting timezone for accurate period calculation 6. **Validate Frequency**: Ensure requested frequency is in product's `available_reporting_frequencies` 7. **Validate Metrics**: Ensure requested metrics are in product's `available_metrics` +8. **PII Compliance**: Never include user-level data in webhook payloads + +### Webhook Health Monitoring + +Publishers SHOULD provide operational visibility into webhook delivery health to help buyers diagnose issues and monitor reliability. + +#### Health Check via get_media_buy_delivery + +The existing `get_media_buy_delivery` task returns webhook health metadata when a webhook is configured: + +```json +{ + "status": "completed", + "message": "Delivery report for campaign campaign_2024", + "data": { + "media_buy_id": "mb_001", + "totals": { ... }, + "webhook_health": { + "webhook_url": "https://buyer.example.com/webhooks/reporting", + "last_delivery_attempt": "2024-02-05T08:00:15Z", + "last_successful_delivery": "2024-02-05T08:00:15Z", + "last_failure": "2024-02-04T08:00:10Z", + "last_failure_reason": "Connection timeout after 10 seconds", + "total_attempts": 42, + "total_successes": 40, + "total_failures": 2, + "circuit_breaker_status": "CLOSED", + "next_scheduled_delivery": "2024-02-06T08:00:00Z" + } + } +} +``` + +**Key Health Fields:** +- `last_delivery_attempt`: When publisher last attempted delivery (any result) +- `last_successful_delivery`: Most recent successful delivery +- `last_failure`: Most recent failed delivery attempt +- `last_failure_reason`: Human-readable error from last failure +- `total_attempts`: Lifetime webhook delivery attempts +- `total_successes`: Lifetime successful deliveries +- `total_failures`: Lifetime failed deliveries +- `circuit_breaker_status`: Current circuit breaker state (`CLOSED`, `OPEN`, `HALF_OPEN`) +- `next_scheduled_delivery`: When next webhook is scheduled + +**Buyer Use Cases:** +1. **Debugging**: "Why am I not receiving webhooks?" → Check `circuit_breaker_status` and `last_failure_reason` +2. **Reliability**: Calculate success rate from `total_successes / total_attempts` +3. **Alerting**: Monitor `last_successful_delivery` to detect prolonged failures +4. **Validation**: Compare `next_scheduled_delivery` to expected frequency + +#### Monitoring Best Practices + +**For Publishers:** +- Include webhook health in every `get_media_buy_delivery` response when webhook configured +- Persist delivery attempt history (last 30 days minimum) +- Emit metrics on webhook delivery rates, latencies, and circuit breaker events +- Alert internal teams when circuit breakers open for high-value buyers +- Provide webhook delivery logs to buyers upon request (via support channels) + +**For Buyers:** +- Poll `get_media_buy_delivery` occasionally to check webhook health +- Alert when `circuit_breaker_status` is `OPEN` (indicates endpoint issues) +- Track `last_successful_delivery` timestamp to detect missed webhooks +- Calculate rolling success rates to monitor publisher reliability +- Implement fallback to polling when webhook health degrades + +**Example Monitoring Query:** +```javascript +async function checkWebhookHealth(mediaBuyId) { + const delivery = await adcp.getMediaBuyDelivery({ media_buy_id: mediaBuyId }); + const health = delivery.data.webhook_health; + + if (!health) { + return { status: 'no_webhook_configured' }; + } + + if (health.circuit_breaker_status === 'OPEN') { + return { + status: 'webhook_failing', + message: `Circuit breaker OPEN. Last failure: ${health.last_failure_reason}`, + action: 'Check buyer endpoint availability and logs' + }; + } + + const timeSinceSuccess = Date.now() - new Date(health.last_successful_delivery).getTime(); + const hoursSinceSuccess = timeSinceSuccess / (1000 * 60 * 60); + + if (hoursSinceSuccess > 48) { + return { + status: 'webhook_stale', + message: `No successful delivery in ${hoursSinceSuccess.toFixed(1)} hours`, + action: 'Verify webhook endpoint is reachable' + }; + } + + const successRate = health.total_successes / health.total_attempts; + if (successRate < 0.95) { + return { + status: 'webhook_degraded', + message: `Success rate: ${(successRate * 100).toFixed(1)}%`, + action: 'Investigate intermittent failures' + }; + } + + return { status: 'healthy', success_rate: successRate }; +} +``` + +### Data Reconciliation and Source of Truth + +Webhooks provide timely notifications but are not guaranteed delivery. The `get_media_buy_delivery` task is the **authoritative source of truth** for all campaign metrics. + +#### Reconciliation Process + +Buyers SHOULD periodically reconcile webhook data against API polling to ensure accuracy: + +**Recommended Reconciliation Schedule:** +- **Hourly webhooks**: Reconcile via API daily +- **Daily webhooks**: Reconcile via API weekly +- **Monthly webhooks**: Reconcile via API at month end + 7 days + +**Reconciliation Logic:** +```javascript +async function reconcileWebhookData(mediaBuyId, startDate, endDate) { + // Get authoritative data from API + const apiData = await adcp.getMediaBuyDelivery({ + media_buy_id: mediaBuyId, + date_range: { start: startDate, end: endDate } + }); + + // Compare with webhook data in local database + const webhookData = await db.getWebhookTotals(mediaBuyId, startDate, endDate); + + const discrepancy = { + impressions: apiData.totals.impressions - webhookData.impressions, + spend: apiData.totals.spend - webhookData.spend, + clicks: apiData.totals.clicks - webhookData.clicks + }; + + // Acceptable discrepancy thresholds + const impressionVariance = Math.abs(discrepancy.impressions) / apiData.totals.impressions; + const spendVariance = Math.abs(discrepancy.spend) / apiData.totals.spend; + + if (impressionVariance > 0.02 || spendVariance > 0.01) { + // Significant discrepancy (>2% impressions or >1% spend) + console.warn(`Reconciliation discrepancy for ${mediaBuyId}:`, discrepancy); + + // Update local database with authoritative API data + await db.updateCampaignTotals(mediaBuyId, apiData.totals); + + // Alert if discrepancy is unusually large + if (impressionVariance > 0.10 || spendVariance > 0.05) { + await alertOps(`Large reconciliation discrepancy detected`, { + media_buy_id: mediaBuyId, + webhook_totals: webhookData, + api_totals: apiData.totals, + discrepancy + }); + } + } + + return { + status: impressionVariance < 0.02 ? 'reconciled' : 'discrepancy_found', + api_data: apiData.totals, + webhook_data: webhookData, + discrepancy + }; +} +``` + +**Why Discrepancies Occur:** +1. **Missed webhooks**: Network failures, circuit breaker drops +2. **Late-arriving data**: Impressions attributed after webhook sent (see below) +3. **Data corrections**: Publisher adjusts metrics after initial reporting +4. **Webhook aggregation**: Rounding errors in aggregated totals +5. **Timezone differences**: Webhook period boundaries may differ from API query + +**Source of Truth Rules:** +- **For billing**: Always use `get_media_buy_delivery` API at campaign end + attribution window +- **For real-time decisions**: Use webhook data for speed, reconcile later +- **For discrepancies**: API data wins, update local records accordingly +- **For audits**: API provides complete historical data, webhooks are ephemeral + +**Best Practices:** +- Store webhook `sequence_number` to detect missed notifications +- Run automated reconciliation daily for active campaigns +- Alert on discrepancies >2% for impressions or >1% for spend +- Use API data for all financial reporting and invoicing +- Document reconciliation process for audit compliance + +### Late-Arriving Impressions + +Ad serving data often arrives with significant delays due to attribution windows, offline tracking, and data pipeline latency. Publishers must handle late-arriving impressions transparently. + +#### Expected Delay Windows + +Publishers declare `expected_delay_minutes` in product's `reporting_capabilities`: +- **Display/Video**: Typically 240-360 minutes (4-6 hours) +- **Audio**: Typically 480-720 minutes (8-12 hours) +- **CTV**: May be 1440+ minutes (24+ hours) + +This represents when **most** data is available, not **all** data. + +#### Late Arrival Handling + +**Scenario**: Campaign runs Feb 1-7. Daily webhook sent Feb 2 at 8am with Feb 1 data. On Feb 3, publisher discovers 1000 additional Feb 1 impressions due to delayed attribution. + +**Publisher Options:** + +**Option 1: Next-Period Correction (Recommended)** +Include correction in next scheduled webhook: +```json +{ + "notification_type": "scheduled", + "sequence_number": 3, + "reporting_period": { + "start": "2024-02-02T00:00:00Z", + "end": "2024-02-02T23:59:59Z" + }, + "media_buy_deliveries": [{ + "media_buy_id": "mb_001", + "totals": { + "impressions": 52000, // Feb 2 delivery + "spend": 1820 + }, + "adjustments": [{ + "period": "2024-02-01", + "impressions": 1000, + "spend": 35, + "reason": "Late-arriving impressions from attribution window" + }] + }] +} +``` + +**Option 2: Out-of-Band Webhook** +Send immediate corrective webhook (only for significant adjustments >5%): +```json +{ + "notification_type": "correction", + "sequence_number": null, // Not part of regular sequence + "reporting_period": { + "start": "2024-02-01T00:00:00Z", + "end": "2024-02-01T23:59:59Z" + }, + "media_buy_deliveries": [{ + "media_buy_id": "mb_001", + "totals": { + "impressions": 51000, // UPDATED total for Feb 1 + "spend": 1785 + }, + "is_correction": true, + "correction_reason": "Late-arriving impressions from attribution window" + }] +} +``` + +**Buyer Processing:** +```javascript +function processWebhook(webhook) { + if (webhook.notification_type === 'correction') { + // Replace previous period data entirely + db.replaceCampaignPeriod( + webhook.media_buy_deliveries[0].media_buy_id, + webhook.reporting_period.start, + webhook.media_buy_deliveries[0].totals + ); + console.log('Applied correction webhook'); + } else if (webhook.media_buy_deliveries[0].adjustments) { + // Apply incremental adjustments + for (const adjustment of webhook.media_buy_deliveries[0].adjustments) { + db.incrementCampaignPeriod( + webhook.media_buy_deliveries[0].media_buy_id, + adjustment.period, + adjustment + ); + } + console.log('Applied period adjustments'); + } + + // Process current period data normally + processCampaignMetrics(webhook.media_buy_deliveries[0]); +} +``` + +**Attribution Window Guidance:** +- **Post-click attribution**: 7-30 days typical +- **Post-view attribution**: 1-7 days typical +- **Offline conversions**: Can be 30-90 days +- **CTV attribution**: Can be 48+ hours for set-top box data + +**Best Practices:** +- Document attribution windows in product descriptions +- Set `expected_delay_minutes` to cover 95th percentile, not median +- Send correction webhooks only for significant adjustments (>5% change) +- Use `adjustments` array for minor late-arriving data +- Include `is_correction` flag when replacing entire period data +- Run final reconciliation at campaign_end + max_attribution_window +- Clearly communicate attribution windows in publisher-buyer agreements ### Webhook Reliability diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index 491305c648..a4ba3d7cfb 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -386,6 +386,345 @@ AdCP webhooks use **at-least-once delivery** semantics with the following charac - **May arrive out of order**: Later events could arrive before earlier ones - **Timeout behavior**: Webhook delivery has limited retry attempts and timeouts +### Security + +#### HMAC Signature Verification + +Publishers MUST sign all webhook payloads with HMAC-SHA256 to ensure message authenticity and integrity. Buyers MUST verify signatures before processing webhook data. + +**Signature Generation (Publisher):** +```javascript +const crypto = require('crypto'); + +function signWebhook(payload, secret) { + const signature = crypto + .createHmac('sha256', secret) + .update(JSON.stringify(payload)) + .digest('hex'); + return signature; +} + +// Add signature to request headers +const payload = { /* webhook data */ }; +const signature = signWebhook(payload, webhookSecret); + +axios.post(webhookUrl, payload, { + headers: { + 'Content-Type': 'application/json', + 'X-ADCP-Signature': `sha256=${signature}`, + 'X-ADCP-Timestamp': new Date().toISOString() + } +}); +``` + +**Signature Verification (Buyer):** +```javascript +function verifyWebhookSignature(req, secret) { + const signature = req.headers['x-adcp-signature']; + const timestamp = req.headers['x-adcp-timestamp']; + + if (!signature || !timestamp) { + throw new Error('Missing signature or timestamp'); + } + + // Verify timestamp to prevent replay attacks (5 minute window) + const requestTime = new Date(timestamp).getTime(); + const now = Date.now(); + if (Math.abs(now - requestTime) > 5 * 60 * 1000) { + throw new Error('Webhook timestamp too old or in future'); + } + + // Extract hash from signature header + const [algorithm, hash] = signature.split('='); + if (algorithm !== 'sha256') { + throw new Error('Unsupported signature algorithm'); + } + + // Compute expected signature + const expectedSignature = crypto + .createHmac('sha256', secret) + .update(JSON.stringify(req.body)) + .digest('hex'); + + // Constant-time comparison to prevent timing attacks + if (!crypto.timingSafeEqual( + Buffer.from(hash), + Buffer.from(expectedSignature) + )) { + throw new Error('Invalid webhook signature'); + } + + return true; +} + +// Webhook endpoint +app.post('/webhooks/adcp', async (req, res) => { + try { + // ALWAYS verify signature first + verifyWebhookSignature(req, process.env.ADCP_WEBHOOK_SECRET); + + // Then process webhook + await processWebhook(req.body); + res.status(200).json({ status: 'processed' }); + } catch (error) { + console.error('Webhook verification failed:', error); + res.status(401).json({ error: 'Unauthorized' }); + } +}); +``` + +**Webhook Secret Management:** +- Secrets are exchanged out-of-band (e.g., during publisher onboarding) +- Use different secrets per buyer or per webhook endpoint +- Store secrets securely (environment variables, secret management systems) +- Support secret rotation without downtime (accept both old and new signatures during rotation) + +**Required Headers:** +- `X-ADCP-Signature`: HMAC signature in format `sha256={hex_signature}` +- `X-ADCP-Timestamp`: ISO 8601 timestamp of webhook generation +- `Content-Type`: Must be `application/json` + +### Retry and Circuit Breaker Patterns + +Publishers MUST implement retry logic with circuit breakers to handle temporary buyer endpoint failures without overwhelming systems or accumulating unbounded queues. + +#### Retry Strategy + +Publishers SHOULD use exponential backoff with jitter for webhook delivery retries: + +```javascript +class WebhookDelivery { + constructor() { + this.maxRetries = 3; + this.baseDelay = 1000; // 1 second + this.maxDelay = 60000; // 1 minute + } + + async deliverWithRetry(url, payload, attempt = 0) { + try { + const response = await this.sendWebhook(url, payload); + + if (response.status >= 200 && response.status < 300) { + return { success: true, attempts: attempt + 1 }; + } + + // Retry on 5xx errors and timeouts + if (response.status >= 500 && attempt < this.maxRetries) { + await this.delayWithJitter(attempt); + return this.deliverWithRetry(url, payload, attempt + 1); + } + + // Don't retry 4xx errors (client errors) + return { success: false, error: 'Client error', attempts: attempt + 1 }; + + } catch (error) { + if (attempt < this.maxRetries) { + await this.delayWithJitter(attempt); + return this.deliverWithRetry(url, payload, attempt + 1); + } + return { success: false, error: error.message, attempts: attempt + 1 }; + } + } + + async delayWithJitter(attempt) { + const exponentialDelay = Math.min( + this.baseDelay * Math.pow(2, attempt), + this.maxDelay + ); + // Add ±25% jitter to prevent thundering herd + const jitter = exponentialDelay * (0.75 + Math.random() * 0.5); + await new Promise(resolve => setTimeout(resolve, jitter)); + } + + async sendWebhook(url, payload) { + return axios.post(url, payload, { + timeout: 10000, // 10 second timeout + headers: { + 'Content-Type': 'application/json', + 'X-ADCP-Signature': this.signPayload(payload), + 'X-ADCP-Timestamp': new Date().toISOString() + } + }); + } +} +``` + +**Retry Schedule:** +- Attempt 1: Immediate +- Attempt 2: After ~1 second (with jitter) +- Attempt 3: After ~2 seconds (with jitter) +- Attempt 4: After ~4 seconds (with jitter) +- Give up after 4 total attempts + +#### Circuit Breaker Pattern + +Publishers MUST implement circuit breakers to prevent webhook queues from growing unbounded when buyer endpoints are down: + +```javascript +class CircuitBreaker { + constructor(endpoint) { + this.endpoint = endpoint; + this.state = 'CLOSED'; // CLOSED, OPEN, HALF_OPEN + this.failureCount = 0; + this.failureThreshold = 5; + this.successThreshold = 2; + this.timeout = 60000; // 1 minute + this.halfOpenTime = null; + this.successCount = 0; + } + + async execute(fn) { + if (this.state === 'OPEN') { + // Check if circuit should move to HALF_OPEN + if (Date.now() - this.halfOpenTime > this.timeout) { + this.state = 'HALF_OPEN'; + this.successCount = 0; + } else { + throw new Error('Circuit breaker is OPEN'); + } + } + + try { + const result = await fn(); + this.onSuccess(); + return result; + } catch (error) { + this.onFailure(); + throw error; + } + } + + onSuccess() { + this.failureCount = 0; + + if (this.state === 'HALF_OPEN') { + this.successCount++; + if (this.successCount >= this.successThreshold) { + this.state = 'CLOSED'; + console.log(`Circuit breaker CLOSED for ${this.endpoint}`); + } + } + } + + onFailure() { + this.failureCount++; + + if (this.failureCount >= this.failureThreshold) { + this.state = 'OPEN'; + this.halfOpenTime = Date.now(); + console.error(`Circuit breaker OPEN for ${this.endpoint}`); + + // Alert monitoring system + this.alertMonitoring(); + } + } + + alertMonitoring() { + // Notify operations team that endpoint is down + console.error(`ALERT: Webhook endpoint ${this.endpoint} is unreachable`); + // Send to monitoring system (e.g., PagerDuty, Datadog) + } + + isOpen() { + return this.state === 'OPEN'; + } +} + +// Usage with webhook delivery +class WebhookManager { + constructor() { + this.circuitBreakers = new Map(); + this.maxQueueSize = 1000; // Per endpoint + this.queues = new Map(); + } + + getCircuitBreaker(endpoint) { + if (!this.circuitBreakers.has(endpoint)) { + this.circuitBreakers.set(endpoint, new CircuitBreaker(endpoint)); + } + return this.circuitBreakers.get(endpoint); + } + + async sendWebhook(endpoint, payload) { + const breaker = this.getCircuitBreaker(endpoint); + + // Check circuit breaker before queuing + if (breaker.isOpen()) { + console.warn(`Dropping webhook for ${endpoint} - circuit breaker OPEN`); + return { success: false, reason: 'circuit_breaker_open' }; + } + + // Check queue size limit + const queue = this.queues.get(endpoint) || []; + if (queue.length >= this.maxQueueSize) { + console.error(`Dropping webhook for ${endpoint} - queue full (${queue.length})`); + return { success: false, reason: 'queue_full' }; + } + + // Attempt delivery through circuit breaker + try { + return await breaker.execute(async () => { + const delivery = new WebhookDelivery(); + return await delivery.deliverWithRetry(endpoint, payload); + }); + } catch (error) { + return { success: false, reason: error.message }; + } + } +} +``` + +**Circuit Breaker States:** +- **CLOSED**: Normal operation, webhooks delivered +- **OPEN**: Endpoint is down, webhooks are dropped (not queued) +- **HALF_OPEN**: Testing if endpoint recovered, limited webhooks sent + +**Why Circuit Breakers Matter:** +At Yahoo scale with thousands of campaigns, a single buyer endpoint being down could queue millions of webhooks. Circuit breakers prevent this by failing fast and dropping webhooks when endpoints are unreachable. + +#### Queue Management + +Publishers SHOULD implement bounded queues with overflow policies: + +```javascript +class BoundedWebhookQueue { + constructor(maxSize = 1000) { + this.maxSize = maxSize; + this.queue = []; + this.droppedCount = 0; + } + + enqueue(webhook) { + if (this.queue.length >= this.maxSize) { + // Overflow policy: drop oldest webhooks + const dropped = this.queue.shift(); + this.droppedCount++; + console.warn(`Dropped webhook ${dropped.id} due to queue overflow`); + } + this.queue.push(webhook); + } + + dequeue() { + return this.queue.shift(); + } + + size() { + return this.queue.length; + } + + getDroppedCount() { + return this.droppedCount; + } +} +``` + +**Best Practices:** +- Set max queue size based on available memory and recovery time +- Monitor queue depth and dropped webhook counts +- Alert operations when queues are consistently full +- Use dead letter queues for manual investigation of persistent failures +- Implement queue per buyer endpoint (not global queue) + ### Implementation Requirements #### Idempotent Webhook Handlers diff --git a/static/schemas/v1/media-buy/get-media-buy-delivery-response.json b/static/schemas/v1/media-buy/get-media-buy-delivery-response.json index 15f444a8af..9cdcd70b19 100644 --- a/static/schemas/v1/media-buy/get-media-buy-delivery-response.json +++ b/static/schemas/v1/media-buy/get-media-buy-delivery-response.json @@ -12,8 +12,17 @@ }, "notification_type": { "type": "string", - "enum": ["scheduled", "final", "delayed"], - "description": "Type of webhook notification (only present in webhook deliveries): scheduled = regular periodic update, final = campaign completed, delayed = data not yet available" + "enum": ["scheduled", "final", "delayed", "correction"], + "description": "Type of webhook notification (only present in webhook deliveries): scheduled = regular periodic update, final = campaign completed, delayed = data not yet available, correction = out-of-band correction for late-arriving data" + }, + "partial_data": { + "type": "boolean", + "description": "Indicates if any media buys in this webhook have missing/delayed data (only present in webhook deliveries)" + }, + "unavailable_count": { + "type": "integer", + "minimum": 0, + "description": "Number of media buys with reporting_delayed or failed status (only present in webhook deliveries when partial_data is true)" }, "sequence_number": { "type": "integer", @@ -32,12 +41,26 @@ "start": { "type": "string", "format": "date-time", - "description": "ISO 8601 start timestamp" + "description": "ISO 8601 start timestamp with timezone offset" }, "end": { "type": "string", "format": "date-time", - "description": "ISO 8601 end timestamp" + "description": "ISO 8601 end timestamp with timezone offset" + }, + "duration_hours": { + "type": "number", + "description": "Actual duration in hours (may be 23, 24, or 25 due to DST transitions)", + "minimum": 0 + }, + "is_dst_transition": { + "type": "boolean", + "description": "Indicates if this period includes a DST transition" + }, + "dst_transition_type": { + "type": "string", + "enum": ["spring_forward", "fall_back"], + "description": "Type of DST transition (only present when is_dst_transition is true)" } }, "required": ["start", "end"], @@ -97,8 +120,57 @@ }, "status": { "type": "string", - "description": "Current media buy status", - "enum": ["pending", "active", "paused", "completed", "failed"] + "description": "Current media buy status. In webhook context, reporting_delayed indicates data temporarily unavailable.", + "enum": ["pending", "active", "paused", "completed", "failed", "reporting_delayed"] + }, + "message": { + "type": "string", + "description": "Human-readable message (typically present when status is reporting_delayed or failed)" + }, + "expected_availability": { + "type": "string", + "format": "date-time", + "description": "When delayed data is expected to be available (only present when status is reporting_delayed)" + }, + "is_correction": { + "type": "boolean", + "description": "Indicates this delivery replaces a previously reported period (for late-arriving data)" + }, + "correction_reason": { + "type": "string", + "description": "Explanation of why correction was needed (only present when is_correction is true)" + }, + "adjustments": { + "type": "array", + "description": "Incremental adjustments to previously reported periods (for late-arriving data)", + "items": { + "type": "object", + "properties": { + "period": { + "type": "string", + "pattern": "^\\d{4}-\\d{2}-\\d{2}$", + "description": "Date being adjusted (YYYY-MM-DD)" + }, + "impressions": { + "type": "number", + "description": "Additional impressions to add" + }, + "spend": { + "type": "number", + "description": "Additional spend to add" + }, + "clicks": { + "type": "number", + "description": "Additional clicks to add" + }, + "reason": { + "type": "string", + "description": "Explanation of adjustment" + } + }, + "required": ["period", "reason"], + "additionalProperties": false + } }, "totals": { "type": "object", @@ -209,6 +281,63 @@ "required": ["date", "impressions", "spend"], "additionalProperties": false } + }, + "webhook_health": { + "type": "object", + "description": "Webhook delivery health metrics (only present in get_media_buy_delivery API responses when webhook is configured, not in webhook notifications themselves)", + "properties": { + "webhook_url": { + "type": "string", + "format": "uri", + "description": "Configured webhook endpoint URL" + }, + "last_delivery_attempt": { + "type": "string", + "format": "date-time", + "description": "When publisher last attempted delivery (success or failure)" + }, + "last_successful_delivery": { + "type": "string", + "format": "date-time", + "description": "Most recent successful delivery" + }, + "last_failure": { + "type": "string", + "format": "date-time", + "description": "Most recent failed delivery attempt" + }, + "last_failure_reason": { + "type": "string", + "description": "Human-readable error from last failure" + }, + "total_attempts": { + "type": "integer", + "minimum": 0, + "description": "Lifetime webhook delivery attempts" + }, + "total_successes": { + "type": "integer", + "minimum": 0, + "description": "Lifetime successful deliveries" + }, + "total_failures": { + "type": "integer", + "minimum": 0, + "description": "Lifetime failed deliveries" + }, + "circuit_breaker_status": { + "type": "string", + "enum": ["CLOSED", "OPEN", "HALF_OPEN"], + "description": "Current circuit breaker state: CLOSED = normal operation, OPEN = endpoint down, HALF_OPEN = testing recovery" + }, + "next_scheduled_delivery": { + "type": "string", + "format": "date-time", + "description": "When next webhook is scheduled" + } + }, + "required": ["webhook_url", "total_attempts", "total_successes", "total_failures", "circuit_breaker_status"], + "additionalProperties": false } }, "required": ["media_buy_id", "status", "totals", "by_package"], From bd675526c932103cb755cd1ed6c7a01dd88c1cca Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Fri, 3 Oct 2025 21:01:56 -0400 Subject: [PATCH 02/13] refactor: simplify webhook specifications based on implementation feedback MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Simplify and clarify webhook specifications based on Yahoo's feedback and team discussion. Focus on practical implementation guidance rather than exhaustive edge cases. Key Simplifications: 1. **Recommend UTC for Reporting** - Eliminates DST complexity for most use cases - Simplifies reconciliation across systems - Reduced DST documentation to brief "if you must use local time" section - Made DST payload fields optional (duration_hours, is_dst_transition) 2. **Clarify Hourly Webhooks Are Optional** - Publishers not required to support all frequencies - Recommend daily for Phase 1 deployments - Note cost considerations (hourly = 24x traffic) 3. **Add Offline Reporting Option** - Cloud storage bucket push for large buyer-seller pairs - 10-100x cost reduction vs. webhooks for high volume - Supports S3, GCS, Azure Blob Storage - Same payload format as webhooks (JSON Lines, CSV, Parquet) - Best for >100 campaigns or hourly reporting needs 4. **Move Reconciliation to General Reporting** - Not webhook-specific - applies to all delivery methods - Webhooks, offline files, and polling all need reconciliation - Late-arriving data affects all methods - API remains authoritative source of truth 5. **Clarify Webhook Correction Advantage** - Webhooks enable "overwrite this period" with notification_type: correction - Polling-only implementations must detect via reconciliation - Makes late-arriving impression handling easier for buyers These changes address the feedback: - "Is it possible / realistic to ask for all reporting in UTC?" → Yes, recommended - "Publishers don't have to support hourly" → Clarified as optional - "Offline reporting mechanism makes sense" → Added cloud storage option - "Reconciliation isn't a webhook issue" → Moved to general reporting section - "Webhooks could fire same range again with 'overwrite'" → Emphasized this advantage No schema changes needed - existing fields support both simplified (UTC, daily) and complex (local timezone, hourly) implementations. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../media-buys/optimization-reporting.md | 264 +++++++----------- 1 file changed, 104 insertions(+), 160 deletions(-) diff --git a/docs/media-buy/media-buys/optimization-reporting.md b/docs/media-buy/media-buys/optimization-reporting.md index cc32c009c3..b8eddf51c7 100644 --- a/docs/media-buy/media-buys/optimization-reporting.md +++ b/docs/media-buy/media-buys/optimization-reporting.md @@ -55,9 +55,48 @@ Stay informed of important campaign events: ## Webhook-Based Reporting -Publishers can proactively push reporting data to buyers on a scheduled basis through webhook notifications. This eliminates the need for continuous polling and provides timely campaign insights. +Publishers can proactively push reporting data to buyers through webhook notifications or offline file delivery. This eliminates continuous polling and provides timely campaign insights. -### Configuration +### Delivery Methods + +**1. Webhook Push (Real-time)** - HTTP POST to buyer endpoint +- Best for: Most buyer-seller relationships +- Latency: Near real-time (seconds to minutes) +- Cost: Standard webhook infrastructure + +**2. Offline File Delivery (Batch)** - Cloud storage bucket push +- Best for: Large buyer-seller pairs (high volume) +- Latency: Scheduled batch delivery (hourly/daily) +- Cost: Significantly lower ($0.01-0.10 per GB vs. $0.50-2.00 per 1M webhooks) +- Format: JSON Lines, CSV, or Parquet files +- Storage: S3, GCS, Azure Blob Storage + +**Example: Offline Delivery** +Publisher pushes daily report files to buyer's cloud storage: +``` +s3://buyer-reports/publisher_name/2024/02/05/media_buy_delivery.json.gz +``` + +File contains same structure as webhook payload but aggregated across all campaigns. Buyer processes files on their schedule. + +**When to Use Offline Delivery:** +- \>100 active campaigns with same buyer +- Hourly reporting requirements (24x cost reduction) +- High data volume (detailed breakdowns, dimensional data) +- Buyer has batch processing infrastructure + +Publishers declare support for offline delivery in product capabilities: +```json +{ + "reporting_capabilities": { + "supports_webhooks": true, + "supports_offline_delivery": true, + "offline_delivery_protocols": ["s3", "gcs"] + } +} +``` + +### Webhook Configuration Configure reporting webhooks when creating a media buy using the `reporting_webhook` parameter: @@ -76,12 +115,14 @@ Configure reporting webhooks when creating a media buy using the `reporting_webh ### Supported Frequencies -Publishers declare supported reporting frequencies in the product's `reporting_capabilities`: +Publishers declare supported reporting frequencies in the product's `reporting_capabilities`. Publishers are **not required** to support all frequencies - choose what makes operational sense for your platform. -- **`hourly`**: Receive notifications every hour during campaign flight -- **`daily`**: Receive notifications once per day (timezone specified by publisher) +- **`hourly`**: Receive notifications every hour during campaign flight (optional, consider cost/complexity) +- **`daily`**: Receive notifications once per day (most common, recommended for Phase 1) - **`monthly`**: Receive notifications once per month (timezone specified by publisher) +**Cost Consideration:** Hourly webhooks generate 24x more traffic than daily. Large buyer-seller pairs may prefer offline reporting mechanisms (see below) for cost efficiency. + ### Available Metrics Publishers declare which metrics they can provide in `reporting_capabilities.available_metrics`. Common metrics include: @@ -145,109 +186,50 @@ Reporting webhooks use the same payload structure as [`get_media_buy_delivery`]( - **`next_expected_at`**: ISO 8601 timestamp for next notification (omitted for final notifications) - **`media_buy_deliveries`**: Array of media buy delivery data (may contain multiple media buys aggregated by publisher) -### Timezone and DST Handling - -For daily and monthly frequencies, the publisher's reporting timezone (from `reporting_capabilities.timezone`) determines period boundaries. Publishers MUST handle Daylight Saving Time (DST) transitions correctly to prevent billing disputes. - -#### Timezone Rules +### Timezone Handling -- **Daily**: Reporting day starts/ends at midnight in publisher's timezone -- **Monthly**: Reporting month starts on 1st and ends on last day of month in publisher's timezone -- **Hourly**: Uses UTC unless otherwise specified in product capabilities - -**Example**: Publisher with `"timezone": "America/New_York"` and daily frequency sends notifications at ~8:00 UTC (midnight ET + expected delay). - -#### DST Transition Handling +**Recommendation: Use UTC for all reporting periods.** This eliminates DST complexity, simplifies reconciliation, and reduces implementation burden for both publishers and buyers. -**Critical Rule**: DST transitions can create 23-hour or 25-hour reporting days. Publishers and buyers MUST account for this in calculations. +#### UTC Reporting (Recommended) -**Spring Forward (Clocks move ahead):** -``` -March 10, 2024 - DST Starts in America/New_York -- 2:00 AM becomes 3:00 AM (clocks skip forward) -- This creates a 23-hour reporting day -- Period: 2024-03-10T00:00:00-05:00 to 2024-03-10T23:59:59-04:00 -- Duration: 23 hours (82800 seconds) -``` - -**Fall Back (Clocks move back):** -``` -November 3, 2024 - DST Ends in America/New_York -- 2:00 AM becomes 1:00 AM (clocks repeat an hour) -- This creates a 25-hour reporting day -- Period: 2024-11-03T00:00:00-04:00 to 2024-11-03T23:59:59-05:00 -- Duration: 25 hours (90000 seconds) +```json +{ + "reporting_capabilities": { + "timezone": "UTC", + "available_reporting_frequencies": ["daily"] + } +} ``` -#### Implementation Requirements +**Benefits:** +- No DST transitions (every day is 24 hours) +- Simpler period calculations +- Easier reconciliation across systems +- Industry standard for programmatic advertising -**Publishers MUST:** -1. Use IANA timezone identifiers (e.g., `America/New_York`, not `EST` or `EDT`) -2. Include timezone offset in all ISO 8601 timestamps -3. Calculate reporting periods using wall-clock time, not duration -4. Document DST behavior in product documentation -5. Test webhook delivery during DST transitions +#### Local Timezone Reporting (If Required) -**Buyers MUST:** -1. Parse timezone offsets from ISO 8601 timestamps -2. Expect 23/25-hour days during DST transitions -3. Calculate durations using actual timestamps, not day counts -4. Validate period boundaries account for DST +If business requirements demand local timezone reporting (e.g., for broadcast media aligned to market schedules): -**Correct Period Calculation (Publisher):** -```javascript -const moment = require('moment-timezone'); - -function getReportingPeriod(date, timezone) { - // Use wall-clock midnight, not duration - const start = moment.tz(date, timezone).startOf('day'); - const end = moment.tz(date, timezone).endOf('day'); - - return { - start: start.toISOString(), // Includes timezone offset - end: end.toISOString(), - duration_hours: end.diff(start, 'hours', true) // Will be 23, 24, or 25 - }; -} +- **Daily**: Reporting day starts/ends at midnight in publisher's timezone +- **Monthly**: Reporting month starts on 1st and ends on last day of month in publisher's timezone +- Use IANA timezone identifiers (e.g., `"America/New_York"`) +- Include timezone offset in all ISO 8601 timestamps -// Example: DST transition day -const period = getReportingPeriod('2024-03-10', 'America/New_York'); -// Returns: -// { -// start: "2024-03-10T00:00:00-05:00", -// end: "2024-03-10T23:59:59-04:00", -// duration_hours: 23 -// } -``` +**Example**: Publisher with `"timezone": "America/New_York"` and daily frequency sends notifications at ~8:00 UTC (midnight ET + expected delay). -**Correct Billing Calculation (Buyer):** -```javascript -function calculateExpectedSpend(period, dailyBudget) { - const start = new Date(period.start); - const end = new Date(period.end); +#### DST Transition Handling (Local Timezone Only) - // Calculate actual duration in hours - const durationHours = (end - start) / (1000 * 60 * 60); +**If using local timezone reporting** (not UTC), be aware that DST transitions create 23-hour or 25-hour reporting days. - // Pro-rate budget based on actual duration - const expectedSpend = (dailyBudget / 24) * durationHours; +**Key Considerations:** +- **Spring Forward**: Creates a 23-hour day (e.g., March 10 in America/New_York) +- **Fall Back**: Creates a 25-hour day (e.g., November 3 in America/New_York) +- Include `duration_hours` field in reporting_period to prevent billing disputes +- Pro-rate budgets based on actual duration, not assumed 24 hours +- Use timezone-aware libraries (moment-timezone, date-fns-tz) for period calculations - return { - expected_spend: expectedSpend, - duration_hours: durationHours, - is_dst_transition: durationHours !== 24 - }; -} - -// Example: $1000 daily budget on 23-hour DST day -const spend = calculateExpectedSpend( - { start: "2024-03-10T00:00:00-05:00", end: "2024-03-10T23:59:59-04:00" }, - 1000 -); -// Returns: { expected_spend: 958.33, duration_hours: 23, is_dst_transition: true } -``` - -**Webhook Payload Format:** +**Optional DST Fields in Webhook Payload:** ```json { "reporting_period": { @@ -256,58 +238,11 @@ const spend = calculateExpectedSpend( "duration_hours": 23, "is_dst_transition": true, "dst_transition_type": "spring_forward" - }, - "totals": { - "impressions": 95833, - "spend": 958.33 } } ``` -#### Common Pitfalls - -**❌ WRONG - Assuming 24-hour days:** -```javascript -// This will over-bill on spring forward days -const dailySpend = impressions * cpm / 1000; -``` - -**✅ CORRECT - Pro-rating by actual duration:** -```javascript -const hourlyRate = dailyCPM / 24; -const actualSpend = (impressions * hourlyRate * durationHours) / 1000; -``` - -**❌ WRONG - Using date arithmetic without timezone:** -```javascript -// This loses DST information -const start = new Date('2024-03-10').toISOString(); -// Returns: "2024-03-10T00:00:00.000Z" (UTC, not local time) -``` - -**✅ CORRECT - Using timezone-aware library:** -```javascript -const start = moment.tz('2024-03-10', 'America/New_York').toISOString(); -// Returns: "2024-03-10T00:00:00-05:00" (preserves timezone) -``` - -#### Testing DST Transitions - -**Key Test Dates (US):** -- **Spring Forward**: Second Sunday of March (23-hour day) -- **Fall Back**: First Sunday of November (25-hour day) - -**Publishers SHOULD test:** -1. Webhook delivery timing during DST transitions -2. Period boundary calculations across DST changes -3. Spend calculations for 23/25-hour days -4. `next_expected_at` calculation across DST transitions - -**Buyers SHOULD test:** -1. Period duration calculation during DST -2. Budget pacing with non-24-hour days -3. Timezone parsing from ISO 8601 strings -4. Reconciliation across DST boundaries +These fields help buyers detect and handle non-24-hour days correctly. Omit if using UTC reporting. ### Delayed Reporting @@ -635,18 +570,25 @@ async function checkWebhookHealth(mediaBuyId) { } ``` -### Data Reconciliation and Source of Truth +## Data Reconciliation -Webhooks provide timely notifications but are not guaranteed delivery. The `get_media_buy_delivery` task is the **authoritative source of truth** for all campaign metrics. +**The `get_media_buy_delivery` API is the authoritative source of truth for all campaign metrics**, regardless of whether you use webhooks, offline delivery, or polling. -#### Reconciliation Process +Reconciliation is important for **any reporting delivery method** because: +- **Webhooks**: May be missed due to network failures or circuit breaker drops +- **Offline files**: May be delayed, corrupted, or fail to process +- **Polling**: May miss data during API outages +- **Late-arriving data**: Impressions can arrive 24-48+ hours after initial reporting (all methods) -Buyers SHOULD periodically reconcile webhook data against API polling to ensure accuracy: +### Reconciliation Process + +Buyers SHOULD periodically reconcile delivered data against API to ensure accuracy: **Recommended Reconciliation Schedule:** -- **Hourly webhooks**: Reconcile via API daily -- **Daily webhooks**: Reconcile via API weekly -- **Monthly webhooks**: Reconcile via API at month end + 7 days +- **Hourly delivery**: Reconcile via API daily +- **Daily delivery**: Reconcile via API weekly +- **Monthly delivery**: Reconcile via API at month end + 7 days +- **Campaign close**: Always reconcile after campaign_end + attribution_window **Reconciliation Logic:** ```javascript @@ -698,17 +640,17 @@ async function reconcileWebhookData(mediaBuyId, startDate, endDate) { ``` **Why Discrepancies Occur:** -1. **Missed webhooks**: Network failures, circuit breaker drops -2. **Late-arriving data**: Impressions attributed after webhook sent (see below) +1. **Delivery failures**: Webhooks missed, offline files corrupted, API timeouts during polling +2. **Late-arriving data**: Impressions attributed after initial reporting (all delivery methods) 3. **Data corrections**: Publisher adjusts metrics after initial reporting -4. **Webhook aggregation**: Rounding errors in aggregated totals -5. **Timezone differences**: Webhook period boundaries may differ from API query +4. **Processing errors**: Buyer-side failures to process delivered data +5. **Timezone differences**: Period boundaries may differ between delivery and API query **Source of Truth Rules:** - **For billing**: Always use `get_media_buy_delivery` API at campaign end + attribution window -- **For real-time decisions**: Use webhook data for speed, reconcile later +- **For real-time decisions**: Use delivered data (webhook/file/poll) for speed, reconcile later - **For discrepancies**: API data wins, update local records accordingly -- **For audits**: API provides complete historical data, webhooks are ephemeral +- **For audits**: API provides complete historical data, delivered data is ephemeral **Best Practices:** - Store webhook `sequence_number` to detect missed notifications @@ -719,7 +661,7 @@ async function reconcileWebhookData(mediaBuyId, startDate, endDate) { ### Late-Arriving Impressions -Ad serving data often arrives with significant delays due to attribution windows, offline tracking, and data pipeline latency. Publishers must handle late-arriving impressions transparently. +Ad serving data often arrives with significant delays due to attribution windows, offline tracking, and data pipeline latency. Publishers must handle late-arriving impressions transparently **regardless of delivery method** (webhooks, offline files, or polling). #### Expected Delay Windows @@ -734,10 +676,12 @@ This represents when **most** data is available, not **all** data. **Scenario**: Campaign runs Feb 1-7. Daily webhook sent Feb 2 at 8am with Feb 1 data. On Feb 3, publisher discovers 1000 additional Feb 1 impressions due to delayed attribution. +**Webhook Advantage:** Publisher can send a correction webhook with `notification_type: "correction"` to overwrite the previous period. With polling-only, buyer must detect discrepancy through reconciliation. + **Publisher Options:** **Option 1: Next-Period Correction (Recommended)** -Include correction in next scheduled webhook: +Include correction in next scheduled delivery (webhook, file, or available via API): ```json { "notification_type": "scheduled", @@ -762,8 +706,8 @@ Include correction in next scheduled webhook: } ``` -**Option 2: Out-of-Band Webhook** -Send immediate corrective webhook (only for significant adjustments >5%): +**Option 2: Out-of-Band Correction (Webhooks Only)** +Send immediate corrective webhook with "overwrite this period" instruction (only for significant adjustments >5%): ```json { "notification_type": "correction", From 41b2baf6c455e24499d90c2e203c1f68442f4b87 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Fri, 3 Oct 2025 21:08:04 -0400 Subject: [PATCH 03/13] refactor: simplify webhook specs to UTC-only with minimal adjustment model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Further simplify webhook specifications based on team consensus: 1. **UTC-Only Reporting (MUST)** - Removed local timezone option entirely - All reporting periods use UTC (no DST complexity) - Daily: 00:00:00Z to 23:59:59Z (always 24 hours) - Removed duration_hours, is_dst_transition, dst_transition_type fields 2. **Simplified Late-Arrival Handling** - Single approach: Resend period with is_adjusted: true - Removed adjustments array (incremental model) - Removed is_correction, correction_reason fields - notification_type: "adjusted" (replaced "correction") - Buyer simply replaces entire period when is_adjusted flag present 3. **Webhook Health via Task Management** - Removed webhook_health object from delivery response - Webhook delivery status tracked through global task management - Consistent with other AdCP operations - No need for media-buy-specific health endpoints Schema Changes: - notification_type: "correction" → "adjusted" - Removed reporting_period.{duration_hours, is_dst_transition, dst_transition_type} - Removed media_buy_deliveries[].{adjustments, is_correction, correction_reason} - Added media_buy_deliveries[].is_adjusted (boolean) - Removed media_buy_deliveries[].webhook_health (use task management) Documentation simplified by ~200 lines while maintaining all critical security and reliability features (HMAC, circuit breakers, PII scrubbing, etc.). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../media-buys/optimization-reporting.md | 261 +++--------------- .../get-media-buy-delivery-response.json | 121 +------- 2 files changed, 49 insertions(+), 333 deletions(-) diff --git a/docs/media-buy/media-buys/optimization-reporting.md b/docs/media-buy/media-buys/optimization-reporting.md index b8eddf51c7..0a4a4c0905 100644 --- a/docs/media-buy/media-buys/optimization-reporting.md +++ b/docs/media-buy/media-buys/optimization-reporting.md @@ -188,9 +188,7 @@ Reporting webhooks use the same payload structure as [`get_media_buy_delivery`]( ### Timezone Handling -**Recommendation: Use UTC for all reporting periods.** This eliminates DST complexity, simplifies reconciliation, and reduces implementation burden for both publishers and buyers. - -#### UTC Reporting (Recommended) +**All reporting MUST use UTC.** This eliminates DST complexity, simplifies reconciliation, and ensures consistent 24-hour reporting periods. ```json { @@ -201,49 +199,21 @@ Reporting webhooks use the same payload structure as [`get_media_buy_delivery`]( } ``` -**Benefits:** -- No DST transitions (every day is 24 hours) -- Simpler period calculations -- Easier reconciliation across systems -- Industry standard for programmatic advertising - -#### Local Timezone Reporting (If Required) - -If business requirements demand local timezone reporting (e.g., for broadcast media aligned to market schedules): - -- **Daily**: Reporting day starts/ends at midnight in publisher's timezone -- **Monthly**: Reporting month starts on 1st and ends on last day of month in publisher's timezone -- Use IANA timezone identifiers (e.g., `"America/New_York"`) -- Include timezone offset in all ISO 8601 timestamps - -**Example**: Publisher with `"timezone": "America/New_York"` and daily frequency sends notifications at ~8:00 UTC (midnight ET + expected delay). - -#### DST Transition Handling (Local Timezone Only) - -**If using local timezone reporting** (not UTC), be aware that DST transitions create 23-hour or 25-hour reporting days. +**Reporting periods:** +- Daily: 00:00:00Z to 23:59:59Z (always 24 hours) +- Hourly: Top of hour to 59:59 seconds (always 1 hour) +- Monthly: First to last day of month -**Key Considerations:** -- **Spring Forward**: Creates a 23-hour day (e.g., March 10 in America/New_York) -- **Fall Back**: Creates a 25-hour day (e.g., November 3 in America/New_York) -- Include `duration_hours` field in reporting_period to prevent billing disputes -- Pro-rate budgets based on actual duration, not assumed 24 hours -- Use timezone-aware libraries (moment-timezone, date-fns-tz) for period calculations - -**Optional DST Fields in Webhook Payload:** +**Example webhook payload:** ```json { "reporting_period": { - "start": "2024-03-10T00:00:00-05:00", - "end": "2024-03-10T23:59:59-04:00", - "duration_hours": 23, - "is_dst_transition": true, - "dst_transition_type": "spring_forward" + "start": "2024-02-05T00:00:00Z", + "end": "2024-02-05T23:59:59Z" } } ``` -These fields help buyers detect and handle non-24-hour days correctly. Omit if using UTC reporting. - ### Delayed Reporting If reporting data is not available within the product's `expected_delay_minutes`, publishers send a notification with `notification_type: "delayed"`: @@ -466,109 +436,17 @@ Publishers MUST scrub personally identifiable information (PII) from all webhook ### Webhook Health Monitoring -Publishers SHOULD provide operational visibility into webhook delivery health to help buyers diagnose issues and monitor reliability. - -#### Health Check via get_media_buy_delivery - -The existing `get_media_buy_delivery` task returns webhook health metadata when a webhook is configured: - -```json -{ - "status": "completed", - "message": "Delivery report for campaign campaign_2024", - "data": { - "media_buy_id": "mb_001", - "totals": { ... }, - "webhook_health": { - "webhook_url": "https://buyer.example.com/webhooks/reporting", - "last_delivery_attempt": "2024-02-05T08:00:15Z", - "last_successful_delivery": "2024-02-05T08:00:15Z", - "last_failure": "2024-02-04T08:00:10Z", - "last_failure_reason": "Connection timeout after 10 seconds", - "total_attempts": 42, - "total_successes": 40, - "total_failures": 2, - "circuit_breaker_status": "CLOSED", - "next_scheduled_delivery": "2024-02-06T08:00:00Z" - } - } -} -``` - -**Key Health Fields:** -- `last_delivery_attempt`: When publisher last attempted delivery (any result) -- `last_successful_delivery`: Most recent successful delivery -- `last_failure`: Most recent failed delivery attempt -- `last_failure_reason`: Human-readable error from last failure -- `total_attempts`: Lifetime webhook delivery attempts -- `total_successes`: Lifetime successful deliveries -- `total_failures`: Lifetime failed deliveries -- `circuit_breaker_status`: Current circuit breaker state (`CLOSED`, `OPEN`, `HALF_OPEN`) -- `next_scheduled_delivery`: When next webhook is scheduled - -**Buyer Use Cases:** -1. **Debugging**: "Why am I not receiving webhooks?" → Check `circuit_breaker_status` and `last_failure_reason` -2. **Reliability**: Calculate success rate from `total_successes / total_attempts` -3. **Alerting**: Monitor `last_successful_delivery` to detect prolonged failures -4. **Validation**: Compare `next_scheduled_delivery` to expected frequency - -#### Monitoring Best Practices - -**For Publishers:** -- Include webhook health in every `get_media_buy_delivery` response when webhook configured -- Persist delivery attempt history (last 30 days minimum) -- Emit metrics on webhook delivery rates, latencies, and circuit breaker events -- Alert internal teams when circuit breakers open for high-value buyers -- Provide webhook delivery logs to buyers upon request (via support channels) - -**For Buyers:** -- Poll `get_media_buy_delivery` occasionally to check webhook health -- Alert when `circuit_breaker_status` is `OPEN` (indicates endpoint issues) -- Track `last_successful_delivery` timestamp to detect missed webhooks -- Calculate rolling success rates to monitor publisher reliability -- Implement fallback to polling when webhook health degrades - -**Example Monitoring Query:** -```javascript -async function checkWebhookHealth(mediaBuyId) { - const delivery = await adcp.getMediaBuyDelivery({ media_buy_id: mediaBuyId }); - const health = delivery.data.webhook_health; - - if (!health) { - return { status: 'no_webhook_configured' }; - } - - if (health.circuit_breaker_status === 'OPEN') { - return { - status: 'webhook_failing', - message: `Circuit breaker OPEN. Last failure: ${health.last_failure_reason}`, - action: 'Check buyer endpoint availability and logs' - }; - } - - const timeSinceSuccess = Date.now() - new Date(health.last_successful_delivery).getTime(); - const hoursSinceSuccess = timeSinceSuccess / (1000 * 60 * 60); +Webhook delivery status is tracked through **AdCP's global task management system** (see [Task Management](../../protocols/task-management.md)). - if (hoursSinceSuccess > 48) { - return { - status: 'webhook_stale', - message: `No successful delivery in ${hoursSinceSuccess.toFixed(1)} hours`, - action: 'Verify webhook endpoint is reachable' - }; - } +When a media buy is created with `reporting_webhook` configured, the publisher creates an ongoing task for webhook delivery. Buyers can monitor webhook health using standard task queries. - const successRate = health.total_successes / health.total_attempts; - if (successRate < 0.95) { - return { - status: 'webhook_degraded', - message: `Success rate: ${(successRate * 100).toFixed(1)}%`, - action: 'Investigate intermittent failures' - }; - } +**Benefits of using task management:** +- Consistent status tracking across all AdCP operations +- Standard polling/webhook notification patterns +- Existing infrastructure for task status, history, and errors +- No need for media-buy-specific webhook health endpoints - return { status: 'healthy', success_rate: successRate }; -} -``` +If webhook delivery fails persistently (circuit breaker opens), publishers update the task status to indicate the issue. Buyers detect this through normal task monitoring. ## Data Reconciliation @@ -661,69 +539,31 @@ async function reconcileWebhookData(mediaBuyId, startDate, endDate) { ### Late-Arriving Impressions -Ad serving data often arrives with significant delays due to attribution windows, offline tracking, and data pipeline latency. Publishers must handle late-arriving impressions transparently **regardless of delivery method** (webhooks, offline files, or polling). - -#### Expected Delay Windows - -Publishers declare `expected_delay_minutes` in product's `reporting_capabilities`: -- **Display/Video**: Typically 240-360 minutes (4-6 hours) -- **Audio**: Typically 480-720 minutes (8-12 hours) -- **CTV**: May be 1440+ minutes (24+ hours) +Ad serving data often arrives with delays due to attribution windows, offline tracking, and pipeline latency. Publishers declare `expected_delay_minutes` in `reporting_capabilities`: +- **Display/Video**: Typically 4-6 hours +- **Audio**: Typically 8-12 hours +- **CTV**: May be 24+ hours This represents when **most** data is available, not **all** data. -#### Late Arrival Handling - -**Scenario**: Campaign runs Feb 1-7. Daily webhook sent Feb 2 at 8am with Feb 1 data. On Feb 3, publisher discovers 1000 additional Feb 1 impressions due to delayed attribution. +#### Handling Late Arrivals -**Webhook Advantage:** Publisher can send a correction webhook with `notification_type: "correction"` to overwrite the previous period. With polling-only, buyer must detect discrepancy through reconciliation. - -**Publisher Options:** - -**Option 1: Next-Period Correction (Recommended)** -Include correction in next scheduled delivery (webhook, file, or available via API): -```json -{ - "notification_type": "scheduled", - "sequence_number": 3, - "reporting_period": { - "start": "2024-02-02T00:00:00Z", - "end": "2024-02-02T23:59:59Z" - }, - "media_buy_deliveries": [{ - "media_buy_id": "mb_001", - "totals": { - "impressions": 52000, // Feb 2 delivery - "spend": 1820 - }, - "adjustments": [{ - "period": "2024-02-01", - "impressions": 1000, - "spend": 35, - "reason": "Late-arriving impressions from attribution window" - }] - }] -} -``` +When late data arrives for a previously reported period, **resend that period** with `is_adjusted: true`: -**Option 2: Out-of-Band Correction (Webhooks Only)** -Send immediate corrective webhook with "overwrite this period" instruction (only for significant adjustments >5%): ```json { - "notification_type": "correction", - "sequence_number": null, // Not part of regular sequence + "notification_type": "adjusted", "reporting_period": { "start": "2024-02-01T00:00:00Z", "end": "2024-02-01T23:59:59Z" }, "media_buy_deliveries": [{ "media_buy_id": "mb_001", + "is_adjusted": true, "totals": { - "impressions": 51000, // UPDATED total for Feb 1 - "spend": 1785 - }, - "is_correction": true, - "correction_reason": "Late-arriving impressions from attribution window" + "impressions": 51000, // Updated total (was 50000) + "spend": 1785 // Updated spend (was 1750) + } }] } ``` @@ -731,45 +571,28 @@ Send immediate corrective webhook with "overwrite this period" instruction (only **Buyer Processing:** ```javascript function processWebhook(webhook) { - if (webhook.notification_type === 'correction') { - // Replace previous period data entirely - db.replaceCampaignPeriod( - webhook.media_buy_deliveries[0].media_buy_id, - webhook.reporting_period.start, - webhook.media_buy_deliveries[0].totals - ); - console.log('Applied correction webhook'); - } else if (webhook.media_buy_deliveries[0].adjustments) { - // Apply incremental adjustments - for (const adjustment of webhook.media_buy_deliveries[0].adjustments) { - db.incrementCampaignPeriod( - webhook.media_buy_deliveries[0].media_buy_id, - adjustment.period, - adjustment + for (const delivery of webhook.media_buy_deliveries) { + if (delivery.is_adjusted) { + // Replace entire period with updated totals + db.replaceCampaignPeriod( + delivery.media_buy_id, + webhook.reporting_period, + delivery.totals ); + } else { + // Normal new period data + db.insertCampaignPeriod(delivery.media_buy_id, webhook.reporting_period, delivery.totals); } - console.log('Applied period adjustments'); } - - // Process current period data normally - processCampaignMetrics(webhook.media_buy_deliveries[0]); } ``` -**Attribution Window Guidance:** -- **Post-click attribution**: 7-30 days typical -- **Post-view attribution**: 1-7 days typical -- **Offline conversions**: Can be 30-90 days -- **CTV attribution**: Can be 48+ hours for set-top box data +**When to send adjusted periods:** +- Significant data changes (>2% impression variance or >1% spend variance) +- Final reconciliation at campaign_end + attribution_window +- Data quality corrections -**Best Practices:** -- Document attribution windows in product descriptions -- Set `expected_delay_minutes` to cover 95th percentile, not median -- Send correction webhooks only for significant adjustments (>5% change) -- Use `adjustments` array for minor late-arriving data -- Include `is_correction` flag when replacing entire period data -- Run final reconciliation at campaign_end + max_attribution_window -- Clearly communicate attribution windows in publisher-buyer agreements +With polling-only, buyers detect adjustments through reconciliation by comparing API results over time. ### Webhook Reliability diff --git a/static/schemas/v1/media-buy/get-media-buy-delivery-response.json b/static/schemas/v1/media-buy/get-media-buy-delivery-response.json index 9cdcd70b19..99c3cd5136 100644 --- a/static/schemas/v1/media-buy/get-media-buy-delivery-response.json +++ b/static/schemas/v1/media-buy/get-media-buy-delivery-response.json @@ -12,8 +12,8 @@ }, "notification_type": { "type": "string", - "enum": ["scheduled", "final", "delayed", "correction"], - "description": "Type of webhook notification (only present in webhook deliveries): scheduled = regular periodic update, final = campaign completed, delayed = data not yet available, correction = out-of-band correction for late-arriving data" + "enum": ["scheduled", "final", "delayed", "adjusted"], + "description": "Type of webhook notification (only present in webhook deliveries): scheduled = regular periodic update, final = campaign completed, delayed = data not yet available, adjusted = resending period with updated data" }, "partial_data": { "type": "boolean", @@ -36,31 +36,17 @@ }, "reporting_period": { "type": "object", - "description": "Date range for the report", + "description": "Date range for the report. All periods use UTC timezone.", "properties": { "start": { "type": "string", "format": "date-time", - "description": "ISO 8601 start timestamp with timezone offset" + "description": "ISO 8601 start timestamp in UTC (e.g., 2024-02-05T00:00:00Z)" }, "end": { "type": "string", "format": "date-time", - "description": "ISO 8601 end timestamp with timezone offset" - }, - "duration_hours": { - "type": "number", - "description": "Actual duration in hours (may be 23, 24, or 25 due to DST transitions)", - "minimum": 0 - }, - "is_dst_transition": { - "type": "boolean", - "description": "Indicates if this period includes a DST transition" - }, - "dst_transition_type": { - "type": "string", - "enum": ["spring_forward", "fall_back"], - "description": "Type of DST transition (only present when is_dst_transition is true)" + "description": "ISO 8601 end timestamp in UTC (e.g., 2024-02-05T23:59:59Z)" } }, "required": ["start", "end"], @@ -132,45 +118,9 @@ "format": "date-time", "description": "When delayed data is expected to be available (only present when status is reporting_delayed)" }, - "is_correction": { + "is_adjusted": { "type": "boolean", - "description": "Indicates this delivery replaces a previously reported period (for late-arriving data)" - }, - "correction_reason": { - "type": "string", - "description": "Explanation of why correction was needed (only present when is_correction is true)" - }, - "adjustments": { - "type": "array", - "description": "Incremental adjustments to previously reported periods (for late-arriving data)", - "items": { - "type": "object", - "properties": { - "period": { - "type": "string", - "pattern": "^\\d{4}-\\d{2}-\\d{2}$", - "description": "Date being adjusted (YYYY-MM-DD)" - }, - "impressions": { - "type": "number", - "description": "Additional impressions to add" - }, - "spend": { - "type": "number", - "description": "Additional spend to add" - }, - "clicks": { - "type": "number", - "description": "Additional clicks to add" - }, - "reason": { - "type": "string", - "description": "Explanation of adjustment" - } - }, - "required": ["period", "reason"], - "additionalProperties": false - } + "description": "Indicates this delivery contains updated data for a previously reported period. Buyer should replace previous period data with these totals." }, "totals": { "type": "object", @@ -281,63 +231,6 @@ "required": ["date", "impressions", "spend"], "additionalProperties": false } - }, - "webhook_health": { - "type": "object", - "description": "Webhook delivery health metrics (only present in get_media_buy_delivery API responses when webhook is configured, not in webhook notifications themselves)", - "properties": { - "webhook_url": { - "type": "string", - "format": "uri", - "description": "Configured webhook endpoint URL" - }, - "last_delivery_attempt": { - "type": "string", - "format": "date-time", - "description": "When publisher last attempted delivery (success or failure)" - }, - "last_successful_delivery": { - "type": "string", - "format": "date-time", - "description": "Most recent successful delivery" - }, - "last_failure": { - "type": "string", - "format": "date-time", - "description": "Most recent failed delivery attempt" - }, - "last_failure_reason": { - "type": "string", - "description": "Human-readable error from last failure" - }, - "total_attempts": { - "type": "integer", - "minimum": 0, - "description": "Lifetime webhook delivery attempts" - }, - "total_successes": { - "type": "integer", - "minimum": 0, - "description": "Lifetime successful deliveries" - }, - "total_failures": { - "type": "integer", - "minimum": 0, - "description": "Lifetime failed deliveries" - }, - "circuit_breaker_status": { - "type": "string", - "enum": ["CLOSED", "OPEN", "HALF_OPEN"], - "description": "Current circuit breaker state: CLOSED = normal operation, OPEN = endpoint down, HALF_OPEN = testing recovery" - }, - "next_scheduled_delivery": { - "type": "string", - "format": "date-time", - "description": "When next webhook is scheduled" - } - }, - "required": ["webhook_url", "total_attempts", "total_successes", "total_failures", "circuit_breaker_status"], - "additionalProperties": false } }, "required": ["media_buy_id", "status", "totals", "by_package"], From 5b5bc36610ecedfbbec4f4f8e5fc3bffeb88525c Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Sun, 5 Oct 2025 10:44:33 -0400 Subject: [PATCH 04/13] docs: clarify webhook behavior for async operations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary Documents webhook notification format and trigger behavior for asynchronous AdCP operations, addressing confusion about when webhooks are called and what payload format is sent. ## Changes ### Webhook Trigger Behavior - **Key rule**: Webhooks are ONLY called when initial response is `submitted` - `completed`/`failed` responses are synchronous - no webhook needed - `working` responses complete within ~120s - no webhook, just wait - `submitted` responses are long-running (hours/days) - webhooks used ### Webhook Payload Format - Webhooks POST the complete task response object (not just status) - Each webhook matches the task's response schema exactly - For `create_media_buy`: full create-media-buy-response.json structure - For `activate_signal`: full activate-signal-response.json structure ### Webhook URL Pattern - Use task name in URL: `/webhooks/adcp/{task_name}/{agent_id}/{op_id}` - Examples: `/create_media_buy/...`, `/activate_signal/...` - Enables clear routing based on task type ### Status Change Notifications For `submitted` operations, webhooks called on: - `input-required` → Human needs to respond/approve - `completed` → Operation finished successfully - `failed` → Operation failed with error details - `canceled` → Operation was canceled ## Files Modified - docs/protocols/core-concepts.md - Added comprehensive webhook section - docs/protocols/task-management.md - Updated webhook integration docs - docs/media-buy/task-reference/create_media_buy.md - Clarified webhook flow - docs/signals/tasks/activate_signal.md - Added webhook examples ## Examples Added - Human-in-the-loop approval flow with webhooks - Synchronous vs async operation handling - Complete webhook payload structures - Webhook handler implementation patterns 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../task-reference/create_media_buy.md | 68 ++++++- docs/protocols/core-concepts.md | 170 +++++++++++++++++- docs/protocols/task-management.md | 135 ++++++++++++-- docs/signals/tasks/activate_signal.md | 38 +++- 4 files changed, 381 insertions(+), 30 deletions(-) diff --git a/docs/media-buy/task-reference/create_media_buy.md b/docs/media-buy/task-reference/create_media_buy.md index b2ec428bc8..aa4813a2d8 100644 --- a/docs/media-buy/task-reference/create_media_buy.md +++ b/docs/media-buy/task-reference/create_media_buy.md @@ -821,19 +821,69 @@ For MCP implementations using polling, use this endpoint to check the status of #### Option 2: Webhooks (MCP) -Register a callback URL to receive push notifications: -```json -{ - "tool": "create_media_buy", - "arguments": { - "buyer_ref": "campaign_2024", - "packages": [...], - "webhook_url": "https://buyer.example.com/mcp/webhooks", - "webhook_auth_token": "bearer-token-xyz" +Register a callback URL to receive push notifications for long-running operations. Webhooks are ONLY used when the initial response is `submitted`. + +**Configuration:** +```javascript +const response = await session.call('create_media_buy', + { + buyer_ref: "campaign_2024", + packages: [...] + }, + { + webhook_url: "https://buyer.example.com/webhooks/adcp/create_media_buy/agent_id/op_id", + webhook_auth: { type: "bearer", credentials: "bearer-token-xyz" } } +); +``` + +**Response patterns:** +- **`completed`** - Synchronous success, webhook NOT called (you have the result) +- **`working`** - Will complete within ~120s, webhook NOT called (wait for response) +- **`submitted`** - Long-running operation, webhook WILL be called on status changes + +**Example webhook flow (only for `submitted` operations):** + +Webhook POST for human approval needed: +```http +POST /webhooks/adcp/create_media_buy/agent_id/op_id HTTP/1.1 +Host: buyer.example.com +Authorization: Bearer bearer-token-xyz +Content-Type: application/json + +{ + "adcp_version": "1.6.0", + "status": "input-required", + "task_id": "task_456", + "buyer_ref": "campaign_2024", + "message": "Campaign budget $150K requires approval to proceed" +} +``` + +**Webhook POST when complete (after approval - full create_media_buy response):** +```http +POST /webhooks/adcp/create_media_buy/agent_id/op_id HTTP/1.1 +Host: buyer.example.com +Authorization: Bearer bearer-token-xyz +Content-Type: application/json + +{ + "adcp_version": "1.6.0", + "status": "completed", + "media_buy_id": "mb_12345", + "buyer_ref": "campaign_2024", + "creative_deadline": "2024-01-30T23:59:59Z", + "packages": [ + { + "package_id": "pkg_001", + "buyer_ref": "ctv_package" + } + ] } ``` +Each webhook receives the full response object for that status. See **[Task Management: Webhook Integration](../../protocols/task-management.md#webhook-integration)** for complete details. + ### A2A Status Checking A2A supports both SSE streaming and webhooks as shown in the examples above. Choose based on your needs: diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index a4ba3d7cfb..d700ca2069 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -326,12 +326,176 @@ await a2a.send({ ### Server Decision on Webhook Usage -The server always decides whether to use webhooks: +The server decides whether to use webhooks based on the initial response status: -- **Quick operations** (< 120s): Server returns `working`, ignores webhook -- **Long operations** (hours/days): Server returns `submitted`, uses webhook if provided +- **`completed`, `failed`, `rejected`**: Synchronous response - webhook is NOT called (client already has complete response) +- **`working`**: Will respond synchronously within ~120 seconds - webhook is NOT called (just wait for the response) +- **`submitted`**: Long-running async operation - webhook WILL be called on ALL subsequent status changes - **Client choice**: Webhook is optional - clients can always poll with `tasks/get` +**Webhook trigger rule:** Webhooks are ONLY used when the initial response status is `submitted`. + +**When webhooks are called (for `submitted` operations):** +- Status changes to `input-required` → Webhook called (human needs to respond) +- Status changes to `completed` → Webhook called (final result) +- Status changes to `failed` → Webhook called (error details) +- Status changes to `canceled` → Webhook called (cancellation confirmation) + +### Webhook POST Format + +When an async operation changes status, the publisher POSTs the **complete task response object** to your webhook URL. + +#### Webhook Scenarios + +**Scenario 1: Synchronous completion (no webhook)** +```javascript +// Initial request +const response = await session.call('create_media_buy', params, { webhook_url: "..." }); + +// Response is immediate and complete - webhook is NOT called +{ + "status": "completed", + "media_buy_id": "mb_12345", + "packages": [...] +} +``` + +**Scenario 2: Quick async processing (no webhook - use working status)** +```javascript +// Initial response indicates processing will complete soon +const response = await session.call('create_media_buy', params, { webhook_url: "..." }); +{ + "status": "working", + "task_id": "task_789", + "message": "Creating media buy..." +} + +// Wait for synchronous response (within ~120 seconds) +// Webhook is NOT called - client should wait for the response to complete +// The call will return the final result synchronously +``` + +**Scenario 3: Long-running operation (webhook IS called)** +```javascript +// Initial request +const response = await session.call('create_media_buy', params, { + webhook_url: "https://buyer.com/webhooks/adcp/create_media_buy/agent_123/op_456" +}); + +// Response indicates long-running async operation +{ + "adcp_version": "1.6.0", + "status": "submitted", + "task_id": "task_456", + "buyer_ref": "nike_q1_campaign_2024", + "message": "Campaign requires sales approval. Expected time: 2-4 hours." +} + +// Later: Webhook POST when approval is needed +POST /webhooks/adcp/create_media_buy/agent_123/op_456 HTTP/1.1 +{ + "adcp_version": "1.6.0", + "status": "input-required", + "task_id": "task_456", + "buyer_ref": "nike_q1_campaign_2024", + "message": "Please approve $150K campaign to proceed" +} + +// Later: Webhook POST when approved and completed (full create_media_buy response) +POST /webhooks/adcp/create_media_buy/agent_123/op_456 HTTP/1.1 +{ + "adcp_version": "1.6.0", + "status": "completed", + "media_buy_id": "mb_12345", + "buyer_ref": "nike_q1_campaign_2024", + "creative_deadline": "2024-01-30T23:59:59Z", + "packages": [ + { + "package_id": "pkg_12345_001", + "buyer_ref": "nike_ctv_sports_package" + }, + { + "package_id": "pkg_12345_002", + "buyer_ref": "nike_audio_drive_package" + } + ] +} +``` + +#### For Other Async Operations + +Each async operation posts its specific response schema: + +- **`activate_signal`** → `activate-signal-response.json` +- **`sync_creatives`** → `sync-creatives-response.json` +- **`update_media_buy`** → `update-media-buy-response.json` + +#### Webhook URL Patterns + +Structure your webhook URLs to identify the operation and agent: + +``` +https://buyer.com/webhooks/adcp/{task_name}/{agent_id}/{operation_id} +``` + +**Example URLs:** +- `https://buyer.com/webhooks/adcp/create_media_buy/agent_abc/op_xyz` +- `https://buyer.com/webhooks/adcp/activate_signal/agent_abc/op_123` +- `https://buyer.com/webhooks/adcp/sync_creatives/agent_abc/op_456` + +Your webhook handler can parse the URL path to route to the correct handler based on the task name. + +#### Webhook Payload Structure + +Every webhook POST contains the complete task response for that status, matching the task's response schema. + +**`input-required` webhook (human needs to respond):** +```json +{ + "adcp_version": "1.6.0", + "status": "input-required", + "task_id": "task_456", + "buyer_ref": "nike_q1_campaign_2024", + "message": "Campaign budget requires VP approval to proceed" +} +``` + +**`completed` webhook (operation finished - full create_media_buy response):** +```json +{ + "adcp_version": "1.6.0", + "status": "completed", + "media_buy_id": "mb_12345", + "buyer_ref": "nike_q1_campaign_2024", + "creative_deadline": "2024-01-30T23:59:59Z", + "packages": [ + { + "package_id": "pkg_001", + "buyer_ref": "nike_ctv_package" + } + ] +} +``` + +**`failed` webhook (operation failed):** +```json +{ + "adcp_version": "1.6.0", + "status": "failed", + "task_id": "task_456", + "buyer_ref": "nike_q1_campaign_2024", + "errors": [ + { + "code": "insufficient_inventory", + "message": "Requested targeting yielded 0 available impressions", + "suggestion": "Broaden geographic targeting or increase budget" + } + ] +} +``` + +**Key principle:** Webhooks are ONLY called for `submitted` operations, and each webhook contains the full response object matching the task's response schema. + ### Task State Reconciliation Use `tasks/list` to recover from lost state: diff --git a/docs/protocols/task-management.md b/docs/protocols/task-management.md index bb74d94245..9edcb5659a 100644 --- a/docs/protocols/task-management.md +++ b/docs/protocols/task-management.md @@ -446,34 +446,135 @@ await a2a.send({ Task management integrates with protocol-level webhook configuration for push notifications. -### Webhook Events +### Webhook Configuration -AdCP sends webhook notifications for task status changes: +Configure webhooks at the protocol level when making async task calls. See **[Core Concepts: Protocol-Level Webhook Configuration](./core-concepts.md#protocol-level-webhook-configuration)** for complete setup examples. + +**Quick example:** +```javascript +const response = await session.call('create_media_buy', + { /* task params */ }, + { + webhook_url: "https://buyer.com/webhooks/adcp/create_media_buy/agent_id/operation_id", + webhook_auth: { type: "bearer", credentials: "secret" } + } +); +``` + +### Webhook POST Format + +When a task's status changes, the publisher POSTs the **complete task response object** to your webhook URL. + +**Webhook trigger rule:** Webhooks are ONLY used when the initial response status is `submitted` (long-running operations). + +**When webhooks are NOT triggered:** +- Initial response is `completed`, `failed`, or `rejected` → Synchronous response, client already has result +- Initial response is `working` → Will complete synchronously within ~120 seconds, client should wait for response + +**When webhooks ARE triggered (for `submitted` operations only):** +- Status changes to `input-required` → Webhook called (alerts that human input needed) +- Status changes to `completed` → Webhook called (final result available) +- Status changes to `failed` → Webhook called (error details provided) +- Status changes to `canceled` → Webhook called (cancellation confirmed) + +**Example: `input-required` webhook (human approval needed):** +```http +POST /webhooks/adcp/create_media_buy/agent_123/op_456 HTTP/1.1 +Host: buyer.example.com +Authorization: Bearer your-secret-token +Content-Type: application/json -```json { - "event_type": "task_status_changed", + "adcp_version": "1.6.0", + "status": "input-required", "task_id": "task_456", - "previous_status": "working", - "current_status": "completed", - "timestamp": "2025-01-22T10:25:00Z", - "task_type": "create_media_buy", - "domain": "media-buy", - "context": { - "buyer_ref": "nike_q1_2025" - }, - "result": { - // Included for completed tasks - "media_buy_id": "mb_987654321" - } + "buyer_ref": "nike_q1_campaign_2024", + "message": "Campaign budget $150K requires VP approval to proceed" } ``` +**Example: `completed` webhook (after approval granted - full create_media_buy response):** +```http +POST /webhooks/adcp/create_media_buy/agent_123/op_456 HTTP/1.1 +Host: buyer.example.com +Authorization: Bearer your-secret-token +Content-Type: application/json + +{ + "adcp_version": "1.6.0", + "status": "completed", + "media_buy_id": "mb_12345", + "buyer_ref": "nike_q1_campaign_2024", + "creative_deadline": "2024-01-30T23:59:59Z", + "packages": [ + { "package_id": "pkg_12345_001", "buyer_ref": "nike_ctv_package" } + ] +} +``` + +The webhook receives the **full response object** for each status, not just a notification. This means your webhook handler gets all the context and data needed to take appropriate action. + +### Webhook Handling Example + +```javascript +app.post('/webhooks/adcp/:task_type/:agent_id/:operation_id', async (req, res) => { + const { task_type, agent_id, operation_id } = req.params; + const response = req.body; + + // Webhooks are only called for 'submitted' operations + // So we only need to handle status changes that occur after submission + switch (response.status) { + case 'input-required': + // Alert human that input is needed + await notifyHuman({ + operation_id, + message: response.message, + context_id: response.context_id, + approval_data: response.data + }); + break; + + case 'completed': + // Process the completed operation + if (task_type === 'media_buy') { + await handleMediaBuyCreated({ + media_buy_id: response.media_buy_id, + buyer_ref: response.buyer_ref, + packages: response.packages, + creative_deadline: response.creative_deadline + }); + } + break; + + case 'failed': + // Handle failure + await handleOperationFailed({ + operation_id, + error: response.error, + message: response.message + }); + break; + + case 'canceled': + // Handle cancellation + await handleOperationCanceled(operation_id, response.message); + break; + } + + res.status(200).json({ status: 'processed' }); +}); +``` + ### Webhook Reliability **Important**: Webhooks use at-least-once delivery semantics and may be duplicated or arrive out of order. -See **[Core Concepts: Webhook Reliability](./core-concepts.md#webhook-reliability)** for detailed implementation guidance including idempotent handlers, sequence handling, security considerations, and polling as backup. +See **[Core Concepts: Webhook Reliability](./core-concepts.md#webhook-reliability)** for detailed implementation guidance including: +- Idempotent webhook handlers +- Sequence handling and out-of-order detection +- Security considerations (signature verification) +- Polling as backup mechanism +- Replay attack prevention ## Error Handling diff --git a/docs/signals/tasks/activate_signal.md b/docs/signals/tasks/activate_signal.md index 82ddf853ed..05c21f7e5f 100644 --- a/docs/signals/tasks/activate_signal.md +++ b/docs/signals/tasks/activate_signal.md @@ -185,10 +185,46 @@ data: {"status": {"state": "completed"}, "artifacts": [{ ``` ### Protocol Transport -- **MCP**: Returns task_id for polling-based asynchronous operation tracking +- **MCP**: Returns task_id for polling-based asynchronous operation tracking or webhook-based push notifications - **A2A**: Uses Server-Sent Events for real-time progress updates and completion - **Data Consistency**: Both protocols contain identical AdCP data structures and version information +### Webhook Support + +For long-running activations (when initial response is `submitted`), configure a webhook to receive the complete response when activation completes: + +```javascript +const response = await session.call('activate_signal', + { + signal_agent_segment_id: "luxury_auto_intenders", + platform: "the-trade-desk", + account: "agency-123-ttd" + }, + { + webhook_url: "https://buyer.com/webhooks/adcp/activate_signal/agent_id/op_id", + webhook_auth: { type: "bearer", credentials: "secret-token" } + } +); +``` + +When activation completes, you receive the full `activate_signal` response: + +```http +POST /webhooks/adcp/activate_signal/agent_id/op_id HTTP/1.1 +Content-Type: application/json +Authorization: Bearer secret-token + +{ + "adcp_version": "1.0.0", + "status": "deployed", + "task_id": "activation_789", + "decisioning_platform_segment_id": "ttd_agency123_lux_auto", + "deployed_at": "2025-01-15T14:30:00Z" +} +``` + +See **[Task Management: Webhook Integration](../../protocols/task-management.md#webhook-integration)** for complete details on webhook configuration and reliability. + ## Scenarios ### Initial Response (Pending) From 7bc3979f3b47d39ccd8bd752400ac69adef2f197 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 04:55:59 -0400 Subject: [PATCH 05/13] docs: clarify webhook behavior for async operations MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clarify that webhook security (HMAC-SHA256) is a core AdCP protocol requirement, not an optional implementation detail. **Key Change: Secure by Default** The problem with "recommended security": - Every integration becomes a negotiation - Different publishers implement different auth schemes - Default becomes "no security" because it's easier - Buyers must handle N different authentication methods - Result: Insecure by default The benefit of required security: - One standardized approach that always works - Publishers know exactly what to implement - Buyers know exactly what to expect - Secure by default across all implementations - Exception is "opt out of security" (conscious choice) **Schema Changes:** In `create-media-buy-request.json`: - Changed `auth_type` + `auth_token` → `secret` (required) - `secret` is shared HMAC-SHA256 key (min 32 chars) - Exchanged out-of-band during publisher onboarding - Used to generate X-ADCP-Signature header **Documentation Changes:** Added "Why Required" rationale to webhook security section: - Explains default-secure philosophy - Clarifies this is protocol requirement, not implementation guidance - Updates examples to use `secret` field instead of auth_type/token **Compliance:** Implementations without HMAC signature verification are non-compliant with AdCP specification. This applies to: - Reporting webhooks (media buy domain) - Task status webhooks (all domains) - Any future webhook functionality 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../media-buy/media-buys/optimization-reporting.md | 10 ++++++++-- docs/protocols/core-concepts.md | 11 +++++++++-- .../v1/media-buy/create-media-buy-request.json | 14 +++++--------- 3 files changed, 22 insertions(+), 13 deletions(-) diff --git a/docs/media-buy/media-buys/optimization-reporting.md b/docs/media-buy/media-buys/optimization-reporting.md index 0a4a4c0905..0ac0bdccaf 100644 --- a/docs/media-buy/media-buys/optimization-reporting.md +++ b/docs/media-buy/media-buys/optimization-reporting.md @@ -106,13 +106,19 @@ Configure reporting webhooks when creating a media buy using the `reporting_webh "packages": [...], "reporting_webhook": { "url": "https://buyer.example.com/webhooks/reporting", - "auth_type": "bearer", - "auth_token": "secret_token", + "secret": "shared_secret_min_32_chars_exchanged_out_of_band", "reporting_frequency": "daily" } } ``` +**Security is Required:** +- `secret` field is mandatory (minimum 32 characters) +- Publisher MUST send `X-ADCP-Signature` header with every webhook +- Buyer MUST verify signature before processing +- Exchanged out-of-band during publisher onboarding +- See [Webhook Security](../../protocols/core-concepts.md#security) for implementation details + ### Supported Frequencies Publishers declare supported reporting frequencies in the product's `reporting_capabilities`. Publishers are **not required** to support all frequencies - choose what makes operational sense for your platform. diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index d700ca2069..4978234c12 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -552,9 +552,16 @@ AdCP webhooks use **at-least-once delivery** semantics with the following charac ### Security -#### HMAC Signature Verification +#### HMAC Signature Verification (Required) -Publishers MUST sign all webhook payloads with HMAC-SHA256 to ensure message authenticity and integrity. Buyers MUST verify signatures before processing webhook data. +**All AdCP webhooks MUST use HMAC-SHA256 signature verification.** This is a core protocol requirement, not an optional feature. + +- Publishers MUST sign all webhook payloads +- Buyers MUST verify signatures before processing webhook data +- Implementations without signature verification are non-compliant + +**Why Required:** +Without mandated security, every integration becomes a negotiation. The default becomes "no security" because it's easier. By making HMAC verification required in the spec, we ensure secure by default across all implementations. **Signature Generation (Publisher):** ```javascript diff --git a/static/schemas/v1/media-buy/create-media-buy-request.json b/static/schemas/v1/media-buy/create-media-buy-request.json index 41f211cc1b..7c958d2b20 100644 --- a/static/schemas/v1/media-buy/create-media-buy-request.json +++ b/static/schemas/v1/media-buy/create-media-buy-request.json @@ -77,21 +77,17 @@ }, "reporting_webhook": { "type": "object", - "description": "Optional webhook configuration for automated reporting delivery", + "description": "Optional webhook configuration for automated reporting delivery. When configured, publisher MUST implement HMAC-SHA256 signature verification using the provided secret (see AdCP Webhook Security specification).", "properties": { "url": { "type": "string", "format": "uri", "description": "Webhook endpoint URL for reporting notifications" }, - "auth_type": { + "secret": { "type": "string", - "enum": ["bearer", "basic", "none"], - "description": "Authentication type for webhook requests" - }, - "auth_token": { - "type": "string", - "description": "Authentication token or credentials (format depends on auth_type)" + "description": "Shared secret for HMAC-SHA256 signature verification. Exchanged out-of-band during publisher onboarding. Publisher uses this to generate X-ADCP-Signature header on all webhook requests.", + "minLength": 32 }, "reporting_frequency": { "type": "string", @@ -108,7 +104,7 @@ "uniqueItems": true } }, - "required": ["url", "auth_type", "reporting_frequency"], + "required": ["url", "secret", "reporting_frequency"], "additionalProperties": false } }, From c36b9c47f773ec3903c776a8f79714eccf5516b3 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 05:00:55 -0400 Subject: [PATCH 06/13] feat: standardize webhook security across all AdCP webhooks MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Apply required HMAC-SHA256 security to ALL webhook configurations in AdCP, not just reporting webhooks. **Universal Webhook Security Model:** All webhook configurations now use: - `webhook_secret` (required, min 32 chars) - NOT `webhook_auth` - `X-ADCP-Signature` header with HMAC-SHA256 signature - `X-ADCP-Timestamp` header for replay protection - Constant-time signature verification **Changes Apply To:** 1. **Protocol-level task webhooks** (MCP, A2A) - `webhook_url` + `webhook_secret` parameters - Used for task status notifications (submitted → completed) - Applies to: create_media_buy, update_media_buy, sync_creatives, etc. 2. **Domain-specific webhooks** (e.g., reporting webhooks) - `reporting_webhook.secret` field - Used for scheduled reporting delivery - Applies to: media buy reporting **Before (inconsistent):** ```javascript // MCP task webhook { webhook_url: "...", webhook_auth: { type: "bearer", ... } } // Reporting webhook { url: "...", auth_type: "bearer", auth_token: "..." } // A2A push notification { webhook_url: "...", auth: { type: "bearer", ... } } ``` **After (standardized):** ```javascript // ALL webhooks use same model { webhook_url: "...", webhook_secret: "min_32_chars..." } { url: "...", secret: "min_32_chars..." } { webhook_url: "...", secret: "min_32_chars..." } ``` **Documentation Updates:** - core-concepts.md: Protocol-level webhook examples - mcp-guide.md: MCP-specific task webhook examples - protocol-comparison.md: Cross-protocol webhook examples - task-management.md: Task webhook configuration - optimization-reporting.md: Reporting webhook examples **Why This Matters:** As Brian said: "If it's clearly laid out in the spec, do it this way, then okay, maybe once in a while people will make an exception, but there's no exception. The default would be secure and best practice." One security model across ALL AdCP webhooks = secure by default everywhere. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/protocols/core-concepts.md | 10 +++++----- docs/protocols/mcp-guide.md | 4 ++-- docs/protocols/protocol-comparison.md | 6 +++--- docs/protocols/task-management.md | 2 +- 4 files changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index 4978234c12..bf1d9edf5c 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -287,19 +287,19 @@ class McpAdcpSession { } if (options.webhook_url) { request.webhook_url = options.webhook_url; - request.webhook_auth = options.webhook_auth; + request.webhook_secret = options.webhook_secret; // HMAC-SHA256 shared secret (required) } - + return await this.mcp.call(request); } } // Usage -const response = await session.call('create_media_buy', +const response = await session.call('create_media_buy', { /* task params */ }, { webhook_url: "https://buyer.com/webhooks/adcp", - webhook_auth: { type: "bearer", credentials: "secret" } + webhook_secret: "shared_secret_min_32_chars_for_hmac_verification" } ); ``` @@ -319,7 +319,7 @@ await a2a.send({ }, push_notification_config: { webhook_url: "https://buyer.com/webhooks/adcp", - auth: { type: "bearer", credentials: "secret" } + secret: "shared_secret_for_hmac_sha256_verification" // AdCP requires HMAC security } }); ``` diff --git a/docs/protocols/mcp-guide.md b/docs/protocols/mcp-guide.md index f95c2a781f..b7ec3fc95c 100644 --- a/docs/protocols/mcp-guide.md +++ b/docs/protocols/mcp-guide.md @@ -157,7 +157,7 @@ class McpAdcpSession { // Include webhook configuration (protocol-level) if (options.webhook_url) { request.webhook_url = options.webhook_url; - request.webhook_auth = options.webhook_auth; + request.webhook_secret = options.webhook_secret; // HMAC-SHA256 shared secret (required) } const response = await this.mcp.call(request); @@ -244,7 +244,7 @@ const response = await session.call('create_media_buy', }, { webhook_url: "https://buyer.com/webhooks/adcp", - webhook_auth: { type: "bearer", credentials: "secret_token" } + webhook_secret: "shared_secret_for_hmac_sha256_verification" } ); diff --git a/docs/protocols/protocol-comparison.md b/docs/protocols/protocol-comparison.md index 756c699bd6..40e405f2c5 100644 --- a/docs/protocols/protocol-comparison.md +++ b/docs/protocols/protocol-comparison.md @@ -114,7 +114,7 @@ const updates = await session.call('tasks/get', { // Optional: Configure webhook at protocol level const response = await session.call('create_media_buy', params, { webhook_url: "https://buyer.com/webhooks", - webhook_auth: { type: "bearer", credentials: "token" } + webhook_secret: "shared_secret_for_hmac_sha256_verification" }); ``` @@ -160,9 +160,9 @@ class McpAdcpSession { if (options.webhook_url) { request.webhook_url = options.webhook_url; - request.webhook_auth = options.webhook_auth; + request.webhook_secret = options.webhook_secret; // HMAC-SHA256 shared secret (required) } - + return await this.mcp.call(request); } } diff --git a/docs/protocols/task-management.md b/docs/protocols/task-management.md index 9edcb5659a..a34abe764b 100644 --- a/docs/protocols/task-management.md +++ b/docs/protocols/task-management.md @@ -456,7 +456,7 @@ const response = await session.call('create_media_buy', { /* task params */ }, { webhook_url: "https://buyer.com/webhooks/adcp/create_media_buy/agent_id/operation_id", - webhook_auth: { type: "bearer", credentials: "secret" } + webhook_secret: "shared_secret_for_hmac_sha256_verification" } ); ``` From 4659ee73116a1a8a932c35fc93c3a094d1f69caf Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 05:09:39 -0400 Subject: [PATCH 07/13] docs: align A2A webhook config with A2A protocol spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Clarify that A2A's push_notification_config structure is defined by the A2A protocol specification (not under our control), and document how AdCP's HMAC security requirement maps to A2A's native auth structure. **Key Insight:** A2A is an external protocol - we adapt to it, not the other way around. **A2A Protocol Structure (Fixed):** ```javascript push_notification_config: { webhook_url: "...", auth: { type: "bearer" | "basic" | "custom", data: { /* auth-specific data */ } } } ``` **AdCP's Approach:** Use A2A's `auth.type: "custom"` extensibility to pass HMAC secret: ```javascript push_notification_config: { webhook_url: "...", auth: { type: "custom", data: { hmac_secret: "..." } // AdCP security requirement } } ``` **Server Implementation:** Regardless of protocol (MCP or A2A), AdCP servers MUST: - Extract HMAC secret from protocol-specific location - Send X-ADCP-Signature and X-ADCP-Timestamp headers - Implement HMAC-SHA256 signature generation **Documentation Added:** New "Protocol Alignment" section explains: - MCP: AdCP adds webhook_secret parameter - A2A: AdCP maps to auth.type: "custom" - Both: Servers always send AdCP security headers This maintains AdCP's security requirements while respecting that A2A's structure is defined by an external specification we cannot change. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/protocols/core-concepts.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index bf1d9edf5c..538148a39a 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -307,6 +307,7 @@ const response = await session.call('create_media_buy', #### A2A Native Support ```javascript // A2A has native webhook support via PushNotificationConfig +// Note: A2A spec defines the structure - we map AdCP security to A2A's auth model await a2a.send({ message: { parts: [{ @@ -319,9 +320,14 @@ await a2a.send({ }, push_notification_config: { webhook_url: "https://buyer.com/webhooks/adcp", - secret: "shared_secret_for_hmac_sha256_verification" // AdCP requires HMAC security + auth: { + type: "custom", + // AdCP passes HMAC secret through A2A's auth.data + data: { hmac_secret: "shared_secret_for_hmac_sha256_verification" } + } } }); +// Server sends: X-ADCP-Signature and X-ADCP-Timestamp headers (AdCP requirement) ``` ### Server Decision on Webhook Usage @@ -563,6 +569,20 @@ AdCP webhooks use **at-least-once delivery** semantics with the following charac **Why Required:** Without mandated security, every integration becomes a negotiation. The default becomes "no security" because it's easier. By making HMAC verification required in the spec, we ensure secure by default across all implementations. +#### Protocol Alignment + +**MCP (Model Context Protocol):** +- AdCP defines `webhook_url` and `webhook_secret` as protocol-level parameters +- MCP doesn't specify webhook security - AdCP adds this requirement + +**A2A (Agent-to-Agent Protocol):** +- A2A defines `push_notification_config` structure (we can't change this) +- A2A supports `auth.type: "custom"` for extensibility +- AdCP maps its HMAC requirement into A2A's `auth.data.hmac_secret` +- Servers still send `X-ADCP-Signature` and `X-ADCP-Timestamp` headers + +**Result:** AdCP security requirements work across all protocols, adapting to each protocol's native structures. + **Signature Generation (Publisher):** ```javascript const crypto = require('crypto'); From b580931c098a4079f359e6e16e11d5d193ed7d59 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 05:43:00 -0400 Subject: [PATCH 08/13] refactor: simplify webhook security to Bearer token authentication MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace complex HMAC-SHA256 signature verification with simple Bearer token authentication. This simplification is appropriate because: 1. We control the AdCP protocol specification 2. TLS already provides transport security 3. Bearer tokens are simpler to implement and debug 4. Uniform auth structure across all webhook types Changes: - Unified webhook_config structure with auth.type and auth.token - Replace X-ADCP-Signature header with Authorization: Bearer - Standardize across protocol-level webhooks, reporting webhooks, and A2A - Update schema to require auth object instead of secret field - Update all documentation examples to use Bearer token pattern Net change: -123 lines (reduced complexity) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../media-buys/optimization-reporting.md | 13 +- docs/protocols/core-concepts.md | 133 +++++------------- docs/protocols/mcp-guide.md | 22 ++- docs/protocols/protocol-comparison.md | 22 ++- docs/protocols/task-management.md | 9 +- .../media-buy/create-media-buy-request.json | 25 +++- 6 files changed, 101 insertions(+), 123 deletions(-) diff --git a/docs/media-buy/media-buys/optimization-reporting.md b/docs/media-buy/media-buys/optimization-reporting.md index 0ac0bdccaf..3026b00804 100644 --- a/docs/media-buy/media-buys/optimization-reporting.md +++ b/docs/media-buy/media-buys/optimization-reporting.md @@ -106,17 +106,20 @@ Configure reporting webhooks when creating a media buy using the `reporting_webh "packages": [...], "reporting_webhook": { "url": "https://buyer.example.com/webhooks/reporting", - "secret": "shared_secret_min_32_chars_exchanged_out_of_band", + "auth": { + "type": "bearer", + "token": "secret_token_min_32_chars_exchanged_out_of_band" + }, "reporting_frequency": "daily" } } ``` **Security is Required:** -- `secret` field is mandatory (minimum 32 characters) -- Publisher MUST send `X-ADCP-Signature` header with every webhook -- Buyer MUST verify signature before processing -- Exchanged out-of-band during publisher onboarding +- `auth` configuration is mandatory with Bearer token (minimum 32 characters) +- Publisher MUST send `Authorization: Bearer ` header with every webhook +- Buyer MUST verify token before processing +- Token exchanged out-of-band during publisher onboarding - See [Webhook Security](../../protocols/core-concepts.md#security) for implementation details ### Supported Frequencies diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index 538148a39a..85bf9216be 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -558,122 +558,59 @@ AdCP webhooks use **at-least-once delivery** semantics with the following charac ### Security -#### HMAC Signature Verification (Required) +#### Webhook Authentication (Required) -**All AdCP webhooks MUST use HMAC-SHA256 signature verification.** This is a core protocol requirement, not an optional feature. +**All AdCP webhooks MUST use Bearer token authentication.** This is simple, standard, and secure. -- Publishers MUST sign all webhook payloads -- Buyers MUST verify signatures before processing webhook data -- Implementations without signature verification are non-compliant - -**Why Required:** -Without mandated security, every integration becomes a negotiation. The default becomes "no security" because it's easier. By making HMAC verification required in the spec, we ensure secure by default across all implementations. - -#### Protocol Alignment - -**MCP (Model Context Protocol):** -- AdCP defines `webhook_url` and `webhook_secret` as protocol-level parameters -- MCP doesn't specify webhook security - AdCP adds this requirement - -**A2A (Agent-to-Agent Protocol):** -- A2A defines `push_notification_config` structure (we can't change this) -- A2A supports `auth.type: "custom"` for extensibility -- AdCP maps its HMAC requirement into A2A's `auth.data.hmac_secret` -- Servers still send `X-ADCP-Signature` and `X-ADCP-Timestamp` headers - -**Result:** AdCP security requirements work across all protocols, adapting to each protocol's native structures. - -**Signature Generation (Publisher):** -```javascript -const crypto = require('crypto'); - -function signWebhook(payload, secret) { - const signature = crypto - .createHmac('sha256', secret) - .update(JSON.stringify(payload)) - .digest('hex'); - return signature; +**Configuration Structure:** +```json +{ + "webhook_config": { + "url": "https://buyer.example.com/webhooks/adcp", + "auth": { + "type": "bearer", + "token": "secret_token_min_32_chars" + } + } } +``` -// Add signature to request headers -const payload = { /* webhook data */ }; -const signature = signWebhook(payload, webhookSecret); - -axios.post(webhookUrl, payload, { +**Publisher Implementation:** +```javascript +// Send webhook with Authorization header +await axios.post(webhookConfig.url, payload, { headers: { 'Content-Type': 'application/json', - 'X-ADCP-Signature': `sha256=${signature}`, - 'X-ADCP-Timestamp': new Date().toISOString() + 'Authorization': `Bearer ${webhookConfig.auth.token}` } }); ``` -**Signature Verification (Buyer):** +**Buyer Implementation:** ```javascript -function verifyWebhookSignature(req, secret) { - const signature = req.headers['x-adcp-signature']; - const timestamp = req.headers['x-adcp-timestamp']; - - if (!signature || !timestamp) { - throw new Error('Missing signature or timestamp'); - } - - // Verify timestamp to prevent replay attacks (5 minute window) - const requestTime = new Date(timestamp).getTime(); - const now = Date.now(); - if (Math.abs(now - requestTime) > 5 * 60 * 1000) { - throw new Error('Webhook timestamp too old or in future'); - } - - // Extract hash from signature header - const [algorithm, hash] = signature.split('='); - if (algorithm !== 'sha256') { - throw new Error('Unsupported signature algorithm'); +app.post('/webhooks/adcp', async (req, res) => { + // Verify Authorization header + const authHeader = req.headers.authorization; + if (!authHeader || !authHeader.startsWith('Bearer ')) { + return res.status(401).json({ error: 'Missing Authorization header' }); } - // Compute expected signature - const expectedSignature = crypto - .createHmac('sha256', secret) - .update(JSON.stringify(req.body)) - .digest('hex'); - - // Constant-time comparison to prevent timing attacks - if (!crypto.timingSafeEqual( - Buffer.from(hash), - Buffer.from(expectedSignature) - )) { - throw new Error('Invalid webhook signature'); + const token = authHeader.substring(7); // Remove 'Bearer ' + if (token !== process.env.ADCP_WEBHOOK_TOKEN) { + return res.status(401).json({ error: 'Invalid token' }); } - return true; -} - -// Webhook endpoint -app.post('/webhooks/adcp', async (req, res) => { - try { - // ALWAYS verify signature first - verifyWebhookSignature(req, process.env.ADCP_WEBHOOK_SECRET); - - // Then process webhook - await processWebhook(req.body); - res.status(200).json({ status: 'processed' }); - } catch (error) { - console.error('Webhook verification failed:', error); - res.status(401).json({ error: 'Unauthorized' }); - } + // Process webhook + await processWebhook(req.body); + res.status(200).json({ status: 'processed' }); }); ``` -**Webhook Secret Management:** -- Secrets are exchanged out-of-band (e.g., during publisher onboarding) -- Use different secrets per buyer or per webhook endpoint -- Store secrets securely (environment variables, secret management systems) -- Support secret rotation without downtime (accept both old and new signatures during rotation) - -**Required Headers:** -- `X-ADCP-Signature`: HMAC signature in format `sha256={hex_signature}` -- `X-ADCP-Timestamp`: ISO 8601 timestamp of webhook generation -- `Content-Type`: Must be `application/json` +**Token Management:** +- Tokens exchanged out-of-band (during publisher onboarding) +- Minimum 32 characters +- Store securely (environment variables, secret management) +- Support rotation (accept old and new tokens during transition) ### Retry and Circuit Breaker Patterns diff --git a/docs/protocols/mcp-guide.md b/docs/protocols/mcp-guide.md index b7ec3fc95c..4c717ec192 100644 --- a/docs/protocols/mcp-guide.md +++ b/docs/protocols/mcp-guide.md @@ -155,9 +155,14 @@ class McpAdcpSession { } // Include webhook configuration (protocol-level) - if (options.webhook_url) { - request.webhook_url = options.webhook_url; - request.webhook_secret = options.webhook_secret; // HMAC-SHA256 shared secret (required) + if (options.webhook_config) { + request.webhook_config = { + url: options.webhook_config.url, + auth: { + type: 'bearer', + token: options.webhook_config.auth.token // Bearer token (required) + } + }; } const response = await this.mcp.call(request); @@ -236,15 +241,20 @@ const refined = await session.call('get_products', { #### Async Operations with Webhooks ```javascript // Create media buy with webhook configuration -const response = await session.call('create_media_buy', +const response = await session.call('create_media_buy', { buyer_ref: "nike_q1_2025", packages: [...], budget: { total: 150000, currency: "USD" } }, { - webhook_url: "https://buyer.com/webhooks/adcp", - webhook_secret: "shared_secret_for_hmac_sha256_verification" + webhook_config: { + url: "https://buyer.com/webhooks/adcp", + auth: { + type: "bearer", + token: "secret_token_min_32_chars_exchanged_out_of_band" + } + } } ); diff --git a/docs/protocols/protocol-comparison.md b/docs/protocols/protocol-comparison.md index 40e405f2c5..a0d02ef77a 100644 --- a/docs/protocols/protocol-comparison.md +++ b/docs/protocols/protocol-comparison.md @@ -113,8 +113,13 @@ const updates = await session.call('tasks/get', { // Optional: Configure webhook at protocol level const response = await session.call('create_media_buy', params, { - webhook_url: "https://buyer.com/webhooks", - webhook_secret: "shared_secret_for_hmac_sha256_verification" + webhook_config: { + url: "https://buyer.com/webhooks", + auth: { + type: "bearer", + token: "secret_token_min_32_chars" + } + } }); ``` @@ -157,10 +162,15 @@ Both protocols support webhooks but with different implementation approaches: class McpAdcpSession { async call(tool, params, options = {}) { const request = { tool, arguments: params }; - - if (options.webhook_url) { - request.webhook_url = options.webhook_url; - request.webhook_secret = options.webhook_secret; // HMAC-SHA256 shared secret (required) + + if (options.webhook_config) { + request.webhook_config = { + url: options.webhook_config.url, + auth: { + type: 'bearer', + token: options.webhook_config.auth.token // Bearer token (required) + } + }; } return await this.mcp.call(request); diff --git a/docs/protocols/task-management.md b/docs/protocols/task-management.md index a34abe764b..fb89a4c1a6 100644 --- a/docs/protocols/task-management.md +++ b/docs/protocols/task-management.md @@ -455,8 +455,13 @@ Configure webhooks at the protocol level when making async task calls. See **[Co const response = await session.call('create_media_buy', { /* task params */ }, { - webhook_url: "https://buyer.com/webhooks/adcp/create_media_buy/agent_id/operation_id", - webhook_secret: "shared_secret_for_hmac_sha256_verification" + webhook_config: { + url: "https://buyer.com/webhooks/adcp/create_media_buy/agent_id/operation_id", + auth: { + type: "bearer", + token: "secret_token_min_32_chars" + } + } } ); ``` diff --git a/static/schemas/v1/media-buy/create-media-buy-request.json b/static/schemas/v1/media-buy/create-media-buy-request.json index 7c958d2b20..5169c9a4ce 100644 --- a/static/schemas/v1/media-buy/create-media-buy-request.json +++ b/static/schemas/v1/media-buy/create-media-buy-request.json @@ -77,17 +77,30 @@ }, "reporting_webhook": { "type": "object", - "description": "Optional webhook configuration for automated reporting delivery. When configured, publisher MUST implement HMAC-SHA256 signature verification using the provided secret (see AdCP Webhook Security specification).", + "description": "Optional webhook configuration for automated reporting delivery. When configured, publisher MUST include Authorization header with Bearer token for authentication.", "properties": { "url": { "type": "string", "format": "uri", "description": "Webhook endpoint URL for reporting notifications" }, - "secret": { - "type": "string", - "description": "Shared secret for HMAC-SHA256 signature verification. Exchanged out-of-band during publisher onboarding. Publisher uses this to generate X-ADCP-Signature header on all webhook requests.", - "minLength": 32 + "auth": { + "type": "object", + "description": "Authentication configuration for webhook delivery", + "properties": { + "type": { + "type": "string", + "enum": ["bearer"], + "description": "Authentication type (currently only 'bearer' is supported)" + }, + "token": { + "type": "string", + "description": "Bearer token for authentication. Publisher will include 'Authorization: Bearer ' header with webhook requests. Minimum 32 characters recommended. Should be exchanged out-of-band (not in request payload in production).", + "minLength": 32 + } + }, + "required": ["type", "token"], + "additionalProperties": false }, "reporting_frequency": { "type": "string", @@ -104,7 +117,7 @@ "uniqueItems": true } }, - "required": ["url", "secret", "reporting_frequency"], + "required": ["url", "auth", "reporting_frequency"], "additionalProperties": false } }, From 24d112289b86c91e1a6d05fa014684d556675620 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 05:54:41 -0400 Subject: [PATCH 09/13] fix: correct A2A authentication structure to match protocol spec MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fix A2A push_notification_config to use the correct authentication structure according to A2A v0.2.5 specification: - Change from `auth: { type, credentials }` to `authentication: { schemes, credentials }` - Use `schemes: ["Bearer"]` (array format per A2A spec) - Update all A2A examples in core-concepts.md and protocol-comparison.md This ensures AdCP correctly maps to A2A's PushNotificationAuthenticationInfo which uses: ```typescript interface PushNotificationAuthenticationInfo { schemes: string[]; // Array of authentication scheme names credentials?: string; // Optional credentials string } ``` Reference: https://a2a-protocol.org/v0.2.5/specification/#69-pushnotificationauthenticationinfo-object 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- docs/protocols/core-concepts.md | 9 ++++----- docs/protocols/protocol-comparison.md | 10 ++++++++-- 2 files changed, 12 insertions(+), 7 deletions(-) diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index 85bf9216be..3cc6263096 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -320,14 +320,13 @@ await a2a.send({ }, push_notification_config: { webhook_url: "https://buyer.com/webhooks/adcp", - auth: { - type: "custom", - // AdCP passes HMAC secret through A2A's auth.data - data: { hmac_secret: "shared_secret_for_hmac_sha256_verification" } + authentication: { + schemes: ["Bearer"], + credentials: "secret_token_min_32_chars" } } }); -// Server sends: X-ADCP-Signature and X-ADCP-Timestamp headers (AdCP requirement) +// Server sends: Authorization: Bearer header (AdCP requirement) ``` ### Server Decision on Webhook Usage diff --git a/docs/protocols/protocol-comparison.md b/docs/protocols/protocol-comparison.md index a0d02ef77a..19f56f4059 100644 --- a/docs/protocols/protocol-comparison.md +++ b/docs/protocols/protocol-comparison.md @@ -145,7 +145,10 @@ await a2a.send({ message: { /* skill invocation */ }, push_notification_config: { webhook_url: "https://buyer.com/webhooks", - auth: { type: "bearer", credentials: "token" } + authentication: { + schemes: ["Bearer"], + credentials: "secret_token_min_32_chars" + } } }); ``` @@ -185,7 +188,10 @@ await a2a.send({ message: { /* task */ }, push_notification_config: { webhook_url: "https://buyer.com/webhooks", - auth: { type: "bearer", credentials: "token" }, + authentication: { + schemes: ["Bearer"], + credentials: "secret_token_min_32_chars" + }, events: ["state_change", "completion"] } }); From bde99bd5a01b67e4c9ab09df44e022345445b5bc Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 10:43:44 -0400 Subject: [PATCH 10/13] refactor: adopt A2A PushNotificationConfig as universal webhook structure MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replace custom webhook_config with A2A's PushNotificationConfig structure across all AdCP webhooks. This provides: 1. **Industry-standard structure**: Uses A2A v0.2.5 specification 2. **Zero protocol mapping**: Same structure works for MCP and A2A 3. **Multi-auth support**: schemes array handles Bearer, HMAC-SHA256, etc. 4. **Simpler implementation**: One webhook config format to learn Structure changes: - FROM: webhook_config { url, auth: { type, token } } - TO: push_notification_config { url, authentication: { schemes, credentials } } Supported authentication schemes: - Bearer: Simple token auth for development - HMAC-SHA256: Signature verification for production (prevents replay attacks) Benefits: - Consistent developer experience across protocols - Natural multi-auth support via schemes array - Future-proof for JWT, mTLS, or other auth types - Eliminates protocol-specific webhook configuration Reference: https://a2a-protocol.org/v0.2.5/specification/ 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../media-buys/optimization-reporting.md | 30 +++- docs/protocols/core-concepts.md | 163 ++++++++++++++---- docs/protocols/mcp-guide.md | 20 +-- docs/protocols/protocol-comparison.md | 38 ++-- docs/protocols/task-management.md | 8 +- .../media-buy/create-media-buy-request.json | 27 +-- 6 files changed, 196 insertions(+), 90 deletions(-) diff --git a/docs/media-buy/media-buys/optimization-reporting.md b/docs/media-buy/media-buys/optimization-reporting.md index 3026b00804..e8643e2463 100644 --- a/docs/media-buy/media-buys/optimization-reporting.md +++ b/docs/media-buy/media-buys/optimization-reporting.md @@ -106,9 +106,25 @@ Configure reporting webhooks when creating a media buy using the `reporting_webh "packages": [...], "reporting_webhook": { "url": "https://buyer.example.com/webhooks/reporting", - "auth": { - "type": "bearer", - "token": "secret_token_min_32_chars_exchanged_out_of_band" + "authentication": { + "schemes": ["Bearer"], + "credentials": "secret_token_min_32_chars" + }, + "reporting_frequency": "daily" + } +} +``` + +**Or with HMAC signature (recommended for production):** +```json +{ + "buyer_ref": "campaign_2024", + "packages": [...], + "reporting_webhook": { + "url": "https://buyer.example.com/webhooks/reporting", + "authentication": { + "schemes": ["HMAC-SHA256"], + "credentials": "shared_secret_min_32_chars" }, "reporting_frequency": "daily" } @@ -116,10 +132,10 @@ Configure reporting webhooks when creating a media buy using the `reporting_webh ``` **Security is Required:** -- `auth` configuration is mandatory with Bearer token (minimum 32 characters) -- Publisher MUST send `Authorization: Bearer ` header with every webhook -- Buyer MUST verify token before processing -- Token exchanged out-of-band during publisher onboarding +- `authentication` configuration is mandatory (minimum 32 characters) +- **Bearer tokens**: Simple, good for development (Authorization header) +- **HMAC-SHA256**: Production-recommended, prevents replay attacks (signature headers) +- Credentials exchanged out-of-band during publisher onboarding - See [Webhook Security](../../protocols/core-concepts.md#security) for implementation details ### Supported Frequencies diff --git a/docs/protocols/core-concepts.md b/docs/protocols/core-concepts.md index 3cc6263096..a3c30a86ea 100644 --- a/docs/protocols/core-concepts.md +++ b/docs/protocols/core-concepts.md @@ -280,26 +280,46 @@ class McpAdcpSession { tool: tool, arguments: params }; - + // Protocol-level extensions (like context_id) if (this.contextId) { request.context_id = this.contextId; } - if (options.webhook_url) { - request.webhook_url = options.webhook_url; - request.webhook_secret = options.webhook_secret; // HMAC-SHA256 shared secret (required) + + // Use A2A-compatible push_notification_config + if (options.push_notification_config) { + request.push_notification_config = options.push_notification_config; } return await this.mcp.call(request); } } -// Usage +// Usage (Bearer token) const response = await session.call('create_media_buy', { /* task params */ }, { - webhook_url: "https://buyer.com/webhooks/adcp", - webhook_secret: "shared_secret_min_32_chars_for_hmac_verification" + push_notification_config: { + url: "https://buyer.com/webhooks/adcp", + authentication: { + schemes: ["Bearer"], + credentials: "secret_token_32_chars" + } + } + } +); + +// Usage (HMAC signature - recommended for production) +const response = await session.call('create_media_buy', + { /* task params */ }, + { + push_notification_config: { + url: "https://buyer.com/webhooks/adcp", + authentication: { + schemes: ["HMAC-SHA256"], + credentials: "shared_secret_32_chars" + } + } } ); ``` @@ -307,7 +327,7 @@ const response = await session.call('create_media_buy', #### A2A Native Support ```javascript // A2A has native webhook support via PushNotificationConfig -// Note: A2A spec defines the structure - we map AdCP security to A2A's auth model +// AdCP uses the same structure - no mapping needed! await a2a.send({ message: { parts: [{ @@ -319,14 +339,13 @@ await a2a.send({ }] }, push_notification_config: { - webhook_url: "https://buyer.com/webhooks/adcp", + url: "https://buyer.com/webhooks/adcp", authentication: { - schemes: ["Bearer"], - credentials: "secret_token_min_32_chars" + schemes: ["HMAC-SHA256"], // or ["Bearer"] + credentials: "shared_secret_32_chars" } } }); -// Server sends: Authorization: Bearer header (AdCP requirement) ``` ### Server Decision on Webhook Usage @@ -559,57 +578,133 @@ AdCP webhooks use **at-least-once delivery** semantics with the following charac #### Webhook Authentication (Required) -**All AdCP webhooks MUST use Bearer token authentication.** This is simple, standard, and secure. +**AdCP adopts A2A's PushNotificationConfig structure** for webhook configuration. This provides a standard, flexible authentication model that supports multiple security schemes. -**Configuration Structure:** +**Configuration Structure (A2A-Compatible):** ```json { - "webhook_config": { + "push_notification_config": { "url": "https://buyer.example.com/webhooks/adcp", - "auth": { - "type": "bearer", - "token": "secret_token_min_32_chars" + "authentication": { + "schemes": ["Bearer"], + "credentials": "secret_token_min_32_chars" } } } ``` -**Publisher Implementation:** +**Supported Authentication Schemes:** + +1. **Bearer Token (Simple, Recommended for Development)** + ```json + { + "authentication": { + "schemes": ["Bearer"], + "credentials": "secret_token_32_chars" + } + } + ``` + +2. **HMAC Signature (Enterprise, Recommended for Production)** + ```json + { + "authentication": { + "schemes": ["HMAC-SHA256"], + "credentials": "shared_secret_32_chars" + } + } + ``` + +**Publisher Implementation (Bearer):** ```javascript -// Send webhook with Authorization header -await axios.post(webhookConfig.url, payload, { - headers: { - 'Content-Type': 'application/json', - 'Authorization': `Bearer ${webhookConfig.auth.token}` - } -}); +const config = pushNotificationConfig; +const scheme = config.authentication.schemes[0]; + +if (scheme === 'Bearer') { + await axios.post(config.url, payload, { + headers: { + 'Content-Type': 'application/json', + 'Authorization': `Bearer ${config.authentication.credentials}` + } + }); +} ``` -**Buyer Implementation:** +**Publisher Implementation (HMAC-SHA256):** +```javascript +if (scheme === 'HMAC-SHA256') { + const timestamp = new Date().toISOString(); + const signature = crypto + .createHmac('sha256', config.authentication.credentials) + .update(timestamp + JSON.stringify(payload)) + .digest('hex'); + + await axios.post(config.url, payload, { + headers: { + 'Content-Type': 'application/json', + 'X-ADCP-Signature': `sha256=${signature}`, + 'X-ADCP-Timestamp': timestamp + } + }); +} +``` + +**Buyer Implementation (Bearer):** ```javascript app.post('/webhooks/adcp', async (req, res) => { - // Verify Authorization header const authHeader = req.headers.authorization; if (!authHeader || !authHeader.startsWith('Bearer ')) { return res.status(401).json({ error: 'Missing Authorization header' }); } - const token = authHeader.substring(7); // Remove 'Bearer ' + const token = authHeader.substring(7); if (token !== process.env.ADCP_WEBHOOK_TOKEN) { return res.status(401).json({ error: 'Invalid token' }); } - // Process webhook await processWebhook(req.body); res.status(200).json({ status: 'processed' }); }); ``` -**Token Management:** -- Tokens exchanged out-of-band (during publisher onboarding) -- Minimum 32 characters +**Buyer Implementation (HMAC-SHA256):** +```javascript +app.post('/webhooks/adcp', async (req, res) => { + const signature = req.headers['x-adcp-signature']; + const timestamp = req.headers['x-adcp-timestamp']; + + if (!signature || !timestamp) { + return res.status(401).json({ error: 'Missing signature headers' }); + } + + // Reject old webhooks (prevent replay attacks) + const eventTime = new Date(timestamp); + if (Date.now() - eventTime > 5 * 60 * 1000) { + return res.status(401).json({ error: 'Webhook too old' }); + } + + // Verify signature + const expectedSig = crypto + .createHmac('sha256', process.env.ADCP_WEBHOOK_SECRET) + .update(timestamp + JSON.stringify(req.body)) + .digest('hex'); + + if (signature !== `sha256=${expectedSig}`) { + return res.status(401).json({ error: 'Invalid signature' }); + } + + await processWebhook(req.body); + res.status(200).json({ status: 'processed' }); +}); +``` + +**Authentication Best Practices:** +- **Bearer tokens**: Simple, good for development and testing +- **HMAC signatures**: Prevents replay attacks, recommended for production +- Credentials exchanged out-of-band (during publisher onboarding) +- Minimum 32 characters for all credentials - Store securely (environment variables, secret management) -- Support rotation (accept old and new tokens during transition) +- Support credential rotation (accept old and new during transition) ### Retry and Circuit Breaker Patterns diff --git a/docs/protocols/mcp-guide.md b/docs/protocols/mcp-guide.md index 4c717ec192..eb972bb8a4 100644 --- a/docs/protocols/mcp-guide.md +++ b/docs/protocols/mcp-guide.md @@ -154,15 +154,9 @@ class McpAdcpSession { request.context_id = this.contextId; } - // Include webhook configuration (protocol-level) - if (options.webhook_config) { - request.webhook_config = { - url: options.webhook_config.url, - auth: { - type: 'bearer', - token: options.webhook_config.auth.token // Bearer token (required) - } - }; + // Include webhook configuration (protocol-level, A2A-compatible) + if (options.push_notification_config) { + request.push_notification_config = options.push_notification_config; } const response = await this.mcp.call(request); @@ -248,11 +242,11 @@ const response = await session.call('create_media_buy', budget: { total: 150000, currency: "USD" } }, { - webhook_config: { + push_notification_config: { url: "https://buyer.com/webhooks/adcp", - auth: { - type: "bearer", - token: "secret_token_min_32_chars_exchanged_out_of_band" + authentication: { + schemes: ["HMAC-SHA256"], // or ["Bearer"] for simple auth + credentials: "shared_secret_32_chars" } } } diff --git a/docs/protocols/protocol-comparison.md b/docs/protocols/protocol-comparison.md index 19f56f4059..f3b65214ad 100644 --- a/docs/protocols/protocol-comparison.md +++ b/docs/protocols/protocol-comparison.md @@ -111,13 +111,13 @@ const updates = await session.call('tasks/get', { include_result: true }); -// Optional: Configure webhook at protocol level +// Optional: Configure webhook at protocol level (A2A-compatible structure) const response = await session.call('create_media_buy', params, { - webhook_config: { + push_notification_config: { url: "https://buyer.com/webhooks", - auth: { - type: "bearer", - token: "secret_token_min_32_chars" + authentication: { + schemes: ["HMAC-SHA256"], // or ["Bearer"] + credentials: "shared_secret_32_chars" } } }); @@ -161,19 +161,14 @@ Both protocols support webhooks but with different implementation approaches: #### MCP: Protocol Wrapper Extension ```javascript -// Webhook config at protocol level (like context_id) +// AdCP uses A2A-compatible structure for MCP as well class McpAdcpSession { async call(tool, params, options = {}) { const request = { tool, arguments: params }; - if (options.webhook_config) { - request.webhook_config = { - url: options.webhook_config.url, - auth: { - type: 'bearer', - token: options.webhook_config.auth.token // Bearer token (required) - } - }; + // Same structure as A2A - no mapping needed + if (options.push_notification_config) { + request.push_notification_config = options.push_notification_config; } return await this.mcp.call(request); @@ -181,22 +176,23 @@ class McpAdcpSession { } ``` -#### A2A: Native Push Notifications +#### A2A: Native Push Notifications ```javascript -// Built-in PushNotificationConfig +// Built-in PushNotificationConfig - AdCP uses this structure universally await a2a.send({ message: { /* task */ }, push_notification_config: { - webhook_url: "https://buyer.com/webhooks", + url: "https://buyer.com/webhooks", authentication: { - schemes: ["Bearer"], - credentials: "secret_token_min_32_chars" - }, - events: ["state_change", "completion"] + schemes: ["HMAC-SHA256"], // or ["Bearer"] + credentials: "shared_secret_32_chars" + } } }); ``` +**Key Insight:** AdCP adopts A2A's `PushNotificationConfig` structure as the universal webhook configuration format across all protocols. This eliminates protocol-specific mapping and provides a consistent developer experience. + ### Task Management Both protocols now provide equivalent task management capabilities: diff --git a/docs/protocols/task-management.md b/docs/protocols/task-management.md index fb89a4c1a6..fec0284924 100644 --- a/docs/protocols/task-management.md +++ b/docs/protocols/task-management.md @@ -455,11 +455,11 @@ Configure webhooks at the protocol level when making async task calls. See **[Co const response = await session.call('create_media_buy', { /* task params */ }, { - webhook_config: { + push_notification_config: { url: "https://buyer.com/webhooks/adcp/create_media_buy/agent_id/operation_id", - auth: { - type: "bearer", - token: "secret_token_min_32_chars" + authentication: { + schemes: ["HMAC-SHA256"], // or ["Bearer"] for simple auth + credentials: "shared_secret_32_chars" } } } diff --git a/static/schemas/v1/media-buy/create-media-buy-request.json b/static/schemas/v1/media-buy/create-media-buy-request.json index 5169c9a4ce..9385e1377f 100644 --- a/static/schemas/v1/media-buy/create-media-buy-request.json +++ b/static/schemas/v1/media-buy/create-media-buy-request.json @@ -77,29 +77,34 @@ }, "reporting_webhook": { "type": "object", - "description": "Optional webhook configuration for automated reporting delivery. When configured, publisher MUST include Authorization header with Bearer token for authentication.", + "description": "Optional webhook configuration for automated reporting delivery using A2A-compatible PushNotificationConfig structure. Supports Bearer tokens (simple) or HMAC signatures (production-recommended).", "properties": { "url": { "type": "string", "format": "uri", "description": "Webhook endpoint URL for reporting notifications" }, - "auth": { + "authentication": { "type": "object", - "description": "Authentication configuration for webhook delivery", + "description": "Authentication configuration for webhook delivery (A2A-compatible)", "properties": { - "type": { - "type": "string", - "enum": ["bearer"], - "description": "Authentication type (currently only 'bearer' is supported)" + "schemes": { + "type": "array", + "description": "Array of authentication schemes. Supported: ['Bearer'] for simple token auth, ['HMAC-SHA256'] for signature verification (recommended for production)", + "items": { + "type": "string", + "enum": ["Bearer", "HMAC-SHA256"] + }, + "minItems": 1, + "maxItems": 1 }, - "token": { + "credentials": { "type": "string", - "description": "Bearer token for authentication. Publisher will include 'Authorization: Bearer ' header with webhook requests. Minimum 32 characters recommended. Should be exchanged out-of-band (not in request payload in production).", + "description": "Credentials for authentication. For Bearer: token sent in Authorization header. For HMAC-SHA256: shared secret used to generate signature. Minimum 32 characters. Exchanged out-of-band during onboarding.", "minLength": 32 } }, - "required": ["type", "token"], + "required": ["schemes", "credentials"], "additionalProperties": false }, "reporting_frequency": { @@ -117,7 +122,7 @@ "uniqueItems": true } }, - "required": ["url", "auth", "reporting_frequency"], + "required": ["url", "authentication", "reporting_frequency"], "additionalProperties": false } }, From 75120f0c2dcbdaa83f2e040d820674da52fce366 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 11:35:35 -0400 Subject: [PATCH 11/13] refactor: extract reusable push-notification-config schema MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Create shared core schema for webhook configuration that can be reused across any task-specific webhooks (reporting, creative sync, etc.). Changes: - New: static/schemas/v1/core/push-notification-config.json - Updated: create-media-buy-request.json uses allOf + $ref pattern - reporting_webhook extends base push_notification_config with reporting-specific fields (reporting_frequency, requested_metrics) Benefits: - Single source of truth for webhook authentication structure - Easy to add webhooks to other async tasks (sync_creatives, etc.) - Maintains A2A PushNotificationConfig compatibility - DRY principle for webhook configuration Note: Protocol-level push_notification_config (for task status notifications) is handled at MCP/A2A wrapper level, not in task schemas. This schema is for task-specific business webhooks. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../v1/core/push-notification-config.json | 44 +++++++++++++++ .../media-buy/create-media-buy-request.json | 54 ++++++------------- 2 files changed, 60 insertions(+), 38 deletions(-) create mode 100644 static/schemas/v1/core/push-notification-config.json diff --git a/static/schemas/v1/core/push-notification-config.json b/static/schemas/v1/core/push-notification-config.json new file mode 100644 index 0000000000..82ab52851b --- /dev/null +++ b/static/schemas/v1/core/push-notification-config.json @@ -0,0 +1,44 @@ +{ + "$schema": "http://json-schema.org/draft-07/schema#", + "$id": "/schemas/v1/core/push-notification-config.json", + "title": "Push Notification Config", + "description": "Webhook configuration for asynchronous task notifications. Uses A2A-compatible PushNotificationConfig structure. Supports Bearer tokens (simple) or HMAC signatures (production-recommended).", + "type": "object", + "properties": { + "url": { + "type": "string", + "format": "uri", + "description": "Webhook endpoint URL for task status notifications" + }, + "token": { + "type": "string", + "description": "Optional client-provided token for webhook validation. Echoed back in webhook payload to validate request authenticity.", + "minLength": 16 + }, + "authentication": { + "type": "object", + "description": "Authentication configuration for webhook delivery (A2A-compatible)", + "properties": { + "schemes": { + "type": "array", + "description": "Array of authentication schemes. Supported: ['Bearer'] for simple token auth, ['HMAC-SHA256'] for signature verification (recommended for production)", + "items": { + "type": "string", + "enum": ["Bearer", "HMAC-SHA256"] + }, + "minItems": 1, + "maxItems": 1 + }, + "credentials": { + "type": "string", + "description": "Credentials for authentication. For Bearer: token sent in Authorization header. For HMAC-SHA256: shared secret used to generate signature. Minimum 32 characters. Exchanged out-of-band during onboarding.", + "minLength": 32 + } + }, + "required": ["schemes", "credentials"], + "additionalProperties": false + } + }, + "required": ["url", "authentication"], + "additionalProperties": false +} diff --git a/static/schemas/v1/media-buy/create-media-buy-request.json b/static/schemas/v1/media-buy/create-media-buy-request.json index 9385e1377f..ced19ac264 100644 --- a/static/schemas/v1/media-buy/create-media-buy-request.json +++ b/static/schemas/v1/media-buy/create-media-buy-request.json @@ -76,54 +76,32 @@ "$ref": "/schemas/v1/core/budget.json" }, "reporting_webhook": { - "type": "object", - "description": "Optional webhook configuration for automated reporting delivery using A2A-compatible PushNotificationConfig structure. Supports Bearer tokens (simple) or HMAC signatures (production-recommended).", - "properties": { - "url": { - "type": "string", - "format": "uri", - "description": "Webhook endpoint URL for reporting notifications" + "allOf": [ + { + "$ref": "/schemas/v1/core/push-notification-config.json" }, - "authentication": { + { "type": "object", - "description": "Authentication configuration for webhook delivery (A2A-compatible)", + "description": "Optional webhook configuration for automated reporting delivery. Uses push_notification_config structure with additional reporting-specific fields.", "properties": { - "schemes": { + "reporting_frequency": { + "type": "string", + "enum": ["hourly", "daily", "monthly"], + "description": "Frequency for automated reporting delivery. Must be supported by all products in the media buy." + }, + "requested_metrics": { "type": "array", - "description": "Array of authentication schemes. Supported: ['Bearer'] for simple token auth, ['HMAC-SHA256'] for signature verification (recommended for production)", + "description": "Optional list of metrics to include in webhook notifications. If omitted, all available metrics are included. Must be subset of product's available_metrics.", "items": { "type": "string", - "enum": ["Bearer", "HMAC-SHA256"] + "enum": ["impressions", "spend", "clicks", "ctr", "video_completions", "completion_rate", "conversions", "viewability", "engagement_rate"] }, - "minItems": 1, - "maxItems": 1 - }, - "credentials": { - "type": "string", - "description": "Credentials for authentication. For Bearer: token sent in Authorization header. For HMAC-SHA256: shared secret used to generate signature. Minimum 32 characters. Exchanged out-of-band during onboarding.", - "minLength": 32 + "uniqueItems": true } }, - "required": ["schemes", "credentials"], - "additionalProperties": false - }, - "reporting_frequency": { - "type": "string", - "enum": ["hourly", "daily", "monthly"], - "description": "Frequency for automated reporting delivery. Must be supported by all products in the media buy." - }, - "requested_metrics": { - "type": "array", - "description": "Optional list of metrics to include in webhook notifications. If omitted, all available metrics are included. Must be subset of product's available_metrics.", - "items": { - "type": "string", - "enum": ["impressions", "spend", "clicks", "ctr", "video_completions", "completion_rate", "conversions", "viewability", "engagement_rate"] - }, - "uniqueItems": true + "required": ["reporting_frequency"] } - }, - "required": ["url", "authentication", "reporting_frequency"], - "additionalProperties": false + ] } }, "required": ["buyer_ref", "packages", "promoted_offering", "start_time", "end_time", "budget"], From 3bf09f62990c7470669151c6788546af461fc4bb Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 11:45:41 -0400 Subject: [PATCH 12/13] feat: add webhook support to async task requests MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add optional push_notification_config to update_media_buy and sync_creatives task requests for task-specific async notifications. Schema changes: - update-media-buy-request.json: Add push_notification_config field - sync-creatives-request.json: Add push_notification_config field - Both reference core/push-notification-config.json for consistency Documentation updates: - update_media_buy.md: Add webhook configuration section - sync_creatives.md: Add webhook configuration section - Clarify when webhooks are triggered (working→completed, submitted→completed) Task-specific vs Protocol-level webhooks: - Task-specific: Passed in task params (this commit) - update_media_buy, sync_creatives, reporting_webhook - Notify about specific task completion - Protocol-level: Passed in MCP/A2A wrapper options - Generic task status notifications - Works for ALL async tasks Benefits: - Consistent webhook structure across all async tasks - Easy to add webhooks to future long-running operations - Clear documentation for when/why to use task-specific webhooks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- .../task-reference/sync_creatives.md | 29 +++++++++++++++++ .../task-reference/update_media_buy.md | 32 +++++++++++++++++++ .../v1/media-buy/sync-creatives-request.json | 4 +++ .../media-buy/update-media-buy-request.json | 4 +++ 4 files changed, 69 insertions(+) diff --git a/docs/media-buy/task-reference/sync_creatives.md b/docs/media-buy/task-reference/sync_creatives.md index eef2af9fbb..5c11365e7c 100644 --- a/docs/media-buy/task-reference/sync_creatives.md +++ b/docs/media-buy/task-reference/sync_creatives.md @@ -32,6 +32,7 @@ The `sync_creatives` task provides a powerful, efficient approach to creative li | `patch` | boolean | No | Partial update mode (default: false) | | `dry_run` | boolean | No | Preview changes without applying (default: false) | | `validation_mode` | enum | No | Validation strictness: "strict" or "lenient" (default: "strict") | +| `push_notification_config` | PushNotificationConfig | No | Optional webhook for async sync notifications (see Webhook Configuration below) | ### Assignment Management @@ -60,6 +61,34 @@ Each creative in the `creatives` array follows the [Creative Asset schema](/sche - `snippet_type: "html"` - `assets` array with sub-assets for template variables +## Webhook Configuration (Task-Specific) + +For large bulk operations or creative approval workflows, you can provide a task-specific webhook to be notified when the sync completes: + +```json +{ + "creatives": [/* up to 100 creatives */], + "push_notification_config": { + "url": "https://buyer.com/webhooks/creative-sync", + "authentication": { + "schemes": ["HMAC-SHA256"], + "credentials": "shared_secret_32_chars" + } + } +} +``` + +**When webhooks are sent:** +- Bulk sync takes longer than ~120 seconds (status: `working` → `completed`) +- Creative approval required (status: `submitted` → `completed`) +- Large creative uploads processing asynchronously + +**Webhook payload:** +- Complete sync_creatives response with summary and results +- Includes action taken for each creative (created/updated/unchanged/failed) + +See [Webhook Security](../../protocols/core-concepts.md#security) for authentication details. + ## Response Format The response provides comprehensive details about the sync operation: diff --git a/docs/media-buy/task-reference/update_media_buy.md b/docs/media-buy/task-reference/update_media_buy.md index 4957c0cf94..5538e1b213 100644 --- a/docs/media-buy/task-reference/update_media_buy.md +++ b/docs/media-buy/task-reference/update_media_buy.md @@ -24,6 +24,7 @@ Update campaign and package settings. This task supports partial updates and han | `end_time` | string | No | New end date/time in ISO 8601 format (UTC unless timezone specified) | | `budget` | Budget | No | New budget configuration (see Budget Object in create_media_buy) | | `packages` | PackageUpdate[] | No | Package-specific updates (see Package Update Object below) | +| `push_notification_config` | PushNotificationConfig | No | Optional webhook for async update notifications (see Webhook Configuration below) | *Either `media_buy_id` or `buyer_ref` must be provided @@ -40,6 +41,37 @@ Update campaign and package settings. This task supports partial updates and han *Either `package_id` or `buyer_ref` must be provided +## Webhook Configuration (Task-Specific) + +For long-running updates (typically requiring approval workflows), you can provide a task-specific webhook to be notified when the update completes: + +```json +{ + "buyer_ref": "nike_q1_campaign_2024", + "budget": { + "total": 150000, + "currency": "USD" + }, + "push_notification_config": { + "url": "https://buyer.com/webhooks/media-buy-updates", + "authentication": { + "schemes": ["HMAC-SHA256"], + "credentials": "shared_secret_32_chars" + } + } +} +``` + +**When webhooks are sent:** +- Update requires manual approval (status: `submitted` → `completed`) +- Update takes longer than ~120 seconds (status: `working` → `completed`) + +**Webhook payload:** +- Complete update_media_buy response with final status +- Includes media_buy_id, affected_packages, and implementation_date + +See [Webhook Security](../../protocols/core-concepts.md#security) for authentication details. + ## Response (Message) The response includes a human-readable message that: diff --git a/static/schemas/v1/media-buy/sync-creatives-request.json b/static/schemas/v1/media-buy/sync-creatives-request.json index 3b7f4f8197..d2e52e3a28 100644 --- a/static/schemas/v1/media-buy/sync-creatives-request.json +++ b/static/schemas/v1/media-buy/sync-creatives-request.json @@ -53,6 +53,10 @@ "enum": ["strict", "lenient"], "default": "strict", "description": "Validation strictness. 'strict' fails entire sync on any validation error. 'lenient' processes valid creatives and reports errors." + }, + "push_notification_config": { + "$ref": "/schemas/v1/core/push-notification-config.json", + "description": "Optional webhook configuration for async sync notifications. Publisher will send webhook when sync completes if operation takes longer than immediate response time (typically for large bulk operations)." } }, "required": ["creatives"], diff --git a/static/schemas/v1/media-buy/update-media-buy-request.json b/static/schemas/v1/media-buy/update-media-buy-request.json index 955e35340f..c0db0a30e7 100644 --- a/static/schemas/v1/media-buy/update-media-buy-request.json +++ b/static/schemas/v1/media-buy/update-media-buy-request.json @@ -78,6 +78,10 @@ ], "additionalProperties": false } + }, + "push_notification_config": { + "$ref": "/schemas/v1/core/push-notification-config.json", + "description": "Optional webhook configuration for async update notifications. Publisher will send webhook when update completes if operation takes longer than immediate response time." } }, "oneOf": [ From c1172c77102c98a5a6444199643c8f120ef7ed20 Mon Sep 17 00:00:00 2001 From: Brian O'Kelley Date: Thu, 9 Oct 2025 12:20:17 -0400 Subject: [PATCH 13/13] docs: clarify webhook triggers include HITL approval MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Update sync_creatives push_notification_config description to mention manual approval/HITL as a webhook trigger, not just large bulk operations. Webhooks are sent when: - Large bulk operations take longer than ~120 seconds - Manual approval required (Human-in-the-Loop workflows) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude --- static/schemas/v1/media-buy/sync-creatives-request.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/static/schemas/v1/media-buy/sync-creatives-request.json b/static/schemas/v1/media-buy/sync-creatives-request.json index d2e52e3a28..a6bfc20dd3 100644 --- a/static/schemas/v1/media-buy/sync-creatives-request.json +++ b/static/schemas/v1/media-buy/sync-creatives-request.json @@ -56,7 +56,7 @@ }, "push_notification_config": { "$ref": "/schemas/v1/core/push-notification-config.json", - "description": "Optional webhook configuration for async sync notifications. Publisher will send webhook when sync completes if operation takes longer than immediate response time (typically for large bulk operations)." + "description": "Optional webhook configuration for async sync notifications. Publisher will send webhook when sync completes if operation takes longer than immediate response time (typically for large bulk operations or manual approval/HITL)." } }, "required": ["creatives"],