Skip to content

feat(catalog): project publisher_properties fan-out into catalog_agent_authorizations (#4841)#4879

Merged
bokelley merged 1 commit into
mainfrom
bokelley/4841-catalog-fanout-projection
May 20, 2026
Merged

feat(catalog): project publisher_properties fan-out into catalog_agent_authorizations (#4841)#4879
bokelley merged 1 commit into
mainfrom
bokelley/4841-catalog-fanout-projection

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

Closes the catalog snapshot gap from #4840's rollout. Adds a new adagents_authoritative evidence value to catalog_agent_authorizations, lifts the cross-publisher refusal specifically for the fan-out shape, and backfills the ~6,800 existing cafemedia rows. /registry/authorizations and /registry/authorizations/snapshot (partner-sync endpoints, catalog-only readers) now see the manager-asserted children.

Why the cross-publisher guard stays

publisher-db.ts:968 refuses to project catalog_agent_authorizations rows where the selector's publisher_domain ≠ the publisher being projected. This is load-bearing for catalog trust — without it, a hostile manager file could project authoritative rows for publishers that didn't authorize anything. Don't lift the guard.

Instead, give the fan-out its own evidence value:

Evidence Source Trust
adagents_json publisher's own /.well-known/adagents.json strongest — bilateral
agent_claim sales-agent list_authorized_properties lower — unverified peer
community AAO moderator curation curated
adagents_authoritative (new) manager file (e.g. cafemedia) names this publisher in publisher_properties[].publisher_domains[] medium — manager-asserted, no bilateral

Consumers that want bilateral-only filter evidence=adagents_json; consumers that want the full picture take both.

Files

  • Migration 488 — CHECK widening + backfill from publishers WHERE discovery_method='adagents_authoritative'agent_publisher_authorizations WHERE source='adagents_json'. ON CONFLICT DO NOTHING.
  • publisher-db.ts — new recordCatalogFanoutAuthorization({ agentUrl, childDomain, authorizedFor? }). INSERT-or-UPDATE with the new evidence value, created_by='system', no property_rid.
  • crawler.ts — fan-out helper calls the new writer per (agent, child) pair alongside recordChildPublisherFromManager (publishers row) and recordAgentFromAdagentsJson (legacy authz row).
  • federated-index-db.ts — 6 evidence-CASE statements add WHEN 'adagents_authoritative' THEN 'adagents_json' so the dual-read source field round-trips correctly.
  • Integration test (catalog-fanout-projection.test.ts) — 6 cases: row shape, idempotent re-write, coexistence with adagents_json for same (agent, publisher), invalid agent_url silent-skip, canonicalization, view round-trip.

Production verification after deploy

# Re-trigger cafemedia crawl (deploy runs the backfill as release_command).
ADMIN_KEY=$(grep "^ADMIN_API_KEY=" .env.local | cut -d= -f2)
curl -X POST 'https://agenticadvertising.org/api/registry/crawl-request' \
  -H "Authorization: Bearer $ADMIN_KEY" -H 'Content-Type: application/json' \
  -d '{"domain":"cafemedia.com"}'

# Wait ~60s, then probe the catalog-only auth snapshot endpoint:
curl -sS 'https://agenticadvertising.org/api/registry/authorizations?agent_url=https://interchange.io&evidence=adagents_authoritative' \
  -H "Authorization: Bearer $ADMIN_KEY" | jq '{count: (.rows | length), evidences: ([.rows[].evidence] | unique)}'
# Expected: ~6,800 rows, evidence=["adagents_authoritative"]

Test plan

  • Typecheck passes
  • 6 integration tests for the new writer
  • Reviewer: backfill query plan on production catalog (~thousands of rows expected)
  • Reviewer (post-deploy): partner snapshot endpoint returns ~6,800 cafemedia children

🤖 Generated with Claude Code

…t_authorizations (#4841)

Closes the gap surfaced after #4840 landed fan-out writes in the
legacy agent_publisher_authorizations arm. The catalog projection in
publisher-db.ts:upsertAdagentsCache deliberately refuses cross-
publisher claims (L968: "publisher_properties claims a different
publisher — cross-publisher refused"). Result: ~6,800 cafemedia
child authorization edges existed in the legacy arm but never landed
in catalog_agent_authorizations. The auth-gated partner-sync endpoints
(/registry/authorizations, /registry/authorizations/snapshot) read
the catalog only and missed all of them.

Design: the existing cross-publisher guard is load-bearing for catalog
trust — a hostile manager file can't project authoritative rows for
publishers that didn't authorize anything. The guard stays. The fan-
out gets its OWN evidence value so the projection is explicit and
consumers can filter.

- Migration 488: widen catalog_agent_authorizations.evidence CHECK to
  add 'adagents_authoritative'. Backfill insert from
  publishers + agent_publisher_authorizations where
  discovery_method='adagents_authoritative'.
- New PublisherDatabase.recordCatalogFanoutAuthorization writes the
  edge with evidence='adagents_authoritative', created_by='system',
  no property_rid (publisher-wide).
- crawler.ts fan-out helper now also calls the new writer per
  (agent, child) pair alongside the existing publishers + legacy
  authz writes.
- 6 evidence-mapping CASE statements in federated-index-db.ts add
  WHEN 'adagents_authoritative' THEN 'adagents_json' so the dual-read
  path round-trips correctly.

Trust profile: 'adagents_authoritative' is lower than 'adagents_json'
(manager-asserted, no bilateral confirmation). Per #4825 inline-
resolution rule, the manager file naming the publisher in
publisher_properties[].publisher_domains[] is the out-of-band
corroboration that makes the projection safe.

Tests pin: catalog row shape, idempotent re-write, coexistence with
adagents_json for same (agent, publisher), invalid agent_url
silent-skip, canonicalization, view round-trip.

Resolves #4841.

@aao-release-bot aao-release-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean fix. The cross-publisher guard at publisher-db.ts:968 is load-bearing for catalog trust — lifting it would have been the wrong move; a distinct evidence value with its own writer is the right shape.

Things I checked

  • ON CONFLICT clause in 488_catalog_fanout_authoritative_evidence.sql:70-74 and publisher-db.ts:314-318 matches the partial unique index idx_caa_unique_active (migration 440) exactly: (agent_url_canonical, COALESCE(property_rid::text,''), COALESCE(publisher_domain,''), evidence) WHERE deleted_at IS NULL. Both sites will fire the upsert path, not fail with no unique constraint matching ON CONFLICT specification.
  • All 5 evidence-CASE sites in federated-index-db.ts (lines 164, 214, 280, 447, 1294) carry the new WHEN 'adagents_authoritative' THEN 'adagents_json' arm. The sort-priority CASE at line 1305 is over source (already-mapped column), not v.evidence, so it correctly stays untouched.
  • Backfill join shape (488_catalog_fanout_authoritative_evidence.sql:65-69): publishers WHERE discovery_method='adagents_authoritative'agent_publisher_authorizations WHERE source='adagents_json'. Right pair — recordChildPublisherFromManager sets the discovery_method, recordAgentFromAdagentsJson sets the legacy source. Bounded to ~thousands of cafemedia rows.
  • recordCatalogFanoutAuthorization uses getClient() + try/finally for client release. Consistent with the writer pattern elsewhere in the file.
  • Invalid agent_url silent-skip at publisher-db.ts:308 (if (!agentCanonical) return). Fail-closed beats fail-open. Test at catalog-fanout-projection.test.ts:131 exercises it.
  • CHECK widening at 488_catalog_fanout_authoritative_evidence.sql:35-39DROP CONSTRAINT + ADD CONSTRAINT CHECK (...) takes ACCESS EXCLUSIVE during validation. At ~thousands of rows the window is sub-second. Acceptable.
  • 6 integration cases cover row shape, idempotent re-write, coexistence with adagents_json for the same (agent, publisher) pair, invalid-url silent-skip, canonicalization, and view round-trip.

Follow-ups (non-blocking)

  • Redundant BEGIN; ... COMMIT; in 488_catalog_fanout_authoritative_evidence.sql:32,76migrate.ts:130,142 already wraps each migration in a transaction, so Postgres will log WARNING: there is already a transaction in progress. Migrations 486/487 do the same, so it's convention now; either drop the inner wrapper repo-wide or accept it.
  • PR body claims "6 evidence-CASE statements" in federated-index-db.ts; actual count is 5 (line 1305 is over source). Doc nit.
  • Two test-plan checkboxes unchecked — backfill query plan on production catalog and post-deploy partner-snapshot probe. Both are reviewer/operator validation, not author-side gaps. The writer is covered by the integration suite; just don't skip the post-deploy probe.

Minor nits (non-blocking)

  1. Raw vs canonical agent_url. publisher-db.ts:322 stores input.agentUrl raw (un-trimmed) in agent_url while writing agentCanonical into agent_url_canonical. Matches the existing writer pattern, so consistent — but the raw column is exposed to consumers; trimming the raw side would prevent " https://x.io " from round-tripping with leading whitespace.

LGTM. Follow-ups noted below.

@bokelley
bokelley merged commit 98a1ec3 into main May 20, 2026
15 checks passed
@bokelley
bokelley deleted the bokelley/4841-catalog-fanout-projection branch May 20, 2026 18:55
bokelley added a commit that referenced this pull request May 20, 2026
…4841 follow-up) (#4884)

#4879 added the adagents_authoritative evidence value to
catalog_agent_authorizations and backfilled ~6,800 cafemedia children
into catalog, but the partner-sync endpoints rejected
?evidence=adagents_authoritative with 400 because the VALID_EVIDENCE
set in authorization-snapshot-db.ts was hardcoded to the pre-#4841
enum:

  $ curl '.../api/registry/authorizations?agent_url=...&evidence=adagents_authoritative'
  {"error":"Invalid evidence value: adagents_authoritative. Expected
  one of: adagents_json, agent_claim, community, override."}

Adds the new value to VALID_EVIDENCE. NOT added to DEFAULT_EVIDENCE —
the partner-sync default stays strict-bilateral adagents_json only;
consumers opt in to weaker evidence explicitly.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant