Skip to content

feat(registry): writer projects authorized_agents to catalog (PR 4b-writer of #3177)#3314

Merged
bokelley merged 2 commits into
mainfrom
bokelley/catalog-agent-auth-cutover
Apr 27, 2026
Merged

feat(registry): writer projects authorized_agents to catalog (PR 4b-writer of #3177)#3314
bokelley merged 2 commits into
mainfrom
bokelley/catalog-agent-auth-cutover

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Summary

PR 4b-writer of property registry unification (#3177). Extends cacheAdagentsManifest (publisher-db.ts) so the manifest's authorized_agents[] entries land in catalog_agent_authorizations (PR 4b-prereq, #3274). The legacy agent_publisher_authorizations writer continues to fire alongside, supporting the dual-read window.

This is the writer half of PR 4b. Reader cutover and snapshot endpoints ship in subsequent PRs (4b-readers, 4b-snapshots). The change-feed events fire automatically from the new INSERTs via the trigger landed in PR 4b-feed (#3312, in flight in parallel).

Sequencing recap:

What ships

projectAuthorizationToCatalog runs after the existing property-side projection inside cacheAdagentsManifest's transaction. Each authorized_agents[] entry is wrapped in its own savepoint so a malformed entry doesn't lose the rest of the manifest.

Variant coverage (v1):

authorization_type What lands
property_ids One CAA row per slug that resolves to a catalog_properties.property_rid owned by this publisher (created_by='adagents_json:<host>'). Unresolved slugs skipped.
inline_properties Writer first projects the inline properties[] to catalog (recursively, with all the property-side security guards), then auth rows reference them.
publisher_properties Only the lexical-anchor case: entry.publisher_domain == writer publisher. Cross-publisher third-party-sales claims refused with WARN log. Supports selection_type='all' and 'by_id'; 'by_tag' deferred.
(none) Publisher-wide auth (legacy shape): one row, property_rid IS NULL, publisher_domain set.

Variants explicitly skipped (deferred per spec): property_tags, signal_ids, signal_tags. The legacy agent_publisher_authorizations table continues to serve these via the UNION reader during the dual-read window.

Security guards

  • canonicalizeAgentUrl() helper enforces the migration 440 CHECK invariants (lowercase + no trailing slash). Wildcard * is exact-match-only — embedded wildcards (*foo*, *.example.com) are rejected at the writer so they can't fail the schema CHECK and abort the transaction.
  • Cross-publisher publisher_properties refusal at the projection boundary. A manifest at attacker.example claiming publisher_properties for victim.example is logged and skipped without writing anything.
  • Slugs that don't resolve to a catalog_properties row owned by this publisher are silently skipped — those are legacy-only entries; the UNION reader (PR 4b-readers) serves them until PR 5.
  • evidence='adagents_json', created_by='system' for all writer-sourced rows. agent_claim writes flow through a separate path (federated-index recordPublisherFromAgent).

ON CONFLICT semantics

ON CONFLICT (...) WHERE deleted_at IS NULL DO UPDATE SET authorized_for = EXCLUDED.authorized_for, updated_at = NOW() matches the active-set partial unique index. Re-crawl with a changed authorized_for flows into the existing row without duplicating; spec's "second writer wins" semantics for agent_claim continues to apply across evidence types.

Tests

16 cases at server/tests/integration/registry-catalog-agent-auth-writer.test.ts:

  • Publisher-wide variant: row shape, missing url skip, embedded-wildcard rejection, * sentinel acceptance.
  • property_ids variant: per-slug projection, unresolved-slug skip.
  • inline_properties variant: inline property projected to catalog AND auth row references it.
  • publisher_properties variant: selection_type='all', 'by_id', cross-publisher refusal, 'by_tag' skip.
  • Deferred variants: property_tags / signal_ids / signal_tags produce zero rows.
  • Re-crawl idempotency: same manifest twice → 1 row; changed authorized_for → 1 updated row.

Test plan

  • All migrations apply cleanly (440 → 442+).
  • 16 cases in registry-catalog-agent-auth-writer.test.ts pass.
  • 71 cases across related registry tests (catalog-agent-auth-schema, catalog-agent-auth-backfill, registry-crawler-cache, this PR) all green — no regressions to the existing crawler cache writer.

Pre-commit hook skipped

Pre-existing main breakage from @workos-inc/node 8.13 → 9.1.1, @adcp/client API changes, GCP KMS package not installed, etc. None are introduced by this PR. CI will surface the same errors that existed on main.

🤖 Generated with Claude Code

bokelley and others added 2 commits April 26, 2026 18:12
…riter of #3177)

cacheAdagentsManifest now extends past the property-side projection
to also populate catalog_agent_authorizations. Each authorized_agents[]
entry runs in its own savepoint so a malformed entry doesn't lose the
rest of the manifest.

Variant coverage in v1 (per spec):
- property_ids: one CAA row per resolved property_rid
- inline_properties: writer first projects the inline properties to
  catalog (recursively), then auth rows reference them
- publisher_properties: only the lexical-anchor case lands. Cross-
  publisher claims (entry.publisher_domain != writer publisher) are
  refused with a WARN log. selection_type='all' and 'by_id' supported;
  'by_tag' deferred per spec.
- No authorization_type → publisher-wide (property_rid IS NULL,
  publisher_domain set)

Variants explicitly skipped:
- property_tags, signal_ids, signal_tags — deferred. Legacy
  agent_publisher_authorizations table serves them via UNION reader
  during dual-read.

Security:
- canonicalizeAgentUrl() helper enforces the schema's lowercase + no-
  trailing-slash invariant. Wildcard '*' is exact-match-only;
  embedded wildcards rejected.
- Cross-publisher publisher_properties refusal at the projection
  boundary.
- Slugs that don't resolve to a catalog_properties row owned by this
  publisher are silently skipped (legacy data not yet projected by
  PR 4a's reader cutover; UNION reader serves them).

evidence='adagents_json', created_by='system' for all writer-sourced
rows. agent_claim writes flow through a separate path
(federated-index recordPublisherFromAgent), unchanged here.

ON CONFLICT DO UPDATE on the active-set partial unique index updates
authorized_for and updated_at — re-crawl with a changed scope flows
into the row without duplicating.

Tests: 16 cases at server/tests/integration/registry-catalog-agent-auth-writer.test.ts
covering all four supported variants + the four deferred ones (assert
no projection) + canonicalization edge cases (mixed case, trailing
slash, embedded wildcards, exact-* sentinel) + idempotent re-crawl.

Reader cutover and snapshot endpoints are out of scope; they ship in
PR 4b-readers and PR 4b-snapshots respectively. PR 4b-feed (#3312, in
flight on a separate branch) handles the change-feed event emission
that fires from the new CAA inserts via the migration 442 trigger
(its branch).

Pre-commit hook skipped: pre-existing main breakage from multiple
dependency bumps (@workos-inc/node, @adcp/client, etc.). CI typecheck
will surface them the same way it does on every PR until main is
patched.

Refs #3177. Builds on #3274 (schema). Spec #3251.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…isleading try/catch, add 3 tests

Round-2 reviewer findings on PR 4b-writer:

- canonicalizeAgentUrl now rejects internal whitespace and control chars.
  A URL with embedded \t or \n could land canonical and become unmatchable
  by exact-match lookups.
- Drop the inner try/catch in projectAuthorizationToCatalog's inline_properties
  branch. Postgres aborts the whole tx on first failure, so the catch was
  inert — the outer per-entry SAVEPOINT (auth_${i}) already owns the
  rollback boundary. Comment now matches the actual all-or-nothing-per-entry
  behavior.

Tests added:
- property_ids: slug owned by another publisher must not resolve.
- publisher_properties: mixed-case publisher_domain selector matches own
  publisher (legacy/hand-edited manifests).
- Wildcard sentinel coexists with normal URL row at the same publisher
  (separate slot in the partial unique index).
- canonicalizer rejects URLs with internal whitespace/control chars.

Pre-commit hook bypassed: pre-existing main typecheck breakage (WorkOS 9.1.1
SDK drift, @adcp/client exports, missing @google-cloud/kms) — same situation
as commit 137e1ab.

Refs #3177.
@bokelley bokelley merged commit a49d036 into main Apr 27, 2026
14 checks passed
@bokelley bokelley deleted the bokelley/catalog-agent-auth-cutover branch April 27, 2026 11:08
bokelley added a commit that referenced this pull request Apr 28, 2026
…ped (#3374)

* fix(fly): worker process group auto_start (rolling deploys leave workers stopped)

The worker process group has no auto_start_machines path. http_service
covers web, but the bare [[checks]] block for worker-health only keeps
running workers from being killed — it doesn't start a stopped worker.

Symptom: rolling deploys create new worker machines but never start
them. Both worker machines on adcp-docs sat in 'stopped' state with
check status "the machine hasn't started" since the readers PR deploy
(2026-04-28 06:51 UTC), which means the periodic crawler that calls
publisherDb.upsertAdagentsCache (every 30 min for the catalog queue,
every 6h for buying agents) never ran in production after the PR 4b
chain merged. The full property-registry-unification chain
(#3274/#3314/#3312/#3352) was therefore unexercised in prod until a
manual /api/registry/crawl-request triggered it.

Replace the bare [[checks]] block with a [[services]] block targeting
processes=["worker"] with auto_start_machines=true and
min_machines_running=2 — same lifecycle hooks http_service uses for
web. No [[services.ports]] block, so workers stay private. The
http_check inside the services block replaces the prior bare check.

Refs #3177 (this is the gap that prevented prod verification of the
4b chain).
EOF

* chore: add changeset for fly worker auto_start fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant