Skip to content

feat(registry): periodic hosted-origin re-verify + lapse→re-claim (#5749 gap #1 follow-up)#5762

Merged
bokelley merged 3 commits into
mainfrom
feat/registry-hosted-origin-reverify
Jun 30, 2026
Merged

feat(registry): periodic hosted-origin re-verify + lapse→re-claim (#5749 gap #1 follow-up)#5762
bokelley merged 3 commits into
mainfrom
feat/registry-hosted-origin-reverify

Conversation

@bokelley

Copy link
Copy Markdown
Contributor

Follow-up to #5752 (bind-on-verify). Closes the domain-transfer / stale-pointer window the security review flagged.

Stacked on #5760 (the time-bomb fix). That commit drops out of this diff once #5760 merges.

Problem

Bind-on-verify binds a domain to its owner while the origin pointer points at AAO. But domains change hands and publishers remove pointers — without re-checking, a transferred domain keeps its stale owner forever and can never be re-claimed (the stale workos_organization_id would block a new owner's bind).

Change

  • Periodic re-verify job (crawler.ts): startPeriodicHostedOriginReverification re-runs verifyHostedPropertyOrigin over verified rows past a 24h TTL (bounded batch + concurrency, mirroring manager-revalidation), wired into startup as an hourly tick. A permanent failure (origin no longer points at AAO) clears origin_verified_at via the existing verifier path — the owner lock lapses. Transient failures (5xx/429/timeout) leave verified state intact by design.
  • Lapse → re-claim (property-db.ts): bindOwnerFromVerifiedClaim now also binds when origin_verified_at IS NULL, so a lapsed domain's stale workos_organization_id no longer blocks a new owner. issueDomainClaim already keys its refusal on origin_verified_at, so a lapsed domain is re-claimable.
  • Verifier ordering fix: bind before stamping origin_verified_at, so the bind guard sees the pre-verification (lapsed) state instead of the lock it's about to re-arm. (Caught by the re-bind test.)
  • getHostedPropertiesDueForReverification: the work-list query — verified rows past the TTL; unverified rows are never candidates.
  • Never deletes on lapse — the rid is forever; only verification + the owner binding lapse, content stays.

Why it's not a takeover vector

A lapse only happens when the real origin stops pointing at AAO (404/mismatch) — an attacker can't force it (no origin control). After a legit lapse, re-binding still requires making the origin point at the new claimant's token — which still requires origin control. The origin_verified_at IS NULL guard clause only widens re-claim for genuinely-lapsed domains.

Tests

New: lapse releases the lock and lets a new owner re-claim + re-bind; due-for-reverification excludes unverified rows. Existing bind (now 7/7) + verifier (10/10) suites green; typecheck clean.

Follow-ups

DNS-TXT pointer flavor; multi-pending-claim griefing table; owner-confirmation of staged properties[] on bind; extend owner-lock to Addie/admin write paths (#5751).

🤖 Generated with Claude Code

@mintlify

mintlify Bot commented Jun 30, 2026

Copy link
Copy Markdown

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
adcp 🟢 Ready View Preview Jun 30, 2026, 5:48 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

@bokelley bokelley force-pushed the feat/registry-hosted-origin-reverify branch from 684bebd to d4e6654 Compare June 30, 2026 05:19

@aao-release-bot aao-release-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. Lapse→re-claim is the right shape: it relaxes the staleness of workos_organization_id, never the bind gate. You still only become owner by proving origin control with your own adcp_claim token.

Things I checked

  • No takeover vector. A lapse is only reachable through the verifier's permanent-failure branches (4xx / invalid JSON / missing-or-mismatched authoritative_location) — each a verdict on the publisher's own origin. safeFetch runs redirect: 'manual' restricted to the same registrable domain, so an off-path attacker can't redirect AAO's probe to flip a verified row. Transient (5xx/429/3xx/timeout/ECONN) preserves verified state. security-reviewer: LOW, bind invariant preserved end-to-end.
  • Lock holds pre-lapse. While verified+locked to ORG_A, issueDomainClaim (property-db.ts:920-926) refuses a different org's claim, so ORG_B can't even plant a claim_token. And bindOwnerFromVerifiedClaim's WHERE fails all three OR branches for ORG_B on a live lock.
  • The ordering fix is load-bearing. Moving stampChecked(true) to after bindOwnerFromVerifiedClaim (hosted-property-origin-verifier.ts:303-317) is the crux: stamp-first would re-arm origin_verified_at, making the new OR origin_verified_at IS NULL guard false and blocking the legitimate re-bind. First-time binds still pass via the workos_organization_id IS NULL OR = claimant_org_id clauses.
  • New clause is tightly scoped. origin_verified_at IS NULL covers exactly never-verified rows (already workos_organization_id IS NULL → no widening) and genuinely-lapsed rows. No live lock has origin_verified_at IS NULL. A NULL-verified row with no pending claim binds no one.
  • TTL work-list query (property-db.ts:990-1015): origin_verified_at IS NOT NULL excludes never-verified rows; ORDER BY origin_last_checked_at ASC NULLS FIRST + LIMIT drains oldest first. Both recordOriginVerification and touchOriginLastCheckedAt stamp origin_last_checked_at = NOW(), so every outcome moves a row out of the window — no re-selection loop.
  • Lifecycle + reentrancy (crawler.ts): unref() keeps the hourly tick from holding the process open, .catch swallows tick rejections, and the hostedReverifyProcessing guard sits flush against the try with a finally that always clears it. The lapsed count !outcome.verified && outcome.reason !== 'transient' is correct against the VerificationOutcome union.
  • brand-handlers UTC math (brand-handlers.ts:940-952): reject only on endMs < startOfTodayMs keeps a campaign ending today licensable, matching the inclusive 23:59:59Z success path. (This rides in from the #5760 stack and drops out on merge.)

Follow-ups (non-blocking — file as issues)

  • Cross-process bind/clear race (security-reviewer: MEDIUM, defensive). The hostedReverifyProcessing flag only serializes ticks within one process. Two concurrent verifier runs on the same domain — the hourly job overlapping a user-triggered verify-origin, or two instances — read-decide-write on origin_verified_at with no row lock, each in its own autocommit txn. No takeover (a bind still needs a matching origin token), but it can flap lock state. Wrap the read-decide-write behind SELECT ... FOR UPDATE or an optimistic WHERE origin_last_checked_at = <read-value> guard before this is heavily relied on. Already implied by your "multi-pending-claim griefing table" follow-up.

Minor nits (non-blocking)

  1. Batch callback has no per-item try/catch. processHostedOriginReverification's processWithConcurrency callback (crawler.ts) doesn't wrap its body, unlike the two sibling call sites at crawler.ts:2068 and :2177. An unexpected throw from verifyHostedPropertyOrigin aborts the whole batch's accounting rather than degrading to a per-item skip. The verifier catches its own failures and each item already persisted origin_last_checked_at, so it's robustness, not correctness.

No schema source touched, so no changeset owed. Bind/lapse/re-bind cycle and the TTL inclusion+exclusion are both covered by behavioral tests.

LGTM. Follow-ups noted below.

@aao-release-bot aao-release-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lapse→re-claim is the right shape: clearing the lock without deleting the row, and keeping the re-bind token-gated, means a transferred domain becomes reclaimable without ever becoming takeable. Verification drives ownership, not the caller.

Things I checked

  • Re-bind stays origin-driven. bindOwnerFromVerifiedClaim (property-db.ts:986-988) widens only which existing workos_organization_id may be overwritten via OR origin_verified_at IS NULL. The bind target is still workos_organization_id = claimant_org_id gated by claim_token = $2 AND claimant_org_id IS NOT NULL. A lapse clears the lock; it does not waive the requirement that the new claimant's adcp_claim token be served at the real origin. security-reviewer: not a takeover primitive, tenant isolation holds.
  • Verifier ordering. Moving bindOwnerFromVerifiedClaim ahead of stampChecked(true) (hosted-property-origin-verifier.ts:303→328) is correct, not just convenient. The bind's UPDATE ... WHERE ... RETURNING reads origin_verified_at from the row inside the same statement; stamping verified first would re-arm the lock and block the legitimate re-bind. The bind is atomic — no mis-bind race between two concurrent verifications.
  • Lapse classification. crawler.ts:2247!outcome.verified && outcome.reason !== 'transient' exactly partitions the verifier's outcomes: every permanent reason routes through stampChecked(false) (clears origin_verified_at), only transient hits stampChecked('preserve'). processWithConcurrency writes results[i] by captured index, so .filter(Boolean).length is an honest lapse count.
  • Work-list query. getHostedPropertiesDueForReverification (property-db.ts:990-996) scopes to origin_verified_at IS NOT NULL — unverified/community rows have no verification state to lapse and are correctly excluded. Test asserts exactly that.
  • DNS classification. EAI_AGAIN now pinned transient before the permanent regex, and EAI_ narrowed to EAI_NONAME (hosted-property-origin-verifier.ts:142) — a temporary resolver blip no longer permanently demotes a verified publisher. Right direction.
  • brand-handlers.ts:940-952endMs < startOfTodayMs (midnight-UTC-today) matches the success path's inclusive valid_until = ...T23:59:59Z. A campaign ending today stays licensable. Test-plan: PR reports bind 7/7, verifier 10/10, typecheck clean; no unchecked manual checkbox on the path this PR changes.
  • Changeset. None present, but CI's check-changeset-protocol-scope.cjs enforces changesets only for static/schemas/source/** / docs/reference/**. This PR is server/src/** + tests only — no wire surface, no changeset required. Not a block.

Follow-ups (non-blocking — file as issues)

  • Reclassify SSRF/private-IP rejections as transient. classifyFetchError (hosted-property-origin-verifier.ts:142) matches the substring private → permanent → lapse. A victim whose DNS an attacker can influence (rebinding to a private IP) trips the SSRF guard's "resolved to a private or internal IP address", which lapses the lock and demotes promoted authorizations. This is denial-of-ownership, not takeover (re-bind still needs origin control), and the regex pre-dates this PR — but the new hourly job now fires it automatically instead of only on a manual verify-origin. An origin resolving to a private IP is an attack signal, not "I no longer point at AAO" — route it to transient. (security-reviewer Medium.)
  • Add failure-count hysteresis before lapsing. The lapse fires on the first permanent-classified response (:226-230, :254-256) — no "N consecutive failures over M hours." Combined with the above, a single spoofed NXDOMAIN releases a verified owner's lock. An origin_consecutive_failures counter reset on any success/transient would close it. (security-reviewer Medium.)

Minor nits (non-blocking)

  1. Double-start leaks the interval. startPeriodicHostedOriginReverification (crawler.ts:2228) overwrites hostedReverifyIntervalId without clearing an existing timer. Only called once from http.ts:9585, so not reachable today — guard it anyway if a restart path ever appears.
  2. stopPeriodicHostedOriginReverification is defined but never called (crawler.ts:2230). Matches the sibling startPeriodicManagerRevalidation pattern and .unref() keeps it from blocking exit, so this is hygiene-only.

Approving on the strength of the token-gated re-bind plus the atomic bind ordering. Two security Follow-ups are availability-only and pre-existing levers — worth an issue, not a hold.

@bokelley

Copy link
Copy Markdown
Contributor Author

Security review — clean (no takeover possible)

A security-reviewer pass on commit d4e6654f9 (the re-verify/lapse work only, excluding the #5760 base) traced all four new paths. Ownership takeover is not possible. The lapse → re-claim widening preserves the bind-on-verify invariant: binding is driven by which adcp_claim token the publisher's own origin carries, and an attacker can't make a victim's origin serve their token.

  • Forced lapse — not possible. origin_verified_at clears only on a permanent verification failure (4xx / no_authoritative_location / authoritative_location_mismatch / NXDOMAIN). Every transient branch (5xx / 429 / 3xx / network) routes to touchOriginLastCheckedAt and preserves verified state. An attacker can't influence the victim's own-origin response.
  • origin_verified_at IS NULL bind clause — not exploitable. It still ANDs claim_token + claimant_org_id; issueDomainClaim refuses while the domain is verified-locked to another org, so a token is only obtainable after a genuine lapse, and binding still requires origin control. The token is consumed (claim_token = NULL) on first bind, so a stale token matches no row.
  • Reorder (bind before stamp) — no exploitable intermediate state. The sub-ms window (owner set, not-yet-verified) is benign; a concurrent claim still needs origin control to bind.
  • Periodic job — no SSRF/DoS surface. Startup-only (no external trigger), work-list keyed on internal state, fetch target = the publisher's own domain via safeFetch, bounded batch/concurrency with unref().

Minor findings (all fixed in d4e6654f9)

  1. NXDOMAIN now lapses"Could not resolve hostname" was classified transient, so an expired domain never released (the one advertised behavior that didn't fire). Now permanent, with a test.
  2. EAI_AGAIN (transient DNS) explicitly kept transient; documented the self-healing property (a lapse clears only origin_verified_at, so a transient blip re-locks on the next successful re-verify).
  3. Stale redirect: 'manual' comment corrected (safeFetch follows + re-validates redirects).

 gap #1 follow-up)

Bind-on-verify (#5752) binds a domain to its owner when the origin pointer points
at AAO. But domains change hands and publishers remove pointers, so a verified
row needs re-checking on a TTL — otherwise a transferred domain keeps its stale
owner forever and can never be re-claimed.

- Periodic re-verify job (crawler): startPeriodicHostedOriginReverification re-runs
  verifyHostedPropertyOrigin over verified rows past a 24h TTL, bounded batch +
  concurrency, mirroring the manager-revalidation pattern. Wired into startup
  (hourly tick). A permanent failure (origin no longer points at AAO) clears
  origin_verified_at via the existing verifier path — the owner lock lapses.
  Transient failures (5xx/429/timeout) leave verified state intact by design.
- Lapse → re-claim: bindOwnerFromVerifiedClaim now also binds when
  origin_verified_at IS NULL, so a lapsed domain's stale workos_organization_id
  no longer blocks a new owner from re-binding. issueDomainClaim already keys its
  refusal on origin_verified_at, so a lapsed domain is re-claimable.
- Verifier ordering fix: bind BEFORE stamping origin_verified_at, so the bind
  guard sees the pre-verification (lapsed) state rather than the lock it's about
  to re-arm. (Found by the re-bind test.)
- getHostedPropertiesDueForReverification: the work-list query (verified rows
  past the TTL; unverified rows are never candidates).
- The row is never deleted on lapse — the rid is forever; verification and the
  owner binding lapse, content stays.

Tests: lapse releases the lock and lets a new owner re-claim and re-bind; the
due-for-reverification query excludes unverified rows. Existing bind + verifier
suites unchanged (7/7 and 10/10).

Stacked on #5760 (the time-bomb fix) — that commit drops out on rebase once #5760
merges.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@aao-release-bot aao-release-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The lapse→re-claim mechanism is the right shape — binding stays anchored to which token the real origin carries, and the lapse clears only origin_verified_at, never workos_organization_id, so a transferred domain is re-claimable but a takeover still requires origin control. One blocker in the failure classification undercuts the PR's own central claim.

MUST FIX

classifyFetchError lapses a legitimately-verified domain on a transient DNS failure. hosted-property-origin-verifier.ts:152 adds could not resolve hostname to the unresolvable (permanent) set. But safeFetch resolves DNS through validateHostResolution (utils/url-security.ts:174-186), which Promise.allSettleds resolve4/resolve6 and throws the single literal string "Could not resolve hostname" whenever both reject — and both reject for transient EAI_AGAIN/SERVFAIL exactly as they do for permanent NXDOMAIN. So a routine resolver hiccup now classifies as permanent → stampChecked(false) clears origin_verified_at → the re-verify job counts a "lapse," demotes the promoted authorizations, and marks the row re-claimable.

This violates the verifier's own load-bearing invariant — hosted-property-origin-verifier.ts:44-47 and :209-211: "a network blip MUST NOT flip a previously-verified row to unverified." And the PR body's "a lapse only happens when the real origin stops pointing at AAO" is false as implemented: a SERVFAIL is not the origin ceasing to point at AAO.

What breaks for adopters: a verified publisher with a momentary DNS resolver outage loses its origin attestation (demoted authorizations) and is marked re-claimable. Because the lapse stamps origin_last_checked_at = NOW(), the row won't be re-checked for the full 24h TTL — so the false lapse persists up to 24h unless verify-origin is manually re-triggered. No takeover results (re-binding still needs origin control, confirmed by security-reviewer — verdict otherwise sound, no High), but the demotion + re-claim window is a concrete, reproducible regression.

The EAI_AGAIN → transient early-guard added in this same diff is a precise guard against a string that never reaches this function in production — it reads as a safety net that isn't load-bearing.

Fix: don't treat the collapsed "could not resolve hostname" as unconditionally permanent. Either (a) require N consecutive failures across ticks before clearing, or (b) classify on a typed cause (err.cause?.codeENOTFOUND/EAI_NONAME = permanent, EAI_AGAIN/SERVFAIL = transient) rather than the message string, which means having validateHostResolution distinguish NXDOMAIN from temporary-failure and propagate it.

Things I checked

  • Bind-before-stamp ordering is correct. First-verify: origin_verified_at is NULL pre-stamp → OR-clause permits the intended first bind. Owner re-verify: claim_token is NULL in the DB after the first bind, so claim_token = $2 never matches and the bind is a correct no-op (property-db.ts:979).
  • origin_verified_at IS NULL OR-clause is idempotent and still token-gated. bindOwnerFromVerifiedClaim (property-db.ts:983-987) keeps claim_token = $2 AND claimant_org_id IS NOT NULL mandatory and nulls the token on bind; the new clause only relaxes the stale-org-id guard for genuinely-lapsed rows.
  • Staging vector is closed. issueDomainClaim (property-db.ts:920-926) refuses a token to a different org while origin_verified_at is set — an attacker can't pre-stage a token to race a lapse.
  • Work-list excludes unverified rows. getHostedPropertiesDueForReverification filters origin_verified_at IS NOT NULL; test confirms the community/unverified row is never a candidate.
  • TTL advances on every outcome. Both recordOriginVerification (property-db.ts:874-875) and touchOriginLastCheckedAt (:894) stamp origin_last_checked_at — no row gets re-checked every tick.
  • Job plumbing mirrors manager-revalidation. unref(), re-entrancy guard, bounded BATCH_SIZE=50/CONCURRENCY=4, .catch() on the tick, lapsed count keyed on reason !== 'transient' — all sound (crawler.ts:2225-2284).

Follow-ups (non-blocking — file as issues)

  • Bind + stamp aren't transactional (security-reviewer Medium). The bind UPDATE and recordOriginVerification(domain, true) are two un-transactioned writes; a concurrent lapse tick or crash between them can leave (workos_organization_id set, origin_verified_at NULL, is_public TRUE) — publicly serving while re-claimable. Fold the verified stamp into the bind UPDATE's RETURNING, or wrap both in one transaction.
  • Flapping dwell. Consider a minimum dwell on origin_verified_at = NULL before honoring a re-bind so a flapping origin can't churn ownership. Hardening, not exploitable.

Minor nits (non-blocking)

  1. EAI_ narrowed to EAI_NONAME. hosted-property-origin-verifier.ts:152 — the old broad EAI_ matched EAI_FAIL/EAI_NODATA; those now fall through to transient. Direction is safe (keeps the lock), but the narrowing is silent — enumerate or comment it.
  2. Redundant ?. on unref(). crawler.tssetInterval always returns a Timeout; this.hostedReverifyIntervalId?.unref() only matters under a future refactor.

Fix the classification (or justify why the transient-DNS demote is acceptable) and this is a clean ship. The lapse design itself is right — it's the permanent-vs-transient boundary that's mislabeled.

@aao-release-bot aao-release-bot Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving. The core security question — does OR origin_verified_at IS NULL in bindOwnerFromVerifiedClaim open a domain-takeover path — resolves no: the clause widens when a row is re-claimable, never what binding requires. Re-binding still demands the claim token appear at the real origin, so a lapse can't be parlayed into taking over a domain the attacker doesn't control.

Things I checked

  • Bind-before-stamp is load-bearing and correct. Moving bindOwnerFromVerifiedClaim ahead of stampChecked(true) (hosted-property-origin-verifier.ts:308-317, stamp at :347) means the bind guard reads the live pre-verification origin_verified_at IS NULL, so a lapsed row is re-bindable. Stamping first would re-arm the lock and block the legitimate re-bind. The comment explains it; the re-bind test catches the regression.
  • Classification fails closed toward preserving the lock. classifyFetchError tests the transient regex (EAI_AGAIN|ESERVFAIL|ETIMEOUT|ECONNREFUSED) before the unresolvable one, and collectErrorText flattens message/code/codes[]/the cause chain into one string — so a mixed codes: ['EAI_AGAIN','ENOTFOUND'] resolves transient. dnsResolutionFailureCause (url-security.ts:191-211) sets code with precedence transient ?? permanent ?? first. A bare "Could not resolve hostname" with no DNS code is deliberately transient. Every ambiguous case keeps the legit owner's lock — under-lapse, not over-lapse.
  • Lapse never reassigns ownership. A permanent failure only NULLs origin_verified_at via recordOriginVerification(domain, false); workos_organization_id is untouched until a new origin-attested token binds it. issueDomainClaim's refusal and the bind guard are now symmetric on origin_verified_at.
  • SSRF enforcement unchanged. The url-security.ts changes are additive error-classification plumbing (extractDnsErrorCode, dnsResolutionFailureCause, unresolvedHostnameError); isPrivateHostname, per-address private-IP rejection, ssrfSafeLookup, and per-hop redirect re-validation are all intact. The 3xx comment change is documentation-only.
  • Job is bounded. processHostedOriginReverification (crawler.ts) uses BATCH_SIZE=50 against a LIMIT $2 query, CONCURRENCY=4 via the existing processWithConcurrency pool, a try/finally hostedReverifyProcessing guard, and unref() on the interval. getHostedPropertiesDueForReverification correctly excludes unverified rows (origin_verified_at IS NOT NULL) — nothing to lapse there.
  • No changeset in the PR — not required: the diff touches only server/src/** and tests, none of the wire-surface paths. Server runtime behavior, not a wire change.

Follow-ups (non-blocking — file as issues)

  • The job runs on a single instance's setInterval with an in-process hostedReverifyProcessing flag. With multiple replicas there's no cross-instance lock, so each replica re-verifies the due batch — harmless (idempotent verify, last-writer-wins SQL), just redundant origin fetches. SELECT ... FOR UPDATE SKIP LOCKED on the work-list query if replica count grows.
  • The lapse path releases an owner lock on a single permanent-classified re-verify with no grace window. The classifier is conservative enough that this is fine, but a "two consecutive permanent failures before lapse" policy would harden against a one-off resolver anomaly that happens to surface a permanent code.

Minor nits (non-blocking)

  1. Document the per-process guard. One line on hostedReverifyProcessing noting it's per-instance and cross-replica overlap is safe-but-wasteful, so a future reader doesn't mistake it for a distributed lock. crawler.ts.

Third re-verify follow-up in the registry ownership series and it lands the same way the prior two did — tight diff, the tradeoff named in the body, the invariant (ownership only ever established by an origin-attested token) preserved. LGTM.

@bokelley bokelley merged commit ee3d423 into main Jun 30, 2026
15 checks passed
@bokelley bokelley deleted the feat/registry-hosted-origin-reverify branch June 30, 2026 08:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant