Stripe webhook: cancellation silently dropped when customer link has drifted

[bokelley]

Context

Discovered during Media.net escalation #321 cleanup sweep. Single drift case across 152 active orgs, but a real bug.

What happened

WPP Media (org_01KDTESPG8WZMME4AJ0VQ0AJKR) lapsed (invoice non-payment, sub auto-canceled by Stripe 2026-04-25T21:26:49Z), but our DB still shows them as subscription_status=active, membership_tier=company_icl, subscription_canceled_at=null.

Stripe delivered the cancellation event (evt_1TQDivHs54uYsO3aenPqnzI2, pending_webhooks: 0 — our endpoint ack'd 200). The handler at server/src/http.ts:3674-3880 did not update the org row.

Root cause hypothesis

WPP's org row has a customer↔sub mismatch:

• stripe_customer_id = cus_TmJv7a2qMH5Ew1 (cameron.saxby@wppmedia.com — current)
• stripe_subscription_id = sub_1SrKXOHs54uYsO3aclNCZ9Dv — belongs to cus_ToyU1L0YjiliFK per Stripe (NOT our linked customer)

When the cancellation event arrived for sub_1SrKXOHs54uYsO3aclNCZ9Dv, the event's customer was cus_ToyU1L0YjiliFK. The resolver (server/src/billing/webhook-helpers.ts:27-90) walks:

1. Fast path: stripe_customer_id lookup → miss (we have a different customer linked)
2. Stripe customer metadata → empty (cus_ToyU1L0YjiliFK has no metadata)
3. Subscription metadata → has workos_organization_id pointing at WPP. Should resolve.
4. Tries setStripeCustomerId(WPP_id, 'cus_ToyU1L0YjiliFK') → throws StripeCustomerConflictError (WPP already has a different customer linked). Caught and logged, function returns WPP.

Per the code, the handler should then proceed to UPDATE WPP's row to subscription_status=canceled, subscription_canceled_at=<ts>. It didn't. Without production logs from 2026-04-25, the exact failure point is unclear.

Leading hypotheses

1. The StripeCustomerConflictError path doesn't actually return the org cleanly — re-read the error/return semantics in webhook-helpers.ts:78-89. Possibly the catch swallows but then the function falls through to a code path that returns null.
2. isStaleNonLiveEvent guard at http.ts:3820-3826 misfires. Verified above it should NOT trigger here, but worth re-reading.
3. UPDATE ran and was later overwritten by another event.

Proposed work

• Trace webhook-helpers.ts:27-90 path with a test that simulates the customer-drift scenario and asserts the org row gets updated.
• Add an audit log entry for every customer.subscription.* event we process, even when no org found, so future drift cases are debuggable from our side without going to Stripe.
• Consider adding a periodic sweep (the diagnostic I ran today: scan all subscription_status='active' orgs, retrieve sub from Stripe, flag drift). Could run weekly as a cronjob and post to an admin channel.

Out of scope

• The customer-link drift itself (how WPP ended up with a sub belonging to a different customer than the linked one). Probably a manual admin action; separate investigation if it recurs.
• WPP's specific data fix — Brian confirmed they lapsed; will reset state separately.

[bokelley]

Triage

Classification: Bug
Bucket(s): admin-tool
Status: drafting-pr

What the experts said:

• code-reviewer: Root cause is a missing stripe_subscription_id DB fallback — all three hypotheses in the issue were red herrings. Resolver returns null when metadata is absent, UPDATE is silently skipped. Fix is non-breaking; invoice.paid drift gap noted as latent follow-up.
• internal-tools-strategist: Fix is proportionate and closes the access-control gap (lapsed org with active status). Periodic sweep (item 3) is redundant — org-row-matches-live-stripe-sub invariant already exists in server/src/audit/integrity/; schedule that instead of building a new cronjob.

My take: The silent drop happens earlier than any of the three hypotheses describe: when the drift customer has no metadata and no customer-ID match, the resolver returns null before reaching the UPDATE gate. Added a stripe_subscription_id DB fallback (step 3.5) so the org is resolved even in the metadata-absent case, plus notifySystemError alerting when resolution still fails. Draft PR: #4172

---

Triaged by Claude Code. Session: https://claude.ai/code/session_017UQZCzNYrKnhBJKWUpBCcT

---

Generated by Claude Code