Skip to content

fix(connect): reboots no longer strand the relay link, 403s now say why - #4988

Merged
t3dotgg merged 2 commits into
mainfrom
t3code/fix-connect-retry-backoff
Jul 30, 2026
Merged

fix(connect): reboots no longer strand the relay link, 403s now say why#4988
t3dotgg merged 2 commits into
mainfrom
t3code/fix-connect-retry-backoff

Conversation

@t3dotgg

@t3dotgg t3dotgg commented Jul 30, 2026

Copy link
Copy Markdown
Member

Problem

Two independent bugs made a reboot look like "T3 Connect is broken" with no useful signal:

  1. On boot, systemd can start the server before the network is usable. Startup link reconciliation retried with Effect.retry({ times: 4 }) and no delay, so all retries burned in ~1 second while the NIC was still coming up, then gave up permanently. The server ran fine locally but no environment link existed at the relay.
  2. Every connect attempt then failed with the generic 403 "Relay environment connection is not authorized". The relay internally knows 8 distinct not-authorized reasons (including environment_link_not_found), but dropped the reason before it reached the client, so "your server never linked" read as a credential problem.

Solution

  • Startup reconcile now retries on an exponential schedule (1s base, capped at 30s, bounded at 10 minutes overall) and skips retrying deterministic local failures (bad request / not linked yet). A boot-time network race now converges instead of giving up.
  • RelayEnvironmentConnectNotAuthorizedError carries an optional reason field threaded from the relay's internal reason set. Optional keeps old-relay responses decoding, and old clients ignore the extra field, so it deploys compatibly in either order. Web and mobile now render environment_link_not_found as "Relay has no active link for this environment" instead of an auth-sounding 403, and show the raw reason for the rest.
  • The relay's internal reason schema is now re-exported from the shared contract so the two lists can't drift.

Typecheck, lint, build, and the relay/contracts/client-runtime/server test suites all pass.


🤖 Generated with Claude Code using Claude Fable 5


Note

Medium Risk
Touches startup cloud link reconciliation and relay/client error contracts; backward-compatible optional reason, but wrong retry bounds could delay or hide permanent link failures.

Overview
Fixes two reboot/connect failure modes: the environment server giving up on relay link reconciliation too quickly after boot, and clients seeing a generic auth-style 403 when the relay knows a specific cause.

Startup link reconcile (server.ts): Replaces four immediate retries with exponential backoff (1s base, 30s cap per delay) over up to ~10 minutes. Retries stop on deterministic local errors (EnvironmentHttpBadRequestError, EnvironmentHttpUnauthorizedError, EnvironmentHttpConflictError) so malformed origins or “not linked yet” don’t spin forever.

403 semantics: Adds shared RelayEnvironmentConnectNotAuthorizedReason in contracts and an optional reason on RelayEnvironmentConnectNotAuthorizedError. The relay API forwards the connector’s reason; relay code uses the shared schema instead of a duplicate list. Web and mobile map environment_link_not_found to a clear “no active link / server may not have re-established yet” message and include other reasons in the error text when present.

Reviewed by Cursor Bugbot for commit 2b9b2ea. Bugbot is set up for automated code reviews on this repo. Configure here.

Note

Fix relay link stranding on reboot and include 403 reason in connect errors

  • Replaces the fixed 4-retry startup reconciliation in server.ts with exponential backoff (1s base, 30s max delay, 10-minute cap), so server reboots no longer exhaust retries immediately and strand the relay link.
  • Skips retries for deterministic 4xx errors (BadRequest, Unauthorized, Conflict) to avoid pointless loops.
  • Adds RelayEnvironmentConnectNotAuthorizedReason to relay.ts contracts and threads the reason through the relay API and both client relayProtectedErrorMessage functions, so 403 responses now surface the specific reason in user-visible messages.
  • Special-cases environment_link_not_found in error messages to clarify it is not a credential issue.

Macroscope summarized 2b9b2ea.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 30, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fa640cdd-568f-4931-a7a9-8962027fda96

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Jul 30, 2026

@macroscopeapp macroscopeapp Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the Effect service conventions for this PR. The retry schedule in apps/server/src/server.ts uses proper effect/Duration and effect/Schedule subpath namespace imports, and the new contract error derives its message from structural attributes. One minor migration nit noted inline.

Posted via Macroscope — Effect Service Conventions

Comment thread infra/relay/src/environments/EnvironmentConnector.ts Outdated

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 5ce17a2. Configure here.

Comment thread apps/server/src/server.ts
@macroscopeapp

macroscopeapp Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved 2b9b2ea

This bug fix improves retry resilience for server reboots (exponential backoff instead of immediate retries) and threads error reasons through to clients for better debugging. Changes are self-contained with clear intent, backward-compatible schema updates, and no security implications.

You can customize Macroscope's approvability policy. Learn more.

…ason alias

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@t3dotgg
t3dotgg merged commit 1877237 into main Jul 30, 2026
17 checks passed
@t3dotgg
t3dotgg deleted the t3code/fix-connect-retry-backoff branch July 30, 2026 14:31
github-actions Bot added a commit to omarcresp/t3code-flake that referenced this pull request Jul 30, 2026
## What's Changed
* docs: split user and maintainer docs, fix 100+ stale claims by @t3dotgg in pingdotgg/t3code#4807
* fix(web): server updates no longer look like warnings by @t3dotgg in pingdotgg/t3code#4992
* fix(connect): reboots no longer strand the relay link, 403s now say why by @t3dotgg in pingdotgg/t3code#4988
* Add project file picker (⌘P) and project content search (⇧⌘F) by @jakeleventhal in pingdotgg/t3code#4855
* Check for mobile app updates on launch by @juliusmarminge in pingdotgg/t3code#4958


**Full Changelog**: pingdotgg/t3code@v0.0.32-nightly.20260730.956...v0.0.32-nightly.20260730.957

Upstream release: https://github.com/pingdotgg/t3code/releases/tag/v0.0.32-nightly.20260730.957
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant