Skip to content

fix: harden orchestration state recovery - #136

Merged
chihsuan merged 3 commits into
mainfrom
fix/orchestrator-review-ci-state-bugs
Jun 30, 2026
Merged

fix: harden orchestration state recovery#136
chihsuan merged 3 commits into
mainfrom
fix/orchestrator-review-ci-state-bugs

Conversation

@chihsuan

Copy link
Copy Markdown
Member

Context

Rework dispatch could lose durable context or get stuck after transition failures, and multi-repo startup cleanup missed secondary repos.

TL;DR

Make rework state changes durable before dispatch and recover cleanly after restarts.

Summary

  • Persist PR review rework context before moving Linear issues back to active.
  • Let CI retries escalate after max attempts, while keeping active-run protection.
  • Clear failed CI dispatch markers so the same SHA can be retried.
  • Interrupt persisted running runs for every configured repo on startup.
  • Reject repo keys that collide after workspace path sanitization.

Alternatives

  • Keep retrying later without ordering changes; rejected because dispatch can run first with missing context.
  • Keep workspace cleanup scoped by raw repo key only; rejected because sanitized paths still collide.

Test Plan

  • make all
  • mix test test/symphony_elixir/ci_poller_test.exs test/symphony_elixir/pr_review_poller_test.exs test/symphony_elixir/config_split_test.exs test/symphony_elixir/orchestrator_status_test.exs
  • mix specs.check

chihsuan added 3 commits June 30, 2026 17:34
Reordering escalation ahead of dispatched_for_sha? reopened a race where the
poll right after the final retry's dispatch could escalate before the agent
reached "running", abandoning it. Hold escalation off for a grace window after
a dispatch lands (until rework_in_progress?/2 takes over).

Also: dedupe workspace key sanitization via Workspace.safe_identifier, refresh
the stale prefetch comment, and cover the Linear-transition-failure durability
path.
@chihsuan
chihsuan merged commit 4390b3a into main Jun 30, 2026
19 of 21 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant