Skip to content

fix(daemon): race condition when shutdownAction is none on first up - #784

Merged
skevetter merged 4 commits into
mainfrom
eloquent-sheep
Jul 27, 2026
Merged

fix(daemon): race condition when shutdownAction is none on first up#784
skevetter merged 4 commits into
mainfrom
eloquent-sheep

Conversation

@skevetter

@skevetter skevetter commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes a race in the machine-provider inactivity daemon where shutdownAction: none was ignored on the first up, so the workspace was auto-stopped despite the setting. This surfaced as the flaky e2e failure test shutdownAction none suppresses inactivity timeout (devsy stop failed — the workspace was already Stopped).

Root cause

The daemon decided whether to auto-stop from a global --shutdown-action flag baked in at install time (checkAndShutdown). installDaemon sources that flag from workspaceInfo.LastDevContainerConfig, but it runs during initialize() before runner.Up resolves the devcontainer config. On the first up LastDevContainerConfig is nil (it's only persisted host-side after a successful up), so the daemon started with an empty shutdownAction, ignored none, and ran the provider's shutdown once activity went stale.

Fix

Resolve shutdownAction per-workspace, at patrol time instead of from a one-shot global flag:

  • After the build resolves the config, persist it back to the machine's workspace.json (persistResolvedConfig + new agent.PersistAgentWorkspaceInfo).
  • The daemon reads the per-workspace LastDevContainerConfig.Config.ShutdownAction (effectiveShutdownAction), falling back to the install-time flag when absent. This is dynamic (picked up next patrol, no systemd restart) and also correct for multiple workspaces under one daemon.

Also fixes a latent bug in writeWorkspaceInfo: it cleared CLIOptions on a clone but marshaled the original, persisting them anyway.

Tests

  • TestEffectiveShutdownAction — per-workspace precedence and fallback paths.
  • TestPersistAgentWorkspaceInfo_* — resolved config round-trips through the daemon's parse path; CLIOptions are not persisted.

CodeRabbit local review: no findings.

Summary by CodeRabbit

  • Bug Fixes

    • Inactivity-based shutdown now respects each workspace’s configured shutdown behavior.
    • Resolved workspace configuration is saved so shutdown decisions remain consistent across operations.
  • Tests

    • Added coverage for workspace-specific shutdown settings, configuration persistence, validation errors, and secure handling of command-line options.

The inactivity daemon decided whether to auto-stop a workspace from a
global --shutdown-action flag baked in at install time. On the first up
that flag was empty: installDaemon runs before the devcontainer config is
resolved, and the host has no LastDevContainerConfig to send yet. The
daemon therefore ignored shutdownAction "none" and stopped the workspace
once activity went stale, failing "shutdownAction none suppresses
inactivity timeout".

Resolve shutdownAction per-workspace at patrol time instead:
- persist the build-resolved devcontainer config back to workspace.json
- daemon reads the per-workspace shutdownAction, falling back to the flag

Also fix writeWorkspaceInfo marshaling the original instead of the clone,
which persisted CLIOptions despite the intent to drop them.
@netlify

netlify Bot commented Jul 27, 2026

Copy link
Copy Markdown

Deploy Preview for images-devsy-sh canceled.

Name Link
🔨 Latest commit 4b378ee
🔍 Latest deploy log https://app.netlify.com/projects/images-devsy-sh/deploys/6a67b409aaa3ee000803af49

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@skevetter, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 15 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 410f1b37-6d88-41ee-a8c8-a7b537b7e2f8

📥 Commits

Reviewing files that changed from the base of the PR and between 790ab92 and 4b378ee.

📒 Files selected for processing (1)
  • cmd/internal/agent_daemon_test.go
📝 Walkthrough

Walkthrough

Resolved devcontainer configuration is persisted during workspace startup. The daemon now uses the workspace’s resolved shutdown action, falling back to its install-time action when unavailable. Persistence tests cover round-tripping, validation, CLI option sanitization, and non-mutation.

Changes

Workspace shutdown configuration

Layer / File(s) Summary
Persist resolved workspace configuration
pkg/agent/agent.go, pkg/agent/persist_test.go
Adds PersistAgentWorkspaceInfo, persists resolved configuration, validates required origin data, and ensures sanitized CLI options are written without mutating the caller.
Save configuration during workspace startup
cmd/internal/agentworkspace/up.go
Stores successful DevContainerConfigWithPath results in workspace information before forwarding the startup result.
Use workspace shutdown action in daemon checks
cmd/internal/agent_daemon.go, cmd/internal/agent_daemon_test.go
Resolves shutdown behavior from workspace configuration when present and tests workspace, fallback, and nil-workspace cases.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant UpCmd
  participant devsyUp
  participant AgentPersistence
  participant workspace.json
  participant Daemon
  UpCmd->>devsyUp: start workspace
  devsyUp-->>UpCmd: resolved devcontainer configuration
  UpCmd->>AgentPersistence: persist workspace configuration
  AgentPersistence->>workspace.json: write sanitized workspace info
  Daemon->>workspace.json: read workspace info
  Daemon->>Daemon: use resolved or fallback shutdown action
Loading

Possibly related PRs

  • devsy-org/devsy#88: Adds runtime enforcement for ShutdownAction, complementing this PR’s persistence and daemon resolution changes.
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 7.69% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly matches the main fix: handling shutdownAction=none on the first up to avoid the race condition.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@netlify

netlify Bot commented Jul 27, 2026

Copy link
Copy Markdown

Deploy Preview for devsydev canceled.

Name Link
🔨 Latest commit 4b378ee
🔍 Latest deploy log https://app.netlify.com/projects/devsydev/deploys/6a67b409aaa3ee000803af44

@skevetter skevetter changed the title fix(daemon): honor shutdownAction none on first up fix(daemon): race condition when shutdownAction is none on first up Jul 27, 2026
@skevetter
skevetter marked this pull request as ready for review July 27, 2026 19:28

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
cmd/internal/agent_daemon_test.go (1)

93-101: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Cover the existing-config/empty-action fallback shape.

workspaceWithShutdownAction("") returns without creating LastDevContainerConfig, so this only tests the nil-config fallback. Add a case where the config wrapper exists but ShutdownAction == "" to cover the separate production branch for older or partial workspace.json files.

Also applies to: 112-116

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cmd/internal/agent_daemon_test.go` around lines 93 - 101, Extend the tests
around workspaceWithShutdownAction to include a workspace whose
LastDevContainerConfig and nested Config exist while ShutdownAction is empty.
Exercise the production fallback path for an existing configuration with an
empty action, separately from the current nil-config case.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/internal/agentworkspace/up.go`:
- Around line 754-765: Update persistResolvedConfig and its startup caller so
PersistAgentWorkspaceInfo failures are propagated or retried and prevent a
successful startup result from being forwarded. Preserve the existing early
returns and successful persistence behavior, and add coverage for the
failed-persistence path to ensure stale shutdown policy is not accepted.

---

Nitpick comments:
In `@cmd/internal/agent_daemon_test.go`:
- Around line 93-101: Extend the tests around workspaceWithShutdownAction to
include a workspace whose LastDevContainerConfig and nested Config exist while
ShutdownAction is empty. Exercise the production fallback path for an existing
configuration with an empty action, separately from the current nil-config case.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4fa7b303-5549-4208-b61c-d9dbc4482733

📥 Commits

Reviewing files that changed from the base of the PR and between 6fbcdfd and 790ab92.

📒 Files selected for processing (5)
  • cmd/internal/agent_daemon.go
  • cmd/internal/agent_daemon_test.go
  • cmd/internal/agentworkspace/up.go
  • pkg/agent/agent.go
  • pkg/agent/persist_test.go

Comment on lines +754 to +765
func persistResolvedConfig(
workspaceInfo *provider.AgentWorkspaceInfo,
result *config2.Result,
) {
if result == nil || result.Error != "" || result.DevContainerConfigWithPath == nil {
return
}

workspaceInfo.LastDevContainerConfig = result.DevContainerConfigWithPath
if err := agent.PersistAgentWorkspaceInfo(workspaceInfo); err != nil {
log.Errorf("persist resolved devcontainer config: %v", err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗄️ Data Integrity & Integration | 🟠 Major | 🏗️ Heavy lift

Do not silently accept failed shutdown-policy persistence.

When PersistAgentWorkspaceInfo fails, this helper only logs and the caller still forwards a successful startup result. The daemon can then fall back to a stale or install-time action, reintroducing the shutdownAction: none failure when workspace.json was not updated. Retry or propagate this failure before reporting startup success, and test the failure path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@cmd/internal/agentworkspace/up.go` around lines 754 - 765, Update
persistResolvedConfig and its startup caller so PersistAgentWorkspaceInfo
failures are propagated or retried and prevent a successful startup result from
being forwarded. Preserve the existing early returns and successful persistence
behavior, and add coverage for the failed-persistence path to ensure stale
shutdown policy is not accepted.

@skevetter
skevetter marked this pull request as draft July 27, 2026 19:33
@skevetter
skevetter marked this pull request as ready for review July 27, 2026 20:12
@skevetter
skevetter enabled auto-merge (squash) July 27, 2026 20:15
@skevetter
skevetter disabled auto-merge July 27, 2026 20:15
@skevetter
skevetter merged commit d927382 into main Jul 27, 2026
65 of 66 checks passed
@skevetter
skevetter deleted the eloquent-sheep branch July 27, 2026 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant