[Phase 1] Improve agent execution resilience: file conflicts and verification recovery

[frankbria]

Problem

When executing tasks, the agent fails ungracefully in two common scenarios:

1. File Conflicts (file_create on existing files)

When a task is re-executed (e.g., after a previous failed run), the agent's plan still uses file_create for files that already exist from the previous attempt. This causes immediate step failures:

Step 1: src/task_tracker/exceptions.py → Failed: File already exists
Step 2: src/task_tracker/validators.py → Failed: File already exists

Expected behavior: The agent should detect existing files and fall back to file_edit or file_overwrite, or the planner should check for existing files before choosing the operation.

2. Cascading Verification Failures

After each file change, the agent runs incremental verification (ruff). When verification fails, the agent marks the step as failed and moves on, but downstream steps that depend on the failed file also fail:

Step 3: src/task_tracker/storage.py → stdout → Failed: Code verification failed
Step 4: src/task_tracker/commands/add.py → stdout → Failed: Code verification failed
Step 5: src/task_tracker/commands/delete.py → stdout → Failed: Code verification failed
... (all remaining steps fail)

This wastes significant time and API tokens — in the observed case, 893 seconds of execution with 20 steps all failing.

Expected behavior: The agent should either:

• Fix verification errors inline before proceeding (self-correction loop exists but doesn't trigger for incremental verification)
• Abort early after N consecutive failures instead of continuing through all 20 steps
• Re-plan remaining steps accounting for the verification failures

Proposed Improvements

File Conflict Handling

• In executor.py file_create: if file exists, fall back to overwrite (or file_edit if content differs)
• In planner.py: scan workspace for existing files before generating the plan, prefer file_edit when target exists
• Add a workspace cleanup command (cf work clean <task-id>) to remove artifacts from failed runs

Verification Recovery

• Apply self-correction loop to incremental verification failures (currently only runs on final verification)
• Add early abort threshold: if 3+ consecutive steps fail verification, pause and re-plan or escalate to blocker
• Track cumulative failure cost (time + tokens) and abort when threshold exceeded

Observability

• Log which verification gate failed and the specific error (currently just "Code verification failed")
• Surface verification error details in SSE output events so the Execution Monitor shows what went wrong

Context

Observed during cf-test workspace execution where all 12 tasks failed. 10 tasks had file_create conflicts from previous runs, and verification failures cascaded through all remaining steps.

Related

• Agent self-correction loop: core/agent.py _run_final_verification() / _attempt_verification_fix()
• Executor file operations: core/executor.py
• Verification gates: core/gates.py

[traycerai]

Plan

Observations

The codebase has a well-structured agent execution system with planner, executor, and verification gates. The main issues are: (1) file_create in file:codeframe/core/executor.py (lines 293-299) fails immediately when files exist without fallback, (2) incremental verification failures in file:codeframe/core/agent.py (lines 621-681) don't trigger early abort—the agent continues through all steps causing cascading failures, and (3) verification error details are captured in file:codeframe/core/gates.py but not surfaced in SSE events.

Approach

The plan addresses all three improvement areas from the ticket: file conflict handling, verification recovery, and observability. For file conflicts, implement fallback logic in executor and workspace-aware planning. For verification recovery, add early abort thresholds and apply self-correction to incremental verification. For observability, enhance SSE events with detailed verification errors. This maintains the existing architecture while adding resilience mechanisms.

Implementation Steps

1. File Conflict Handling

1.1 Add Fallback in Executor

File: file:codeframe/core/executor.py

Modify _execute_file_create method (lines 285-332):

• When file exists, check if content differs from what would be generated
• If content is identical or similar, return SUCCESS with message "File already exists with correct content"
• If content differs, log a warning and fall back to _execute_file_edit behavior
• Add a new parameter allow_overwrite: bool = True to control fallback behavior
• Update the error message to be more informative: "File already exists: {path}. Use file_edit to modify existing files."

1.2 Workspace-Aware Planning

File: file:codeframe/core/planner.py

Enhance _build_prompt method (lines 278-355):

• Add a new section listing existing files in the workspace that match the task's target files
• Query the workspace's file tree from context.file_tree to identify files that already exist
• Include this information in the planning prompt: "## Existing Files\nThe following files already exist in the workspace:\n- {file_path}\n\nUse file_edit instead of file_create for these files."
• This guides the LLM to choose the correct operation type

Update PLANNING_SYSTEM_PROMPT (lines 162-234):

• Add explicit guidance: "Before choosing file_create, check if the file already exists in the repository structure. If it exists, use file_edit instead."
• Emphasize in the "Planning Guidelines" section: "Always prefer file_edit over file_create when the target file already exists"

1.3 Workspace Cleanup Command

File: file:codeframe/cli/tasks_commands.py (or create new file:codeframe/cli/workspace_commands.py)

Add a new command cf work clean <task-id>:

• Create a new Typer app for workspace commands if it doesn't exist
• Implement clean command that:
  • Takes task_id as argument
  • Finds all files created during failed runs for that task (query from runs table and file_changes)
  • Prompts user for confirmation before deletion
  • Deletes files and updates task status to BACKLOG
  • Logs cleanup event to events table
• Add --force flag to skip confirmation
• Add --dry-run flag to show what would be deleted without actually deleting

2. Verification Recovery

2.1 Early Abort Threshold

File: file:codeframe/core/agent.py

Add early abort logic in _execute_plan method (lines 558-816):

• Add a new constant at module level: MAX_CONSECUTIVE_VERIFICATION_FAILURES = 3
• Track consecutive verification failures separately from step failures
• In the incremental verification section (lines 621-681), increment a counter when verification fails and self-correction doesn't fix it
• When consecutive_verification_failures >= MAX_CONSECUTIVE_VERIFICATION_FAILURES:
  • Log warning: "Too many consecutive verification failures, aborting execution"
  • Create a blocker with detailed information about the verification failures
  • Set self.state.status = AgentStatus.BLOCKED
  • Return early to stop execution
• Reset the counter when a verification passes

Add cumulative cost tracking:

• Track total execution time and estimated token usage
• Add constants: MAX_EXECUTION_TIME_SECONDS = 900 (15 minutes), MAX_ESTIMATED_TOKENS = 100000
• Before each step, check if thresholds exceeded
• If exceeded, create blocker and abort with message: "Execution exceeded resource limits (time: {time}s, tokens: ~{tokens})"

2.2 Apply Self-Correction to Incremental Verification

File: file:codeframe/core/agent.py

Enhance incremental verification handling (lines 621-681):

• The self-correction loop already exists but may not be triggered correctly
• Ensure that when _try_auto_fix fails (line 626), the self-correction loop is entered
• Currently the code enters self-correction (lines 645-680), but verify it's using the correct error context
• Improve the error message passed to _attempt_self_correction to include full ruff output from gate_result.checks
• Extract specific file paths and line numbers from ruff errors to help self-correction target the right locations

Modify _run_incremental_verification (lines 818-830):

• Add parameter verbose: bool = True to capture full error details
• Return the full GateResult with detailed output instead of summarized version
• Store the full verification output in self.state for later reference

2.3 Re-planning on Verification Failures

File: file:codeframe/core/agent.py

Add re-planning capability (new method):

• Create new method _replan_after_failure(failed_step: PlanStep, error: str) -> Optional[ImplementationPlan]
• This method:
  • Analyzes the failure and remaining steps
  • Calls the planner with updated context including the failure information
  • Generates a new plan for remaining work
  • Replaces self.state.plan.steps with the new steps
  • Resets self.state.current_step to 0
• Call this method when consecutive verification failures reach threshold (before creating blocker)
• Add a counter to limit re-planning attempts: MAX_REPLAN_ATTEMPTS = 2

3. Observability

3.1 Enhanced Verification Error Logging

File: file:codeframe/core/gates.py

Modify gate execution functions (_run_ruff, _run_pytest, etc.):

• Always capture full output in GateCheck.output field (don't truncate)
• Add a new field to GateCheck dataclass: detailed_errors: Optional[List[Dict[str, Any]]] = None
• For ruff, parse the output to extract structured error information:
  • File path
  • Line number
  • Error code
  • Error message
• Store this in detailed_errors as a list of dicts
• Keep the summarized output in the output field for backward compatibility

Update GateResult class:

• Add method get_error_summary() -> str that formats all errors in a readable way
• Add method get_errors_by_file() -> Dict[str, List[str]] to group errors by file

3.2 Surface Verification Errors in SSE Events

File: file:codeframe/core/agent.py

Enhance error event emission in _execute_plan (lines 628-631):

• When verification fails, emit a more detailed ErrorEvent
• Include the specific gate that failed (e.g., "ruff", "pytest")
• Include the error count and file paths affected
• Include a preview of the first few errors (up to 5)

Modify the event emission:

await self._publish_event(
    task_id,
    ErrorEvent(
        task_id=task_id,
        error_type="verification_failed",
        error=f"{gate_name} failed: {error_count} errors in {len(affected_files)} files",
        traceback=error_preview,  # First 5 errors formatted
    ),
)

File: file:codeframe/core/executor.py

Enhance execute_step_async method (lines 812-869):

• When a verification step fails, include the full verification output in the ErrorEvent
• Parse the output to extract key information (file paths, error types)
• Add this to the ErrorEvent's traceback field

3.3 Execution Monitor Display

File: file:codeframe/ui/routers/streaming_v2.py (or relevant SSE endpoint)

Ensure SSE events include all necessary fields:

• Verify that ErrorEvent with verification details is properly serialized
• Add event type "verification_failed" to the event stream
• Include gate name, error count, and affected files in the event data

File: Web UI components (if applicable)

Update the Execution Monitor component to:

• Display verification errors in a dedicated section
• Show which gate failed (ruff, pytest, etc.)
• Display affected files and error counts
• Provide expandable details for each error

4. Testing Considerations

While not explicitly requested, these changes should be validated:

4.1 File Conflict Tests

File: file:tests/core/test_executor.py

Add test cases:

• Test file_create with existing file (should fall back gracefully)
• Test planner choosing file_edit when file exists in context
• Test workspace cleanup command

4.2 Verification Recovery Tests

File: file:tests/core/test_agent.py

Add test cases:

• Test early abort after N consecutive verification failures
• Test cumulative cost tracking and abort
• Test self-correction on incremental verification failures
• Test re-planning after verification failures

4.3 Observability Tests

File: file:tests/core/test_gates.py

Add test cases:

• Test detailed error extraction from ruff output
• Test error grouping by file
• Test SSE event emission with verification details

Architecture Diagram

sequenceDiagram
    participant P as Planner
    participant E as Executor
    participant G as Gates
    participant A as Agent
    participant SSE as SSE Events

    Note over P: Phase 1: Planning
    P->>P: Check workspace for existing files
    P->>P: Choose file_edit vs file_create
    P->>A: Return plan

    Note over E,A: Phase 2: Execution
    loop For each step
        A->>E: Execute step
        alt File Create & File Exists
            E->>E: Check if content differs
            alt Content differs
                E->>E: Fallback to file_edit
            else Content same
                E->>A: SUCCESS (already exists)
            end
        end
        
        E->>A: Step result
        
        alt File operation succeeded
            A->>G: Run incremental verification (ruff)
            G->>A: Verification result with detailed errors
            
            alt Verification failed
                A->>A: Try auto-fix (ruff --fix)
                alt Auto-fix failed
                    A->>A: Self-correction loop (max 3 attempts)
                    alt Self-correction failed
                        A->>A: Increment consecutive_failures
                        alt consecutive_failures >= 3
                            A->>SSE: Emit detailed error event
                            A->>A: Create blocker & abort
                        end
                    end
                end
            end
        end
        
        A->>A: Check cumulative cost (time/tokens)
        alt Cost exceeded
            A->>A: Create blocker & abort
        end
    end

    Note over A,SSE: Phase 3: Observability
    A->>SSE: Emit progress events
    A->>SSE: Emit error events with details
    SSE->>SSE: Stream to UI/Monitor

Loading

Summary

This implementation plan addresses all three improvement areas from the ticket while maintaining the existing architecture. The changes are focused on adding resilience mechanisms (fallback, early abort, re-planning) and improving observability (detailed error events) without requiring major refactoring. The plan prioritizes the most impactful changes: file conflict fallback in executor, early abort threshold in agent, and enhanced error events for SSE streaming.

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a well-structured agent execution system with planner, executor, and verification gates. The main issues are: (1) `file_create` in `file:codeframe/core/executor.py` (lines 293-299) fails immediately when files exist without fallback, (2) incremental verification failures in `file:codeframe/core/agent.py` (lines 621-681) don't trigger early abort—the agent continues through all steps causing cascading failures, and (3) verification error details are captured in `file:codeframe/core/gates.py` but not surfaced in SSE events.

## Approach

The plan addresses all three improvement areas from the ticket: file conflict handling, verification recovery, and observability. For file conflicts, implement fallback logic in executor and workspace-aware planning. For verification recovery, add early abort thresholds and apply self-correction to incremental verification. For observability, enhance SSE events with detailed verification errors. This maintains the existing architecture while adding resilience mechanisms.

## Implementation Steps

### 1. File Conflict Handling

#### 1.1 Add Fallback in Executor
**File:** `file:codeframe/core/executor.py`

Modify `_execute_file_create` method (lines 285-332):
- When file exists, check if content differs from what would be generated
- If content is identical or similar, return SUCCESS with message "File already exists with correct content"
- If content differs, log a warning and fall back to `_execute_file_edit` behavior
- Add a new parameter `allow_overwrite: bool = True` to control fallback behavior
- Update the error message to be more informative: "File already exists: {path}. Use file_edit to modify existing files."

#### 1.2 Workspace-Aware Planning
**File:** `file:codeframe/core/planner.py`

Enhance `_build_prompt` method (lines 278-355):
- Add a new section listing existing files in the workspace that match the task's target files
- Query the workspace's file tree from `context.file_tree` to identify files that already exist
- Include this information in the planning prompt: "## Existing Files\nThe following files already exist in the workspace:\n- {file_path}\n\nUse file_edit instead of file_create for these files."
- This guides the LLM to choose the correct operation type

Update `PLANNING_SYSTEM_PROMPT` (lines 162-234):
- Add explicit guidance: "Before choosing file_create, check if the file already exists in the repository structure. If it exists, use file_edit instead."
- Emphasize in the "Planning Guidelines" section: "Always prefer file_edit over file_create when the target file already exists"

#### 1.3 Workspace Cleanup Command
**File:** `file:codeframe/cli/tasks_commands.py` (or create new `file:codeframe/cli/workspace_commands.py`)

Add a new command `cf work clean <task-id>`:
- Create a new Typer app for workspace commands if it doesn't exist
- Implement `clean` command that:
  - Takes task_id as argument
  - Finds all files created during failed runs for that task (query from runs table and file_changes)
  - Prompts user for confirmation before deletion
  - Deletes files and updates task status to BACKLOG
  - Logs cleanup event to events table
- Add `--force` flag to skip confirmation
- Add `--dry-run` flag to show what would be deleted without actually deleting

### 2. Verification Recovery

#### 2.1 Early Abort Threshold
**File:** `file:codeframe/core/agent.py`

Add early abort logic in `_execute_plan` method (lines 558-816):
- Add a new constant at module level: `MAX_CONSECUTIVE_VERIFICATION_FAILURES = 3`
- Track consecutive verification failures separately from step failures
- In the incremental verification section (lines 621-681), increment a counter when verification fails and self-correction doesn't fix it
- When `consecutive_verification_failures >= MAX_CONSECUTIVE_VERIFICATION_FAILURES`:
  - Log warning: "Too many consecutive verification failures, aborting execution"
  - Create a blocker with detailed information about the verification failures
  - Set `self.state.status = AgentStatus.BLOCKED`
  - Return early to stop execution
- Reset the counter when a verification passes

Add cumulative cost tracking:
- Track total execution time and estimated token usage
- Add constants: `MAX_EXECUTION_TIME_SECONDS = 900` (15 minutes), `MAX_ESTIMATED_TOKENS = 100000`
- Before each step, check if thresholds exceeded
- If exceeded, create blocker and abort with message: "Execution exceeded resource limits (time: {time}s, tokens: ~{tokens})"

#### 2.2 Apply Self-Correction to Incremental Verification
**File:** `file:codeframe/core/agent.py`

Enhance incremental verification handling (lines 621-681):
- The self-correction loop already exists but may not be triggered correctly
- Ensure that when `_try_auto_fix` fails (line 626), the self-correction loop is entered
- Currently the code enters self-correction (lines 645-680), but verify it's using the correct error context
- Improve the error message passed to `_attempt_self_correction` to include full ruff output from `gate_result.checks`
- Extract specific file paths and line numbers from ruff errors to help self-correction target the right locations

Modify `_run_incremental_verification` (lines 818-830):
- Add parameter `verbose: bool = True` to capture full error details
- Return the full GateResult with detailed output instead of summarized version
- Store the full verification output in `self.state` for later reference

#### 2.3 Re-planning on Verification Failures
**File:** `file:codeframe/core/agent.py`

Add re-planning capability (new method):
- Create new method `_replan_after_failure(failed_step: PlanStep, error: str) -> Optional[ImplementationPlan]`
- This method:
  - Analyzes the failure and remaining steps
  - Calls the planner with updated context including the failure information
  - Generates a new plan for remaining work
  - Replaces `self.state.plan.steps` with the new steps
  - Resets `self.state.current_step` to 0
- Call this method when consecutive verification failures reach threshold (before creating blocker)
- Add a counter to limit re-planning attempts: `MAX_REPLAN_ATTEMPTS = 2`

### 3. Observability

#### 3.1 Enhanced Verification Error Logging
**File:** `file:codeframe/core/gates.py`

Modify gate execution functions (`_run_ruff`, `_run_pytest`, etc.):
- Always capture full output in `GateCheck.output` field (don't truncate)
- Add a new field to `GateCheck` dataclass: `detailed_errors: Optional[List[Dict[str, Any]]] = None`
- For ruff, parse the output to extract structured error information:
  - File path
  - Line number
  - Error code
  - Error message
- Store this in `detailed_errors` as a list of dicts
- Keep the summarized output in the `output` field for backward compatibility

Update `GateResult` class:
- Add method `get_error_summary() -> str` that formats all errors in a readable way
- Add method `get_errors_by_file() -> Dict[str, List[str]]` to group errors by file

#### 3.2 Surface Verification Errors in SSE Events
**File:** `file:codeframe/core/agent.py`

Enhance error event emission in `_execute_plan` (lines 628-631):
- When verification fails, emit a more detailed ErrorEvent
- Include the specific gate that failed (e.g., "ruff", "pytest")
- Include the error count and file paths affected
- Include a preview of the first few errors (up to 5)

Modify the event emission:
```python
await self._publish_event(
    task_id,
    ErrorEvent(
        task_id=task_id,
        error_type="verification_failed",
        error=f"{gate_name} failed: {error_count} errors in {len(affected_files)} files",
        traceback=error_preview,  # First 5 errors formatted
    ),
)
```

**File:** `file:codeframe/core/executor.py`

Enhance `execute_step_async` method (lines 812-869):
- When a verification step fails, include the full verification output in the ErrorEvent
- Parse the output to extract key information (file paths, error types)
- Add this to the ErrorEvent's traceback field

#### 3.3 Execution Monitor Display
**File:** `file:codeframe/ui/routers/streaming_v2.py` (or relevant SSE endpoint)

Ensure SSE events include all necessary fields:
- Verify that ErrorEvent with verification details is properly serialized
- Add event type "verification_failed" to the event stream
- Include gate name, error count, and affected files in the event data

**File:** Web UI components (if applicable)

Update the Execution Monitor component to:
- Display verification errors in a dedicated section
- Show which gate failed (ruff, pytest, etc.)
- Display affected files and error counts
- Provide expandable details for each error

### 4. Testing Considerations

While not explicitly requested, these changes should be validated:

#### 4.1 File Conflict Tests
**File:** `file:tests/core/test_executor.py`

Add test cases:
- Test `file_create` with existing file (should fall back gracefully)
- Test planner choosing `file_edit` when file exists in context
- Test workspace cleanup command

#### 4.2 Verification Recovery Tests
**File:** `file:tests/core/test_agent.py`

Add test cases:
- Test early abort after N consecutive verification failures
- Test cumulative cost tracking and abort
- Test self-correction on incremental verification failures
- Test re-planning after verification failures

#### 4.3 Observability Tests
**File:** `file:tests/core/test_gates.py`

Add test cases:
- Test detailed error extraction from ruff output
- Test error grouping by file
- Test SSE event emission with verification details

## Architecture Diagram

```mermaid
sequenceDiagram
    participant P as Planner
    participant E as Executor
    participant G as Gates
    participant A as Agent
    participant SSE as SSE Events

    Note over P: Phase 1: Planning
    P->>P: Check workspace for existing files
    P->>P: Choose file_edit vs file_create
    P->>A: Return plan

    Note over E,A: Phase 2: Execution
    loop For each step
        A->>E: Execute step
        alt File Create & File Exists
            E->>E: Check if content differs
            alt Content differs
                E->>E: Fallback to file_edit
            else Content same
                E->>A: SUCCESS (already exists)
            end
        end
        
        E->>A: Step result
        
        alt File operation succeeded
            A->>G: Run incremental verification (ruff)
            G->>A: Verification result with detailed errors
            
            alt Verification failed
                A->>A: Try auto-fix (ruff --fix)
                alt Auto-fix failed
                    A->>A: Self-correction loop (max 3 attempts)
                    alt Self-correction failed
                        A->>A: Increment consecutive_failures
                        alt consecutive_failures >= 3
                            A->>SSE: Emit detailed error event
                            A->>A: Create blocker & abort
                        end
                    end
                end
            end
        end
        
        A->>A: Check cumulative cost (time/tokens)
        alt Cost exceeded
            A->>A: Create blocker & abort
        end
    end

    Note over A,SSE: Phase 3: Observability
    A->>SSE: Emit progress events
    A->>SSE: Emit error events with details
    SSE->>SSE: Stream to UI/Monitor
```

## Summary

This implementation plan addresses all three improvement areas from the ticket while maintaining the existing architecture. The changes are focused on adding resilience mechanisms (fallback, early abort, re-planning) and improving observability (detailed error events) without requiring major refactoring. The plan prioritizes the most impactful changes: file conflict fallback in executor, early abort threshold in agent, and enhanced error events for SSE streaming.

---

Execution Information

Branch: main
Commit: 02ce2fc

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Related PRs

#125 - Implement WorkerAgent execute_task method [merged]
#151 - Integrate QualityTracker with workflow system [merged]
#246 - feat(agents): trigger multi-agent execution after task approval [merged]
#300 - feat(diagnostics): add self-diagnosis system for failed task analysis [merged]
#319 - feat(cli): add cf work follow for live execution streaming [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!