[Phase 2.5] Per-edit lint gate returns FAILED when linter binary is missing

[frankbria]

Problem

gates.run_lint_on_file() returns GateStatus.FAILED when the linter binary (e.g., ruff) is not installed in the target project's venv. This causes the ReactAgent to waste all 30 iterations trying to "fix" phantom lint errors that don't actually exist.

Root Cause

In codeframe/core/gates.py, the run_lint_on_file() function:

1. Checks if the binary is available via shutil.which(cfg.check_available) (line 651)
2. If not found but cfg.use_uv is True and uv is on PATH, it proceeds (line 652-653)
3. Runs ["uv", "run", "ruff", "check", ...] in the target project directory
4. If uv run ruff fails because ruff isn't a dependency (e.g., it's under optional dev deps only), returncode != 0 → GateStatus.FAILED

The gate cannot distinguish between "ruff not found" (exit code 2, stderr: "Failed to spawn: ruff") and "ruff found lint errors" (exit code 1, stdout: error details). Both result in GateStatus.FAILED.

Impact

• Every file creation by the ReactAgent gets marked FAILED
• Agent wastes all 30 iterations trying to fix errors that don't exist
• Effectively makes the agent unable to complete any task on projects without dev dependencies installed

Observed Behavior

[ReactAgent] Tool: create_file
AGENT_STEP_COMPLETED path=src/task_tracker/models.py
AGENT_STEP_COMPLETED path=src/task_tracker/models.py status=FAILED  ← ruff not found

After installing ruff: same file creation now produces real lint errors the agent CAN fix.

Proposed Fix

In gates.run_lint_on_file(), after running the subprocess:

1. Check stderr for "Failed to spawn" or similar tool-not-found patterns
2. If the tool was not found, return GateStatus.SKIPPED instead of GateStatus.FAILED
3. Alternatively, parse exit code 2 specifically (ruff uses 2 for "command not found" vs 1 for "lint errors found")

Files

• codeframe/core/gates.py — run_lint_on_file() function (line 631+)
• codeframe/core/react_agent.py — _execute_tool_with_lint() and _run_lint_on_file() (lines 564+)

Test

After fix: running a task on a project without ruff installed should show status=SKIPPED for lint checks, not status=FAILED.

[traycerai]

Plan

Observations

The run_lint_on_file() function in file:codeframe/core/gates.py (lines 631-700) currently cannot distinguish between "linter binary not found" and "lint errors detected". When use_uv=True and uv is available, the function proceeds to execute uv run <linter> even if the linter isn't installed in the target project's dependencies. If the tool is missing, uv run fails with exit code 2 and stderr containing "Failed to spawn", but this gets treated as GateStatus.FAILED instead of GateStatus.SKIPPED, causing the ReactAgent to waste iterations trying to fix phantom errors.

Approach

The fix involves enhancing the error detection logic in run_lint_on_file() to detect when a linter tool is not found versus when it found actual lint errors. After subprocess execution, inspect stderr for tool-not-found patterns (e.g., "Failed to spawn", "No such file or directory") and return GateStatus.SKIPPED instead of GateStatus.FAILED. This aligns with the existing behavior when the binary check fails upfront (lines 651-654) and matches the test expectation in file:tests/core/test_gates_observability.py (lines 249-256).

Implementation Steps

1. Update run_lint_on_file() in file:codeframe/core/gates.py

Location: Lines 631-700

Changes:

After the subprocess execution completes (after line 675), add logic to detect tool-not-found scenarios:

• Define a list of stderr patterns that indicate the tool was not found:

  • "Failed to spawn" - uv's error when command not found
  • "No such file or directory" - common OS error
  • "command not found" - shell error
  • "not found" combined with the tool name

• After capturing stderr (line 673-674), check if any of these patterns appear in stderr

• If a tool-not-found pattern is detected:

  • Return GateCheck with status=GateStatus.SKIPPED
  • Set output to a descriptive message like "{cfg.name} not found in project dependencies"
  • Do not set detailed_errors

• Otherwise, proceed with the existing logic (lines 677-689)

Specific implementation details:

# After line 675 (output = output.strip())
# Add tool-not-found detection

# Patterns indicating the linter tool is not installed
tool_not_found_patterns = [
    "Failed to spawn",
    "No such file or directory",
    "command not found",
]

# Check stderr for tool-not-found indicators
stderr_lower = result.stderr.lower() if result.stderr else ""
tool_name_in_error = cfg.check_available and cfg.check_available.lower() in stderr_lower

if result.returncode != 0 and result.stderr:
    for pattern in tool_not_found_patterns:
        if pattern.lower() in stderr_lower:
            return GateCheck(
                name=cfg.name,
                status=GateStatus.SKIPPED,
                output=f"{cfg.name} not found in project dependencies",
                duration_ms=duration_ms,
            )
    
    # Additional check: if tool name appears with "not found"
    if tool_name_in_error and "not found" in stderr_lower:
        return GateCheck(
            name=cfg.name,
            status=GateStatus.SKIPPED,
            output=f"{cfg.name} not found in project dependencies",
            duration_ms=duration_ms,
        )

# Continue with existing logic for actual lint errors

2. Update Test Coverage in file:tests/core/test_gates_observability.py

Location: Add new test after line 256

Changes:

Add a test case that simulates the uv run scenario where the tool is not found:

• Test name: test_uv_run_missing_tool_returns_skipped
• Mock subprocess.run to return:
  • returncode=2 (or non-zero)
  • stderr="Failed to spawn: ruff\nCaused by: No such file or directory (os error 2)"
• Mock shutil.which to return "/usr/bin/uv" for "uv" and None for "ruff"
• Verify that run_lint_on_file() returns GateStatus.SKIPPED
• Verify the output message indicates the tool was not found

3. Verify ReactAgent Behavior

Location: file:codeframe/core/react_agent.py, lines 584-640

No code changes needed - verify that the existing logic correctly handles GateStatus.SKIPPED:

• Line 637-640: Only GateStatus.FAILED is surfaced to the LLM
• GateStatus.SKIPPED returns empty string, preventing phantom error messages
• This is the desired behavior

4. Update Documentation

Location: file:codeframe/core/gates.py, docstring for run_lint_on_file()

Changes:

Update the docstring (lines 637-641) to clarify the SKIPPED status behavior:

"""Run the appropriate linter on a single file.

Returns a ``GateCheck`` with status PASSED / FAILED / SKIPPED / ERROR.
SKIPPED is returned when:
- No linter is registered for the file extension
- The required binary is not installed (checked via shutil.which)
- The tool is not found in the project's dependencies (when using uv run)
"""

Testing Strategy

Manual Testing

1. Create a test project without ruff in dependencies
2. Run ReactAgent with a task that creates/edits Python files
3. Verify that lint checks return status=SKIPPED instead of status=FAILED
4. Verify that the agent doesn't waste iterations trying to fix phantom errors

Automated Testing

1. Run existing test: test_missing_linter_binary_skips (line 249-256)
2. Run new test: test_uv_run_missing_tool_returns_skipped
3. Verify all gate-related tests pass: pytest tests/core/test_gates_observability.py -v

Sequence Diagram

sequenceDiagram
    participant RA as ReactAgent
    participant G as gates.run_lint_on_file()
    participant SP as subprocess.run()
    participant UV as uv run ruff

    RA->>G: run_lint_on_file(file_path)
    G->>G: Find linter config (ruff)
    G->>G: Check if ruff available
    Note over G: ruff not found, but uv is available
    G->>SP: subprocess.run(["uv", "run", "ruff", ...])
    SP->>UV: Execute uv run ruff
    UV-->>SP: Exit code 2, stderr: "Failed to spawn: ruff"
    SP-->>G: result (returncode=2, stderr=...)
    G->>G: Detect "Failed to spawn" in stderr
    G-->>RA: GateCheck(status=SKIPPED)
    Note over RA: Empty string returned, no phantom errors

Loading

Files Modified

• file:codeframe/core/gates.py - Add tool-not-found detection logic in run_lint_on_file()
• file:tests/core/test_gates_observability.py - Add test for uv run missing tool scenario

Expected Outcome

After implementing this fix:

1. When a linter binary is not found in the project's dependencies, run_lint_on_file() returns GateStatus.SKIPPED
2. ReactAgent no longer receives phantom lint errors for missing tools
3. Agent iterations are not wasted trying to fix non-existent errors
4. The behavior is consistent whether the tool is missing from PATH or from project dependencies

Import In IDE

🤖 Prompt for AI Agents

## Observations

The `run_lint_on_file()` function in `file:codeframe/core/gates.py` (lines 631-700) currently cannot distinguish between "linter binary not found" and "lint errors detected". When `use_uv=True` and `uv` is available, the function proceeds to execute `uv run <linter>` even if the linter isn't installed in the target project's dependencies. If the tool is missing, `uv run` fails with exit code 2 and stderr containing "Failed to spawn", but this gets treated as `GateStatus.FAILED` instead of `GateStatus.SKIPPED`, causing the ReactAgent to waste iterations trying to fix phantom errors.

## Approach

The fix involves enhancing the error detection logic in `run_lint_on_file()` to detect when a linter tool is not found versus when it found actual lint errors. After subprocess execution, inspect stderr for tool-not-found patterns (e.g., "Failed to spawn", "No such file or directory") and return `GateStatus.SKIPPED` instead of `GateStatus.FAILED`. This aligns with the existing behavior when the binary check fails upfront (lines 651-654) and matches the test expectation in `file:tests/core/test_gates_observability.py` (lines 249-256).

## Implementation Steps

### 1. Update `run_lint_on_file()` in `file:codeframe/core/gates.py`

**Location:** Lines 631-700

**Changes:**

After the subprocess execution completes (after line 675), add logic to detect tool-not-found scenarios:

- Define a list of stderr patterns that indicate the tool was not found:
  - `"Failed to spawn"` - uv's error when command not found
  - `"No such file or directory"` - common OS error
  - `"command not found"` - shell error
  - `"not found"` combined with the tool name

- After capturing stderr (line 673-674), check if any of these patterns appear in stderr

- If a tool-not-found pattern is detected:
  - Return `GateCheck` with `status=GateStatus.SKIPPED`
  - Set output to a descriptive message like `"{cfg.name} not found in project dependencies"`
  - Do not set `detailed_errors`

- Otherwise, proceed with the existing logic (lines 677-689)

**Specific implementation details:**

```python
# After line 675 (output = output.strip())
# Add tool-not-found detection

# Patterns indicating the linter tool is not installed
tool_not_found_patterns = [
    "Failed to spawn",
    "No such file or directory",
    "command not found",
]

# Check stderr for tool-not-found indicators
stderr_lower = result.stderr.lower() if result.stderr else ""
tool_name_in_error = cfg.check_available and cfg.check_available.lower() in stderr_lower

if result.returncode != 0 and result.stderr:
    for pattern in tool_not_found_patterns:
        if pattern.lower() in stderr_lower:
            return GateCheck(
                name=cfg.name,
                status=GateStatus.SKIPPED,
                output=f"{cfg.name} not found in project dependencies",
                duration_ms=duration_ms,
            )
    
    # Additional check: if tool name appears with "not found"
    if tool_name_in_error and "not found" in stderr_lower:
        return GateCheck(
            name=cfg.name,
            status=GateStatus.SKIPPED,
            output=f"{cfg.name} not found in project dependencies",
            duration_ms=duration_ms,
        )

# Continue with existing logic for actual lint errors
```

### 2. Update Test Coverage in `file:tests/core/test_gates_observability.py`

**Location:** Add new test after line 256

**Changes:**

Add a test case that simulates the `uv run` scenario where the tool is not found:

- Test name: `test_uv_run_missing_tool_returns_skipped`
- Mock `subprocess.run` to return:
  - `returncode=2` (or non-zero)
  - `stderr="Failed to spawn: ruff\nCaused by: No such file or directory (os error 2)"`
- Mock `shutil.which` to return `"/usr/bin/uv"` for "uv" and `None` for "ruff"
- Verify that `run_lint_on_file()` returns `GateStatus.SKIPPED`
- Verify the output message indicates the tool was not found

### 3. Verify ReactAgent Behavior

**Location:** `file:codeframe/core/react_agent.py`, lines 584-640

**No code changes needed** - verify that the existing logic correctly handles `GateStatus.SKIPPED`:

- Line 637-640: Only `GateStatus.FAILED` is surfaced to the LLM
- `GateStatus.SKIPPED` returns empty string, preventing phantom error messages
- This is the desired behavior

### 4. Update Documentation

**Location:** `file:codeframe/core/gates.py`, docstring for `run_lint_on_file()`

**Changes:**

Update the docstring (lines 637-641) to clarify the SKIPPED status behavior:

```python
"""Run the appropriate linter on a single file.

Returns a ``GateCheck`` with status PASSED / FAILED / SKIPPED / ERROR.
SKIPPED is returned when:
- No linter is registered for the file extension
- The required binary is not installed (checked via shutil.which)
- The tool is not found in the project's dependencies (when using uv run)
"""
```

## Testing Strategy

### Manual Testing

1. Create a test project without ruff in dependencies
2. Run ReactAgent with a task that creates/edits Python files
3. Verify that lint checks return `status=SKIPPED` instead of `status=FAILED`
4. Verify that the agent doesn't waste iterations trying to fix phantom errors

### Automated Testing

1. Run existing test: `test_missing_linter_binary_skips` (line 249-256)
2. Run new test: `test_uv_run_missing_tool_returns_skipped`
3. Verify all gate-related tests pass: `pytest tests/core/test_gates_observability.py -v`

## Sequence Diagram

```mermaid
sequenceDiagram
    participant RA as ReactAgent
    participant G as gates.run_lint_on_file()
    participant SP as subprocess.run()
    participant UV as uv run ruff

    RA->>G: run_lint_on_file(file_path)
    G->>G: Find linter config (ruff)
    G->>G: Check if ruff available
    Note over G: ruff not found, but uv is available
    G->>SP: subprocess.run(["uv", "run", "ruff", ...])
    SP->>UV: Execute uv run ruff
    UV-->>SP: Exit code 2, stderr: "Failed to spawn: ruff"
    SP-->>G: result (returncode=2, stderr=...)
    G->>G: Detect "Failed to spawn" in stderr
    G-->>RA: GateCheck(status=SKIPPED)
    Note over RA: Empty string returned, no phantom errors
```

## Files Modified

- `file:codeframe/core/gates.py` - Add tool-not-found detection logic in `run_lint_on_file()`
- `file:tests/core/test_gates_observability.py` - Add test for uv run missing tool scenario

## Expected Outcome

After implementing this fix:

1. When a linter binary is not found in the project's dependencies, `run_lint_on_file()` returns `GateStatus.SKIPPED`
2. ReactAgent no longer receives phantom lint errors for missing tools
3. Agent iterations are not wasted trying to fix non-existent errors
4. The behavior is consistent whether the tool is missing from PATH or from project dependencies

---

Execution Information

Branch: main
Commit: e04112c

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

🔗 Related PRs

#28 - feat: Complete session lifecycle implementation and fix integration tests [merged]
#342 - feat: agent execution resilience - file conflicts, verification recovery, observability [merged]
#356 - fix: agent execution resilience + Phase 2.5 architecture research [merged]
#366 - feat: language-aware per-edit lint gate in ReactAgent [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!