[P1] Discovery enters stuck state on new project creation - no current question

[frankbria]

Problem

Newly created projects enter a "stuck state" in the discovery phase where no current question is available. The backend logs show repeated warnings about this condition.

Backend Logs (repeated for each new project)

Project 89: Discovery in stuck state - no current question
Project 90: Discovery in stuck state - no current question
Project 91: Discovery in stuck state - no current question
...

Observed Pattern

• Issue occurs immediately after project creation
• Affects projects created via UI during E2E tests
• Projects are in discovery phase but have no question to display
• May be related to [P1] Restart Discovery button non functional #247 (Restart Discovery non-functional) but affects NEW projects

Impact

• New project creation appears broken to users
• Discovery can't proceed without a question
• User left on dashboard with no actionable state
• Forces manual intervention or project recreation

Possible Root Causes

1. Discovery initialization not generating first question
2. Race condition between project creation and discovery start
3. Question generation fails silently
4. Discovery state not properly initialized

Related Issues

• [P1] Restart Discovery button non functional #247 - Restart Discovery button non functional (different: affects existing projects)

Affected Files

• codeframe/discovery/ module
• codeframe/ui/routers/discovery.py
• codeframe/agents/lead_agent.py (discovery orchestration)

Expected Behavior

1. User creates new project
2. Discovery starts automatically
3. First discovery question is immediately available
4. User can answer questions without intervention

[traycerai]

Plan

Observations

The issue occurs when new projects are created via the UI. The flow is: (1) Project creation via POST /api/projects creates the project record and workspace but does NOT start discovery, (2) ProjectList.tsx automatically calls POST /api/projects/{id}/start to start the agent and discovery, (3) start_agent() in file:codeframe/ui/shared.py calls agent.start_discovery() which should generate the first question, (4) If start_discovery() fails or is interrupted before setting _current_question_id and _current_question_text, the state becomes "discovering" but without a question, (5) When get_discovery_status() is called (line 641-644 in file:codeframe/agents/lead_agent.py), it detects this stuck state and logs the warning. The frontend's DiscoveryProgress component has timeout detection (10 seconds) and a "Restart Discovery" button, but the backend logs show the issue occurs immediately on project creation.

Approach

The root cause is a race condition or failure in the start_discovery() method where the state is saved as "discovering" (line 387-388 in file:codeframe/agents/lead_agent.py) before the question is generated and saved. If the AI question generation fails or times out, the fallback mechanisms may also fail, leaving the state inconsistent. The fix involves: (1) Implementing atomic state updates - only save "discovering" state AFTER successfully generating and storing the first question, (2) Adding robust error handling with automatic recovery to idle state if question generation fails, (3) Ensuring all three fallback paths (AI-generated, framework question, default question) properly set both _current_question_id and _current_question_text before saving state, (4) Adding validation in _save_discovery_state() to prevent saving "discovering" state without a question. This ensures new projects either start discovery successfully with a question, or remain in "idle" state for manual retry.

Implementation Steps

1. Fix Atomic State Update in start_discovery() Method

File: file:codeframe/agents/lead_agent.py (lines 372-458)

Modify the start_discovery() method to ensure atomic state updates:

• Move state transition: Currently line 387 sets self._discovery_state = "discovering" and line 388 saves it immediately. This should be moved to AFTER successful question generation
• Restructure the method:
  • First, generate the question (AI, framework fallback, or default fallback)
  • Store the question ID and text in local variables
  • Only then set _discovery_state = "discovering", _current_question_id, and _current_question_text
  • Call _save_discovery_state() once at the end with all values set
• Add error handling: Wrap the entire question generation logic in try-except. If all three fallback mechanisms fail, log the error and leave state as "idle" instead of transitioning to "discovering"
• Validation: Before calling _save_discovery_state(), assert that both _current_question_id and _current_question_text are not None/empty

2. Add State Validation in _save_discovery_state() Method

File: file:codeframe/agents/lead_agent.py (lines 294-327)

Add validation to prevent saving inconsistent state:

• Add validation check at the beginning of _save_discovery_state():
  • If self._discovery_state == "discovering" but self._current_question_id is None or empty, log a critical error and raise an exception
  • This prevents the stuck state from being persisted to the database
• Add debug logging: Log the state being saved (state, question_id, question_text) for troubleshooting

3. Improve Error Handling in start_agent() Function

File: file:codeframe/ui/shared.py (lines 309-348)

Enhance error handling when starting discovery:

• Expand try-except block around agent.start_discovery() (lines 310-312):
  • Currently catches generic Exception and broadcasts discovery_error but continues with agent startup
  • Add specific handling for the case where discovery fails to generate a question
  • If discovery fails, consider whether to remove the agent from running_agents dict or leave it for manual retry
• Add timeout: Wrap agent.start_discovery() in asyncio.wait_for() with a reasonable timeout (e.g., 30 seconds) to prevent indefinite hangs
• Improve error broadcast: Include more context in the discovery_error WebSocket message (e.g., whether it's a timeout, API failure, or question generation failure)

4. Add Recovery Mechanism in get_discovery_status() Method

File: file:codeframe/agents/lead_agent.py (lines 639-647)

Enhance the stuck state detection with automatic recovery:

• Current behavior: Lines 641-644 detect stuck state and set needs_recovery flag, but don't fix it
• Add auto-recovery: When stuck state is detected:
  • Log the detection with full context (project_id, state, question_id, question_text)
  • Automatically call self.reset_discovery() to transition back to "idle" state
  • Update the status dict to reflect the reset state
  • This prevents the stuck state from persisting across multiple API calls
• Add recovery metadata: Include in the status dict: {"auto_recovered": True, "recovery_timestamp": <timestamp>} so the frontend knows recovery occurred

5. Enhance Frontend Error Handling

File: file:web-ui/src/components/DiscoveryProgress.tsx (lines 346-379)

Improve the stuck state detection and user feedback:

• Reduce timeout: The current STUCK_TIMEOUT_MS is 10 seconds (line 52). Consider reducing to 5 seconds for faster detection
• Auto-retry: When stuck state is detected (line 370), instead of just showing the "Restart Discovery" button, automatically call handleRestartDiscovery() once. If it fails again, then show the manual button
• Better error messages: Update the stuck state UI (around line 700-750) to explain what happened: "Discovery failed to start. We've automatically reset it. Click 'Start Discovery' to try again."
• Add telemetry: Log stuck state occurrences to help track how often this happens in production

6. Add Integration Test for Race Condition

File: file:tests/discovery/test_discovery_integration.py

Add a test case to verify the fix:

• Test name: test_start_discovery_handles_ai_failure_gracefully
• Setup: Mock AnthropicProvider.send_message() to raise an exception
• Action: Call agent.start_discovery()
• Assertions:
  • Verify state remains "idle" (not "discovering")
  • Verify _current_question_id is None
  • Verify no stuck state is persisted to database
  • Verify calling get_discovery_status() returns idle state without warnings

7. Add Monitoring and Logging

Files: file:codeframe/agents/lead_agent.py, file:codeframe/ui/shared.py

Improve observability:

• Add structured logging in start_discovery():
  • Log when entering the method with project_id and project_description length
  • Log which fallback path is taken (AI, framework, default)
  • Log the generated question ID and text length
  • Log when state transition completes successfully
• Add metrics: If the codebase has a metrics system, add counters for:
  • discovery_start_success
  • discovery_start_failure
  • discovery_stuck_state_detected
  • discovery_auto_recovered

Sequence Diagram

sequenceDiagram
    participant UI as ProjectList UI
    participant API as POST /api/projects/{id}/start
    participant SA as start_agent()
    participant LA as LeadAgent
    participant DB as Database
    participant WS as WebSocket

    UI->>API: Start project after creation
    API->>SA: start_agent(project_id)
    SA->>LA: new LeadAgent(project_id)
    LA->>DB: Load discovery state (idle)
    SA->>LA: start_discovery(description)
    
    alt AI Question Generation Success
        LA->>LA: Generate AI question
        LA->>LA: Set state="discovering", question_id, question_text
        LA->>DB: Save state atomically
        LA-->>SA: Return question text
        SA->>WS: Broadcast discovery_question_ready
    else AI Fails, Framework Fallback Success
        LA->>LA: AI generation fails
        LA->>LA: Get framework question
        LA->>LA: Set state="discovering", question_id, question_text
        LA->>DB: Save state atomically
        LA-->>SA: Return question text
        SA->>WS: Broadcast discovery_question_ready
    else All Fallbacks Fail
        LA->>LA: All question generation fails
        LA->>LA: Keep state="idle" (NO state change)
        LA-->>SA: Raise exception
        SA->>WS: Broadcast discovery_error
        SA->>UI: Continue (agent running, discovery idle)
    end
    
    Note over LA,DB: CRITICAL: State only changes to "discovering"<br/>AFTER question is successfully generated

Loading

Validation Checklist

• start_discovery() only transitions to "discovering" state after successfully generating a question
• All three fallback paths (AI, framework, default) set both _current_question_id and _current_question_text before saving state
• _save_discovery_state() validates that "discovering" state has a question before persisting
• get_discovery_status() auto-recovers from stuck state by calling reset_discovery()
• start_agent() has timeout protection and improved error handling
• Frontend reduces stuck detection timeout and implements auto-retry
• Integration test verifies graceful handling of AI failures
• Structured logging added for troubleshooting
• E2E test test_project_creation.spec.ts passes without stuck state warnings in logs

Import In IDE

🤖 Prompt for AI Agents

## Observations

The issue occurs when new projects are created via the UI. The flow is: (1) Project creation via `POST /api/projects` creates the project record and workspace but does NOT start discovery, (2) `ProjectList.tsx` automatically calls `POST /api/projects/{id}/start` to start the agent and discovery, (3) `start_agent()` in `file:codeframe/ui/shared.py` calls `agent.start_discovery()` which should generate the first question, (4) If `start_discovery()` fails or is interrupted before setting `_current_question_id` and `_current_question_text`, the state becomes "discovering" but without a question, (5) When `get_discovery_status()` is called (line 641-644 in `file:codeframe/agents/lead_agent.py`), it detects this stuck state and logs the warning. The frontend's `DiscoveryProgress` component has timeout detection (10 seconds) and a "Restart Discovery" button, but the backend logs show the issue occurs immediately on project creation.

## Approach

The root cause is a race condition or failure in the `start_discovery()` method where the state is saved as "discovering" (line 387-388 in `file:codeframe/agents/lead_agent.py`) before the question is generated and saved. If the AI question generation fails or times out, the fallback mechanisms may also fail, leaving the state inconsistent. The fix involves: (1) Implementing atomic state updates - only save "discovering" state AFTER successfully generating and storing the first question, (2) Adding robust error handling with automatic recovery to idle state if question generation fails, (3) Ensuring all three fallback paths (AI-generated, framework question, default question) properly set both `_current_question_id` and `_current_question_text` before saving state, (4) Adding validation in `_save_discovery_state()` to prevent saving "discovering" state without a question. This ensures new projects either start discovery successfully with a question, or remain in "idle" state for manual retry.

## Implementation Steps

### 1. Fix Atomic State Update in `start_discovery()` Method
**File:** `file:codeframe/agents/lead_agent.py` (lines 372-458)

Modify the `start_discovery()` method to ensure atomic state updates:

- **Move state transition**: Currently line 387 sets `self._discovery_state = "discovering"` and line 388 saves it immediately. This should be moved to AFTER successful question generation
- **Restructure the method**:
  - First, generate the question (AI, framework fallback, or default fallback)
  - Store the question ID and text in local variables
  - Only then set `_discovery_state = "discovering"`, `_current_question_id`, and `_current_question_text`
  - Call `_save_discovery_state()` once at the end with all values set
- **Add error handling**: Wrap the entire question generation logic in try-except. If all three fallback mechanisms fail, log the error and leave state as "idle" instead of transitioning to "discovering"
- **Validation**: Before calling `_save_discovery_state()`, assert that both `_current_question_id` and `_current_question_text` are not None/empty

### 2. Add State Validation in `_save_discovery_state()` Method
**File:** `file:codeframe/agents/lead_agent.py` (lines 294-327)

Add validation to prevent saving inconsistent state:

- **Add validation check** at the beginning of `_save_discovery_state()`:
  - If `self._discovery_state == "discovering"` but `self._current_question_id` is None or empty, log a critical error and raise an exception
  - This prevents the stuck state from being persisted to the database
- **Add debug logging**: Log the state being saved (state, question_id, question_text) for troubleshooting

### 3. Improve Error Handling in `start_agent()` Function
**File:** `file:codeframe/ui/shared.py` (lines 309-348)

Enhance error handling when starting discovery:

- **Expand try-except block** around `agent.start_discovery()` (lines 310-312):
  - Currently catches generic Exception and broadcasts `discovery_error` but continues with agent startup
  - Add specific handling for the case where discovery fails to generate a question
  - If discovery fails, consider whether to remove the agent from `running_agents` dict or leave it for manual retry
- **Add timeout**: Wrap `agent.start_discovery()` in `asyncio.wait_for()` with a reasonable timeout (e.g., 30 seconds) to prevent indefinite hangs
- **Improve error broadcast**: Include more context in the `discovery_error` WebSocket message (e.g., whether it's a timeout, API failure, or question generation failure)

### 4. Add Recovery Mechanism in `get_discovery_status()` Method
**File:** `file:codeframe/agents/lead_agent.py` (lines 639-647)

Enhance the stuck state detection with automatic recovery:

- **Current behavior**: Lines 641-644 detect stuck state and set `needs_recovery` flag, but don't fix it
- **Add auto-recovery**: When stuck state is detected:
  - Log the detection with full context (project_id, state, question_id, question_text)
  - Automatically call `self.reset_discovery()` to transition back to "idle" state
  - Update the status dict to reflect the reset state
  - This prevents the stuck state from persisting across multiple API calls
- **Add recovery metadata**: Include in the status dict: `{"auto_recovered": True, "recovery_timestamp": <timestamp>}` so the frontend knows recovery occurred

### 5. Enhance Frontend Error Handling
**File:** `file:web-ui/src/components/DiscoveryProgress.tsx` (lines 346-379)

Improve the stuck state detection and user feedback:

- **Reduce timeout**: The current `STUCK_TIMEOUT_MS` is 10 seconds (line 52). Consider reducing to 5 seconds for faster detection
- **Auto-retry**: When stuck state is detected (line 370), instead of just showing the "Restart Discovery" button, automatically call `handleRestartDiscovery()` once. If it fails again, then show the manual button
- **Better error messages**: Update the stuck state UI (around line 700-750) to explain what happened: "Discovery failed to start. We've automatically reset it. Click 'Start Discovery' to try again."
- **Add telemetry**: Log stuck state occurrences to help track how often this happens in production

### 6. Add Integration Test for Race Condition
**File:** `file:tests/discovery/test_discovery_integration.py`

Add a test case to verify the fix:

- **Test name**: `test_start_discovery_handles_ai_failure_gracefully`
- **Setup**: Mock `AnthropicProvider.send_message()` to raise an exception
- **Action**: Call `agent.start_discovery()`
- **Assertions**:
  - Verify state remains "idle" (not "discovering")
  - Verify `_current_question_id` is None
  - Verify no stuck state is persisted to database
  - Verify calling `get_discovery_status()` returns idle state without warnings

### 7. Add Monitoring and Logging
**Files:** `file:codeframe/agents/lead_agent.py`, `file:codeframe/ui/shared.py`

Improve observability:

- **Add structured logging** in `start_discovery()`:
  - Log when entering the method with project_id and project_description length
  - Log which fallback path is taken (AI, framework, default)
  - Log the generated question ID and text length
  - Log when state transition completes successfully
- **Add metrics**: If the codebase has a metrics system, add counters for:
  - `discovery_start_success`
  - `discovery_start_failure`
  - `discovery_stuck_state_detected`
  - `discovery_auto_recovered`

## Sequence Diagram

```mermaid
sequenceDiagram
    participant UI as ProjectList UI
    participant API as POST /api/projects/{id}/start
    participant SA as start_agent()
    participant LA as LeadAgent
    participant DB as Database
    participant WS as WebSocket

    UI->>API: Start project after creation
    API->>SA: start_agent(project_id)
    SA->>LA: new LeadAgent(project_id)
    LA->>DB: Load discovery state (idle)
    SA->>LA: start_discovery(description)
    
    alt AI Question Generation Success
        LA->>LA: Generate AI question
        LA->>LA: Set state="discovering", question_id, question_text
        LA->>DB: Save state atomically
        LA-->>SA: Return question text
        SA->>WS: Broadcast discovery_question_ready
    else AI Fails, Framework Fallback Success
        LA->>LA: AI generation fails
        LA->>LA: Get framework question
        LA->>LA: Set state="discovering", question_id, question_text
        LA->>DB: Save state atomically
        LA-->>SA: Return question text
        SA->>WS: Broadcast discovery_question_ready
    else All Fallbacks Fail
        LA->>LA: All question generation fails
        LA->>LA: Keep state="idle" (NO state change)
        LA-->>SA: Raise exception
        SA->>WS: Broadcast discovery_error
        SA->>UI: Continue (agent running, discovery idle)
    end
    
    Note over LA,DB: CRITICAL: State only changes to "discovering"<br/>AFTER question is successfully generated
```

## Validation Checklist

- [ ] `start_discovery()` only transitions to "discovering" state after successfully generating a question
- [ ] All three fallback paths (AI, framework, default) set both `_current_question_id` and `_current_question_text` before saving state
- [ ] `_save_discovery_state()` validates that "discovering" state has a question before persisting
- [ ] `get_discovery_status()` auto-recovers from stuck state by calling `reset_discovery()`
- [ ] `start_agent()` has timeout protection and improved error handling
- [ ] Frontend reduces stuck detection timeout and implements auto-retry
- [ ] Integration test verifies graceful handling of AI failures
- [ ] Structured logging added for troubleshooting
- [ ] E2E test `test_project_creation.spec.ts` passes without stuck state warnings in logs

---

Execution Information

Branch: main
Commit: 84c9ded

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

⚠️ Possible Duplicate Issue

• The project discovery process doesn't kick off on new project creation #170

🔗 Related PRs

#83 - refactor: Extract FastAPI routers for improved maintainability (94% reduction in server.py) [merged]
#181 - fix(discovery): Allow starting discovery when project is running but discovery idle [merged]
#191 - feat(e2e): Enable conditional tests and fix discovery startup [merged]
#193 - Fix discovery questions repeating in staging [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[frankbria]

Closing as part of v2 roadmap cleanup. This v1-specific issue is retained with the v1-legacy label as reference material for Phase 3 (Web UI rebuild). See the v2 strategic roadmap for current priorities.