[Phase 1] cf work follow - Live execution streaming

[frankbria]

Summary

Implement cf work follow <task-id> command for real-time streaming of agent execution output.

Problem

Currently --verbose mode outputs progress but requires watching from the start. Users need ability to attach to running executions and see real-time progress.

Proposed Solution

Command Interface

cf work follow <task-id>           # Attach to running execution
cf work follow <task-id> --tail N  # Show last N lines then stream

Features

• Real-time stdout/stderr streaming from agent execution
• Attach to already-running executions
• Show buffered output when attaching mid-execution
• Graceful handling of execution completion

Technical Implementation

Options

1. File-based streaming: Agent writes to log file, follow command tails it
2. WebSocket/SSE: Real-time event stream (prepares for server layer)
3. Named pipe: UNIX-style inter-process communication

Recommendation: Start with file-based (simplest), design for upgrade to WebSocket.

Files to modify

• codeframe/core/runtime.py - Add log streaming support
• codeframe/core/agent.py - Write to streamable output
• codeframe/cli/app.py - Add work follow command

Acceptance Criteria

• cf work follow <task-id> streams live output
• Can attach to already-running executions
• Shows buffered output on attach
• Handles execution completion gracefully
• Output matches --verbose mode content

Related

• Part of Phase 1 CLI completion
• Prepares foundation for Phase 2 server real-time events

[traycerai]

Plan

Observations

The codebase already has a robust event streaming infrastructure (events.tail()) and structured logging system (RunLogger writing to run_logs table). A batch follow command exists that streams batch execution events in real-time. The --verbose flag in agent execution prints progress to stdout using _verbose_print() method. The system uses SQLite for persistence with events, run logs, and runs tables. The ticket requests implementing individual task execution streaming (not batch), which doesn't currently exist.

Approach

Implement a file-based streaming approach as recommended in the ticket. The agent will write verbose output to a dedicated log file per run (similar to debug logs), and the follow command will tail this file while also streaming events from the database. This provides both real-time stdout/stderr capture and structured event streaming. The implementation leverages existing infrastructure (events system, RunLogger) and follows the pattern established by batch follow, adapted for individual task execution.

Implementation Steps

1. Enhance Runtime Logging Infrastructure

File: file:codeframe/core/runtime.py

Modify execute_agent() function to create a streamable output log file for each run:

• After creating RunLogger (around line 600), create an additional file-based output logger
• Store log file path in a predictable location: .codeframe/runs/<run_id>/output.log
• Ensure directory exists before creating the file
• Pass the output log file handle to the Agent class

File: file:codeframe/core/agent.py

Enhance the Agent class to write verbose output to both stdout and the log file:

• Add output_log_file parameter to __init__() method (around line 400)
• Store the file handle as instance variable
• Modify _verbose_print() method (line 436) to write to both stdout and the log file when file handle is provided
• Ensure all verbose output (verification attempts, step execution, self-correction) is captured
• Add flush calls after writes to ensure real-time streaming
• Close the file handle when agent execution completes

2. Create Work Follow Command

File: file:codeframe/cli/app.py

Add new follow command to the work_app Typer application (after update-description command, around line 2183):

Command signature:

@work_app.command("follow")
def work_follow(
    task_id: str,
    workspace_path: Optional[Path] = None,
    tail: Optional[int] = None,
)

Implementation logic:

1. Resolve task ID - Find task by partial ID match (similar to other work commands)
2. Get active run - Use runtime.get_active_run(workspace, task_id) to find running execution
3. Validate run exists - If no active run, check for recent completed runs and show final output
4. Determine log file path - Construct path: .codeframe/runs/<run_id>/output.log
5. Show buffered output - If --tail N specified, read and display last N lines from log file
6. Stream new output - Use file tailing (polling-based) to stream new lines as they're written
7. Stream events - Simultaneously tail events table for run-related events using events.tail()
8. Handle completion - Detect when run completes (via events or run status change) and exit gracefully

3. Implement File Tailing Utility

File: file:codeframe/core/runtime.py or create new file:codeframe/core/streaming.py

Create a file tailing generator function:

def tail_run_output(
    workspace: Workspace,
    run_id: str,
    since_line: int = 0,
) -> Iterator[str]:

Implementation:

• Open log file in read mode
• Seek to appropriate position based on since_line
• Poll file for new lines (similar to events.tail() pattern)
• Yield new lines as they appear
• Handle file not existing yet (run just started)
• Use 0.5 second poll interval to match events polling

4. Update Run Status Tracking

File: file:codeframe/core/runtime.py

Ensure run status transitions emit appropriate events:

• Verify complete_run(), fail_run(), and block_run() emit events (already done)
• These events will signal the follow command to stop streaming

5. Add Helper for Buffered Output Display

File: file:codeframe/cli/app.py

Create helper function _show_buffered_output() within the follow command:

• Read last N lines from log file if --tail specified
• Display with appropriate formatting (dimmed for historical output)
• Return the line number where live streaming should start

6. Handle Edge Cases

In the follow command implementation:

• Run not started yet - Wait briefly for run to start, show message if timeout
• Run already completed - Show final output and exit immediately
• Log file doesn't exist - Fall back to event-only streaming with message
• Multiple runs - Only follow the most recent active run
• Keyboard interrupt - Handle Ctrl+C gracefully, show exit message
• File rotation/deletion - Handle file being removed during streaming

7. Integration with Existing Verbose Mode

File: file:codeframe/core/runtime.py

Modify execute_agent() to conditionally create output log:

• Always create output log file for runs (not just when verbose=True)
• This allows following even non-verbose runs
• When verbose=True, output goes to both stdout and log file
• When verbose=False, output only goes to log file

8. Documentation and Help Text

File: file:codeframe/cli/app.py

Add comprehensive docstring to work_follow command:

"""Follow real-time execution output of a running task.

Attaches to an active task execution and streams output as it happens.
Shows buffered output when attaching to already-running executions.

Example:
    cf work follow abc123              # Follow from current point
    cf work follow abc123 --tail 50    # Show last 50 lines then stream
"""

Visual Flow

sequenceDiagram
    participant User
    participant CLI as cf work follow
    participant Runtime
    participant Agent
    participant LogFile as output.log
    participant DB as SQLite

    User->>CLI: cf work follow task-123
    CLI->>Runtime: get_active_run(task-123)
    Runtime->>DB: Query runs table
    DB-->>Runtime: Run details
    Runtime-->>CLI: Run object
    
    CLI->>LogFile: Check if exists
    LogFile-->>CLI: File path
    
    alt --tail specified
        CLI->>LogFile: Read last N lines
        LogFile-->>CLI: Buffered output
        CLI->>User: Display buffered output
    end
    
    par Stream output
        loop Every 0.5s
            CLI->>LogFile: Read new lines
            LogFile-->>CLI: New output
            CLI->>User: Display output
        end
    and Stream events
        loop Every 0.5s
            CLI->>DB: Query events since last_id
            DB-->>CLI: New events
            CLI->>User: Display events
        end
    end
    
    Agent->>LogFile: Write verbose output
    Agent->>DB: Emit events
    
    Runtime->>DB: Update run status (COMPLETED)
    DB-->>CLI: RUN_COMPLETED event
    CLI->>User: Show completion message
    CLI->>User: Exit

Loading

File References

• file:codeframe/core/runtime.py - Modify execute_agent() to create output log file
• file:codeframe/core/agent.py - Enhance _verbose_print() to write to log file
• file:codeframe/cli/app.py - Add work_follow() command after line 2183
• file:codeframe/core/events.py - Reference existing tail() function for pattern
• file:codeframe/core/diagnostics.py - Reference RunLogger for structured logging

Key Design Decisions

Aspect	Decision	Rationale
Storage	File-based in .codeframe/runs/<run_id>/output.log	Simple, matches debug log pattern, easy to tail
Streaming	Polling-based (0.5s interval)	Consistent with existing events.tail() implementation
Buffering	Optional --tail N flag	Allows catching up on missed output when attaching mid-execution
Dual stream	Both file output and events	File for stdout/stderr, events for structured progress
Verbose mode	Always create log, conditionally print to stdout	Enables following non-verbose runs
Completion	Detect via RUN_COMPLETED/FAILED events	Clean exit when execution finishes

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase already has a robust event streaming infrastructure (`events.tail()`) and structured logging system (`RunLogger` writing to `run_logs` table). A `batch follow` command exists that streams batch execution events in real-time. The `--verbose` flag in agent execution prints progress to stdout using `_verbose_print()` method. The system uses SQLite for persistence with events, run logs, and runs tables. The ticket requests implementing individual task execution streaming (not batch), which doesn't currently exist.

## Approach

Implement a file-based streaming approach as recommended in the ticket. The agent will write verbose output to a dedicated log file per run (similar to debug logs), and the `follow` command will tail this file while also streaming events from the database. This provides both real-time stdout/stderr capture and structured event streaming. The implementation leverages existing infrastructure (events system, RunLogger) and follows the pattern established by `batch follow`, adapted for individual task execution.

## Implementation Steps

### 1. Enhance Runtime Logging Infrastructure

**File:** `file:codeframe/core/runtime.py`

Modify `execute_agent()` function to create a streamable output log file for each run:

- After creating `RunLogger` (around line 600), create an additional file-based output logger
- Store log file path in a predictable location: `.codeframe/runs/<run_id>/output.log`
- Ensure directory exists before creating the file
- Pass the output log file handle to the Agent class

**File:** `file:codeframe/core/agent.py`

Enhance the Agent class to write verbose output to both stdout and the log file:

- Add `output_log_file` parameter to `__init__()` method (around line 400)
- Store the file handle as instance variable
- Modify `_verbose_print()` method (line 436) to write to both stdout and the log file when file handle is provided
- Ensure all verbose output (verification attempts, step execution, self-correction) is captured
- Add flush calls after writes to ensure real-time streaming
- Close the file handle when agent execution completes

### 2. Create Work Follow Command

**File:** `file:codeframe/cli/app.py`

Add new `follow` command to the `work_app` Typer application (after `update-description` command, around line 2183):

**Command signature:**
```
@work_app.command("follow")
def work_follow(
    task_id: str,
    workspace_path: Optional[Path] = None,
    tail: Optional[int] = None,
)
```

**Implementation logic:**

1. **Resolve task ID** - Find task by partial ID match (similar to other work commands)
2. **Get active run** - Use `runtime.get_active_run(workspace, task_id)` to find running execution
3. **Validate run exists** - If no active run, check for recent completed runs and show final output
4. **Determine log file path** - Construct path: `.codeframe/runs/<run_id>/output.log`
5. **Show buffered output** - If `--tail N` specified, read and display last N lines from log file
6. **Stream new output** - Use file tailing (polling-based) to stream new lines as they're written
7. **Stream events** - Simultaneously tail events table for run-related events using `events.tail()`
8. **Handle completion** - Detect when run completes (via events or run status change) and exit gracefully

### 3. Implement File Tailing Utility

**File:** `file:codeframe/core/runtime.py` or create new `file:codeframe/core/streaming.py`

Create a file tailing generator function:

```python
def tail_run_output(
    workspace: Workspace,
    run_id: str,
    since_line: int = 0,
) -> Iterator[str]:
```

**Implementation:**
- Open log file in read mode
- Seek to appropriate position based on `since_line`
- Poll file for new lines (similar to `events.tail()` pattern)
- Yield new lines as they appear
- Handle file not existing yet (run just started)
- Use 0.5 second poll interval to match events polling

### 4. Update Run Status Tracking

**File:** `file:codeframe/core/runtime.py`

Ensure run status transitions emit appropriate events:

- Verify `complete_run()`, `fail_run()`, and `block_run()` emit events (already done)
- These events will signal the follow command to stop streaming

### 5. Add Helper for Buffered Output Display

**File:** `file:codeframe/cli/app.py`

Create helper function `_show_buffered_output()` within the follow command:

- Read last N lines from log file if `--tail` specified
- Display with appropriate formatting (dimmed for historical output)
- Return the line number where live streaming should start

### 6. Handle Edge Cases

**In the follow command implementation:**

- **Run not started yet** - Wait briefly for run to start, show message if timeout
- **Run already completed** - Show final output and exit immediately
- **Log file doesn't exist** - Fall back to event-only streaming with message
- **Multiple runs** - Only follow the most recent active run
- **Keyboard interrupt** - Handle Ctrl+C gracefully, show exit message
- **File rotation/deletion** - Handle file being removed during streaming

### 7. Integration with Existing Verbose Mode

**File:** `file:codeframe/core/runtime.py`

Modify `execute_agent()` to conditionally create output log:

- Always create output log file for runs (not just when verbose=True)
- This allows following even non-verbose runs
- When verbose=True, output goes to both stdout and log file
- When verbose=False, output only goes to log file

### 8. Documentation and Help Text

**File:** `file:codeframe/cli/app.py`

Add comprehensive docstring to `work_follow` command:

```python
"""Follow real-time execution output of a running task.

Attaches to an active task execution and streams output as it happens.
Shows buffered output when attaching to already-running executions.

Example:
    cf work follow abc123              # Follow from current point
    cf work follow abc123 --tail 50    # Show last 50 lines then stream
"""
```

## Visual Flow

```mermaid
sequenceDiagram
    participant User
    participant CLI as cf work follow
    participant Runtime
    participant Agent
    participant LogFile as output.log
    participant DB as SQLite

    User->>CLI: cf work follow task-123
    CLI->>Runtime: get_active_run(task-123)
    Runtime->>DB: Query runs table
    DB-->>Runtime: Run details
    Runtime-->>CLI: Run object
    
    CLI->>LogFile: Check if exists
    LogFile-->>CLI: File path
    
    alt --tail specified
        CLI->>LogFile: Read last N lines
        LogFile-->>CLI: Buffered output
        CLI->>User: Display buffered output
    end
    
    par Stream output
        loop Every 0.5s
            CLI->>LogFile: Read new lines
            LogFile-->>CLI: New output
            CLI->>User: Display output
        end
    and Stream events
        loop Every 0.5s
            CLI->>DB: Query events since last_id
            DB-->>CLI: New events
            CLI->>User: Display events
        end
    end
    
    Agent->>LogFile: Write verbose output
    Agent->>DB: Emit events
    
    Runtime->>DB: Update run status (COMPLETED)
    DB-->>CLI: RUN_COMPLETED event
    CLI->>User: Show completion message
    CLI->>User: Exit
```

## File References

- `file:codeframe/core/runtime.py` - Modify `execute_agent()` to create output log file
- `file:codeframe/core/agent.py` - Enhance `_verbose_print()` to write to log file
- `file:codeframe/cli/app.py` - Add `work_follow()` command after line 2183
- `file:codeframe/core/events.py` - Reference existing `tail()` function for pattern
- `file:codeframe/core/diagnostics.py` - Reference `RunLogger` for structured logging

## Key Design Decisions

| Aspect | Decision | Rationale |
|--------|----------|-----------|
| Storage | File-based in `.codeframe/runs/<run_id>/output.log` | Simple, matches debug log pattern, easy to tail |
| Streaming | Polling-based (0.5s interval) | Consistent with existing `events.tail()` implementation |
| Buffering | Optional `--tail N` flag | Allows catching up on missed output when attaching mid-execution |
| Dual stream | Both file output and events | File for stdout/stderr, events for structured progress |
| Verbose mode | Always create log, conditionally print to stdout | Enables following non-verbose runs |
| Completion | Detect via RUN_COMPLETED/FAILED events | Clean exit when execution finishes |

---

Execution Information

Branch: main
Commit: e768c1d

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.