[P2] Integrate QualityTracker into WorkerAgent workflow

[frankbria]

Overview

QualityTracker is fully implemented but NOT integrated into the WorkerAgent workflow. This prevents quality degradation detection.

Current State

• Implemented: codeframe/enforcement/quality_tracker.py (fully functional)
• Not integrated: WorkerAgent doesn't record quality metrics
• Issue: No quality ratcheting or degradation alerts

Expected Behavior

Record quality metrics:

• After each task completion (test pass rate, coverage)
• After each test run (success/failure)
• On checkpoint creation
• On context reset

Check for degradation:

• Before marking task complete
• Periodically during long-running sessions
• When context approaches limit

Implementation Requirements

1. Import QualityTracker in worker_agent.py
2. Record metrics after each test run
3. Check degradation before task completion
4. Create blocker if quality degraded >10%
5. Recommend context reset when appropriate

Integration Points

# In WorkerAgent.__init__()
self.quality_tracker = QualityTracker(project_path=self.project_path)

# After test run
metrics = QualityMetrics(
    timestamp=datetime.now().isoformat(),
    response_count=self.response_count,
    test_pass_rate=test_result.pass_rate,
    coverage_percentage=test_result.coverage,
    # ... other fields ...
)
self.quality_tracker.record(metrics)

# Before task completion
degradation = self.quality_tracker.check_degradation()
if degradation["has_degradation"]:
    return self._create_blocker(
        task,
        f"Quality degradation detected: {degradation['issues']}",
        {"recommendation": degradation["recommendation"]}
    )

Acceptance Criteria

• Quality metrics recorded after test runs
• Degradation check before task completion
• Blocker created on degradation >10%
• Context reset recommended when appropriate
• Dashboard displays quality trends
• Integration tests verify tracking

Priority

P2 - Essential: Prevents quality regressions during long-running sessions.

References

• Tracker implementation: codeframe/enforcement/quality_tracker.py
• README: codeframe/enforcement/README.md lines 25
• Issues Analysis: CODEFRAME_ISSUES_ANALYSIS.md lines 53-64

[traycerai]

Plan

Observations

The QualityTracker class in file:codeframe/enforcement/quality_tracker.py is fully implemented with all required methods (record(), check_degradation(), should_reset_context()). However, it's completely isolated from the WorkerAgent workflow. The WorkerAgent in file:codeframe/agents/worker_agent.py has a comprehensive complete_task() method that runs quality gates, but doesn't record quality metrics or check for degradation. The quality gates in file:codeframe/lib/quality_gates.py already extract test results and coverage data, providing the necessary inputs for quality tracking.

Approach

The integration follows a layered approach: (1) Initialize QualityTracker in WorkerAgent.__init__() alongside existing components, (2) Add response count tracking as an instance variable, (3) Record metrics after quality gate execution in complete_task(), (4) Check degradation before allowing task completion, (5) Integrate with checkpoint creation and context reset mechanisms. This approach leverages existing quality gate infrastructure and maintains consistency with the current architecture patterns.

Implementation Steps

1. Add QualityTracker Initialization to WorkerAgent

File: file:codeframe/agents/worker_agent.py

In the WorkerAgent.__init__() method (around line 43-77):

• Add instance variable self.response_count = 0 to track AI conversation length
• Import QualityTracker and QualityMetrics from codeframe.enforcement.quality_tracker
• Initialize self.quality_tracker = None (will be set when project context is established)
• Add a helper method _ensure_quality_tracker() that lazily initializes the tracker when project_path is available from the current task

Initialization pattern:
- Store response_count as instance variable
- Lazy-initialize quality_tracker when project context is available
- Use self._get_project_id() to get project context
- Get project workspace_path from database to initialize tracker

2. Record Quality Metrics After Test Execution

File: file:codeframe/agents/worker_agent.py

In the complete_task() method (around line 812-930), after quality gates run successfully:

• After line 883 where quality_result = await quality_gates.run_all_gates(task) completes
• Before the task status update (line 888-897)
• Extract metrics from quality gate results:
  • Parse test pass rate from quality_result
  • Extract coverage percentage from quality gate coverage results
  • Calculate total_tests, passed_tests, failed_tests from test gate results
• Create QualityMetrics object with:
  • timestamp: Current ISO timestamp
  • response_count: self.response_count
  • test_pass_rate: Calculated from test results (passed/total * 100)
  • coverage_percentage: From coverage gate
  • total_tests, passed_tests, failed_tests: From test gate
  • language: Detect using LanguageDetector from codeframe.enforcement.language_detector
  • framework: From language detection result
• Call self.quality_tracker.record(metrics) to save the checkpoint

Metric extraction logic:
- Access quality_result.failures to find test/coverage gate results
- For test gate: count passed vs failed from gate output
- For coverage gate: extract percentage from gate details
- Use LanguageDetector(project_root).detect() for language/framework

3. Check for Quality Degradation Before Task Completion

File: file:codeframe/agents/worker_agent.py

In the complete_task() method, after recording metrics but before marking task as completed:

• After recording quality metrics (new code from step 2)
• Before line 888 where task status is updated to COMPLETED
• Call degradation = self.quality_tracker.check_degradation(threshold_percent=10.0)
• If degradation["has_degradation"] is True:
  • Create a blocker using existing _create_quality_blocker() method pattern
  • Format blocker message with degradation details from degradation["issues"]
  • Include recommendation from degradation["recommendation"]
  • Return failure response similar to lines 922-929
  • Keep task status as in_progress (don't mark completed)

Degradation check flow:
1. Record metrics first (establishes baseline)
2. Check degradation immediately after
3. If degraded > 10%:
   - Create SYNC blocker with degradation details
   - Return {"success": False, "status": "blocked", "message": "..."}
4. If not degraded:
   - Continue to mark task as completed

4. Increment Response Count on Task Execution

File: file:codeframe/agents/worker_agent.py

In the execute_task() method (around line 219-503):

• After successful LLM call (around line 432-438 where result is returned)
• Increment self.response_count += 1 before returning the result
• This tracks the conversation length for quality degradation detection

5. Integrate with Checkpoint Creation

File: file:codeframe/lib/checkpoint_manager.py

In the create_checkpoint() method (around line 54-143):

• After line 94 where metadata is generated
• Before saving checkpoint to database (line 97)
• Check if quality tracker exists for the project
• If available, get current quality stats using quality_tracker.get_stats()
• Add quality metrics to checkpoint metadata:
  • Include current quality metrics in metadata dict
  • Add peak quality metrics for reference
  • Store quality trend (improving/stable/declining)

Integration approach:
- Import QualityTracker in checkpoint_manager.py
- Initialize tracker with project_root
- Call get_stats() to retrieve current quality state
- Merge quality stats into existing metadata structure

6. Add Context Reset Recommendation

File: file:codeframe/agents/worker_agent.py

Add a new method should_recommend_context_reset() after the should_flash_save() method (around line 644-673):

• Check if quality tracker is initialized
• Call self.quality_tracker.should_reset_context(response_count=self.response_count, max_responses=20, check_degradation=True)
• Return the recommendation dict with should_reset, reasons, and recommendation fields
• This method can be called periodically or before task execution

Usage pattern:
- Call before starting new tasks
- Check during long-running sessions
- Integrate with existing flash_save checks
- Log recommendations for monitoring

7. Dashboard Integration for Quality Trends

File: Create new API endpoint in file:codeframe/ui/routers/ (or extend existing metrics router)

Add endpoint to expose quality history:

• Route: GET /api/projects/{project_id}/quality-history
• Initialize QualityTracker with project workspace path
• Call tracker.get_stats() to get current/peak/average metrics
• Call tracker.load_history() to get full history
• Return JSON with quality trends for dashboard visualization

File: file:web-ui/src/components/Dashboard.tsx

Add quality metrics visualization:

• Fetch quality history from new API endpoint
• Display current vs peak quality metrics
• Show quality trend chart (test pass rate and coverage over time)
• Highlight degradation warnings if present
• Show response count and context reset recommendations

8. Add Integration Tests

File: Create file:tests/integration/test_quality_tracker_integration.py

Test the complete workflow:

• Test 1: Quality metrics recorded after task completion
• Test 2: Degradation detected and blocker created
• Test 3: Context reset recommended when quality degrades
• Test 4: Checkpoint includes quality metrics
• Test 5: Dashboard displays quality trends

Test structure:
1. Create WorkerAgent with database
2. Execute task with quality gates
3. Verify metrics recorded in .codeframe/quality_history.json
4. Simulate degradation by recording lower metrics
5. Verify blocker created and task blocked
6. Verify context reset recommendation triggered

Integration Flow Diagram

sequenceDiagram
    participant WA as WorkerAgent
    participant QG as QualityGates
    participant QT as QualityTracker
    participant DB as Database
    participant CM as CheckpointManager

    Note over WA: Task Execution Flow
    WA->>WA: execute_task(task)
    WA->>WA: response_count += 1
    
    Note over WA: Task Completion Flow
    WA->>QG: complete_task(task)
    QG->>QG: run_all_gates(task)
    QG-->>WA: quality_result
    
    alt Quality Gates Pass
        WA->>QT: record(QualityMetrics)
        QT->>QT: save to quality_history.json
        WA->>QT: check_degradation()
        QT-->>WA: degradation result
        
        alt No Degradation
            WA->>DB: mark task completed
            WA-->>WA: success
        else Degradation > 10%
            WA->>DB: create_blocker()
            WA-->>WA: blocked (task stays in_progress)
        end
    else Quality Gates Fail
        QG->>DB: create_blocker()
        WA-->>WA: blocked
    end
    
    Note over CM: Checkpoint Creation
    CM->>QT: get_stats()
    QT-->>CM: quality metrics
    CM->>DB: save checkpoint with quality data
    
    Note over WA: Context Reset Check
    WA->>QT: should_reset_context(response_count)
    QT-->>WA: recommendation

Loading

Key Files Modified

File	Changes
file:codeframe/agents/worker_agent.py	Add QualityTracker initialization, response_count tracking, metric recording, degradation checks
file:codeframe/lib/checkpoint_manager.py	Integrate quality metrics into checkpoint metadata
file:codeframe/ui/routers/	Add quality history API endpoint
file:web-ui/src/components/Dashboard.tsx	Add quality trends visualization
file:tests/integration/test_quality_tracker_integration.py	Add comprehensive integration tests

Data Flow

Trigger	Source	Metrics Captured	Action
After test run	QualityGates.run_all_gates()	test_pass_rate, coverage_percentage, total_tests	Record to quality_history.json
Before task completion	WorkerAgent.complete_task()	All metrics + response_count	Check degradation, create blocker if >10%
Checkpoint creation	CheckpointManager.create_checkpoint()	Current quality stats	Include in checkpoint metadata
Context reset check	WorkerAgent.should_recommend_context_reset()	response_count + quality trend	Return recommendation

Configuration

Quality tracking behavior can be configured via environment variables:

• QUALITY_DEGRADATION_THRESHOLD: Percentage threshold for degradation detection (default: 10.0)
• MAX_RESPONSE_COUNT: Maximum responses before context reset recommendation (default: 20)
• QUALITY_TRACKING_ENABLED: Enable/disable quality tracking (default: true)

Import In IDE

🤖 Prompt for AI Agents

## Observations

The `QualityTracker` class in `file:codeframe/enforcement/quality_tracker.py` is fully implemented with all required methods (`record()`, `check_degradation()`, `should_reset_context()`). However, it's completely isolated from the `WorkerAgent` workflow. The `WorkerAgent` in `file:codeframe/agents/worker_agent.py` has a comprehensive `complete_task()` method that runs quality gates, but doesn't record quality metrics or check for degradation. The quality gates in `file:codeframe/lib/quality_gates.py` already extract test results and coverage data, providing the necessary inputs for quality tracking.

## Approach

The integration follows a layered approach: (1) Initialize `QualityTracker` in `WorkerAgent.__init__()` alongside existing components, (2) Add response count tracking as an instance variable, (3) Record metrics after quality gate execution in `complete_task()`, (4) Check degradation before allowing task completion, (5) Integrate with checkpoint creation and context reset mechanisms. This approach leverages existing quality gate infrastructure and maintains consistency with the current architecture patterns.

## Implementation Steps

### 1. Add QualityTracker Initialization to WorkerAgent

**File:** `file:codeframe/agents/worker_agent.py`

In the `WorkerAgent.__init__()` method (around line 43-77):

- Add instance variable `self.response_count = 0` to track AI conversation length
- Import `QualityTracker` and `QualityMetrics` from `codeframe.enforcement.quality_tracker`
- Initialize `self.quality_tracker = None` (will be set when project context is established)
- Add a helper method `_ensure_quality_tracker()` that lazily initializes the tracker when `project_path` is available from the current task

```
Initialization pattern:
- Store response_count as instance variable
- Lazy-initialize quality_tracker when project context is available
- Use self._get_project_id() to get project context
- Get project workspace_path from database to initialize tracker
```

### 2. Record Quality Metrics After Test Execution

**File:** `file:codeframe/agents/worker_agent.py`

In the `complete_task()` method (around line 812-930), after quality gates run successfully:

- After line 883 where `quality_result = await quality_gates.run_all_gates(task)` completes
- Before the task status update (line 888-897)
- Extract metrics from quality gate results:
  - Parse test pass rate from `quality_result` 
  - Extract coverage percentage from quality gate coverage results
  - Calculate total_tests, passed_tests, failed_tests from test gate results
- Create `QualityMetrics` object with:
  - `timestamp`: Current ISO timestamp
  - `response_count`: `self.response_count`
  - `test_pass_rate`: Calculated from test results (passed/total * 100)
  - `coverage_percentage`: From coverage gate
  - `total_tests`, `passed_tests`, `failed_tests`: From test gate
  - `language`: Detect using `LanguageDetector` from `codeframe.enforcement.language_detector`
  - `framework`: From language detection result
- Call `self.quality_tracker.record(metrics)` to save the checkpoint

```
Metric extraction logic:
- Access quality_result.failures to find test/coverage gate results
- For test gate: count passed vs failed from gate output
- For coverage gate: extract percentage from gate details
- Use LanguageDetector(project_root).detect() for language/framework
```

### 3. Check for Quality Degradation Before Task Completion

**File:** `file:codeframe/agents/worker_agent.py`

In the `complete_task()` method, after recording metrics but before marking task as completed:

- After recording quality metrics (new code from step 2)
- Before line 888 where task status is updated to COMPLETED
- Call `degradation = self.quality_tracker.check_degradation(threshold_percent=10.0)`
- If `degradation["has_degradation"]` is True:
  - Create a blocker using existing `_create_quality_blocker()` method pattern
  - Format blocker message with degradation details from `degradation["issues"]`
  - Include recommendation from `degradation["recommendation"]`
  - Return failure response similar to lines 922-929
  - Keep task status as `in_progress` (don't mark completed)

```
Degradation check flow:
1. Record metrics first (establishes baseline)
2. Check degradation immediately after
3. If degraded > 10%:
   - Create SYNC blocker with degradation details
   - Return {"success": False, "status": "blocked", "message": "..."}
4. If not degraded:
   - Continue to mark task as completed
```

### 4. Increment Response Count on Task Execution

**File:** `file:codeframe/agents/worker_agent.py`

In the `execute_task()` method (around line 219-503):

- After successful LLM call (around line 432-438 where result is returned)
- Increment `self.response_count += 1` before returning the result
- This tracks the conversation length for quality degradation detection

### 5. Integrate with Checkpoint Creation

**File:** `file:codeframe/lib/checkpoint_manager.py`

In the `create_checkpoint()` method (around line 54-143):

- After line 94 where metadata is generated
- Before saving checkpoint to database (line 97)
- Check if quality tracker exists for the project
- If available, get current quality stats using `quality_tracker.get_stats()`
- Add quality metrics to checkpoint metadata:
  - Include current quality metrics in metadata dict
  - Add peak quality metrics for reference
  - Store quality trend (improving/stable/declining)

```
Integration approach:
- Import QualityTracker in checkpoint_manager.py
- Initialize tracker with project_root
- Call get_stats() to retrieve current quality state
- Merge quality stats into existing metadata structure
```

### 6. Add Context Reset Recommendation

**File:** `file:codeframe/agents/worker_agent.py`

Add a new method `should_recommend_context_reset()` after the `should_flash_save()` method (around line 644-673):

- Check if quality tracker is initialized
- Call `self.quality_tracker.should_reset_context(response_count=self.response_count, max_responses=20, check_degradation=True)`
- Return the recommendation dict with `should_reset`, `reasons`, and `recommendation` fields
- This method can be called periodically or before task execution

```
Usage pattern:
- Call before starting new tasks
- Check during long-running sessions
- Integrate with existing flash_save checks
- Log recommendations for monitoring
```

### 7. Dashboard Integration for Quality Trends

**File:** Create new API endpoint in `file:codeframe/ui/routers/` (or extend existing metrics router)

Add endpoint to expose quality history:

- Route: `GET /api/projects/{project_id}/quality-history`
- Initialize `QualityTracker` with project workspace path
- Call `tracker.get_stats()` to get current/peak/average metrics
- Call `tracker.load_history()` to get full history
- Return JSON with quality trends for dashboard visualization

**File:** `file:web-ui/src/components/Dashboard.tsx`

Add quality metrics visualization:

- Fetch quality history from new API endpoint
- Display current vs peak quality metrics
- Show quality trend chart (test pass rate and coverage over time)
- Highlight degradation warnings if present
- Show response count and context reset recommendations

### 8. Add Integration Tests

**File:** Create `file:tests/integration/test_quality_tracker_integration.py`

Test the complete workflow:

- Test 1: Quality metrics recorded after task completion
- Test 2: Degradation detected and blocker created
- Test 3: Context reset recommended when quality degrades
- Test 4: Checkpoint includes quality metrics
- Test 5: Dashboard displays quality trends

```
Test structure:
1. Create WorkerAgent with database
2. Execute task with quality gates
3. Verify metrics recorded in .codeframe/quality_history.json
4. Simulate degradation by recording lower metrics
5. Verify blocker created and task blocked
6. Verify context reset recommendation triggered
```

## Integration Flow Diagram

```mermaid
sequenceDiagram
    participant WA as WorkerAgent
    participant QG as QualityGates
    participant QT as QualityTracker
    participant DB as Database
    participant CM as CheckpointManager

    Note over WA: Task Execution Flow
    WA->>WA: execute_task(task)
    WA->>WA: response_count += 1
    
    Note over WA: Task Completion Flow
    WA->>QG: complete_task(task)
    QG->>QG: run_all_gates(task)
    QG-->>WA: quality_result
    
    alt Quality Gates Pass
        WA->>QT: record(QualityMetrics)
        QT->>QT: save to quality_history.json
        WA->>QT: check_degradation()
        QT-->>WA: degradation result
        
        alt No Degradation
            WA->>DB: mark task completed
            WA-->>WA: success
        else Degradation > 10%
            WA->>DB: create_blocker()
            WA-->>WA: blocked (task stays in_progress)
        end
    else Quality Gates Fail
        QG->>DB: create_blocker()
        WA-->>WA: blocked
    end
    
    Note over CM: Checkpoint Creation
    CM->>QT: get_stats()
    QT-->>CM: quality metrics
    CM->>DB: save checkpoint with quality data
    
    Note over WA: Context Reset Check
    WA->>QT: should_reset_context(response_count)
    QT-->>WA: recommendation
```

## Key Files Modified

| File | Changes |
|------|---------|
| `file:codeframe/agents/worker_agent.py` | Add QualityTracker initialization, response_count tracking, metric recording, degradation checks |
| `file:codeframe/lib/checkpoint_manager.py` | Integrate quality metrics into checkpoint metadata |
| `file:codeframe/ui/routers/` | Add quality history API endpoint |
| `file:web-ui/src/components/Dashboard.tsx` | Add quality trends visualization |
| `file:tests/integration/test_quality_tracker_integration.py` | Add comprehensive integration tests |

## Data Flow

| Trigger | Source | Metrics Captured | Action |
|---------|--------|------------------|--------|
| After test run | `QualityGates.run_all_gates()` | test_pass_rate, coverage_percentage, total_tests | Record to quality_history.json |
| Before task completion | `WorkerAgent.complete_task()` | All metrics + response_count | Check degradation, create blocker if >10% |
| Checkpoint creation | `CheckpointManager.create_checkpoint()` | Current quality stats | Include in checkpoint metadata |
| Context reset check | `WorkerAgent.should_recommend_context_reset()` | response_count + quality trend | Return recommendation |

## Configuration

Quality tracking behavior can be configured via environment variables:

- `QUALITY_DEGRADATION_THRESHOLD`: Percentage threshold for degradation detection (default: 10.0)
- `MAX_RESPONSE_COUNT`: Maximum responses before context reset recommendation (default: 20)
- `QUALITY_TRACKING_ENABLED`: Enable/disable quality tracking (default: true)

---

Execution Information

Branch: main
Commit: 3fa0e9c

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

_{You can configure auto-planning by selecting labels in the issue_enrichment configuration.}

🔗 Similar Issues

Possible Duplicates

• Add Quality Ratchet System #14

Related Issues

• [P1] Implement WorkerAgent.execute_task() with Anthropic API integration #98
• Implement Quality Gates Panel in Dashboard #43
• [P1] Add token tracking after LLM calls in WorkerAgent #99
• [P1] Implement LeadAgent.detect_bottlenecks() logic #101

🔗 Related PRs

#50 - feat: Implement Quality Gates Panel in Dashboard (#43) [merged]

👤 Suggested Assignees

• frankbria

---

🧪 Issue enrichment is currently in open beta.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord or schedule a call!

[traycerai]

I have updated the Implementation Plan to reflect the recent changes in the ticket.

New Plan

Observations

The QualityTracker class in file:codeframe/enforcement/quality_tracker.py is fully implemented with all required methods (record(), check_degradation(), should_reset_context()). However, it's completely isolated from the WorkerAgent workflow. The WorkerAgent in file:codeframe/agents/worker_agent.py has a comprehensive complete_task() method that runs quality gates, but doesn't record quality metrics or check for degradation. The quality gates in file:codeframe/lib/quality_gates.py already extract test results and coverage data, providing the necessary inputs for quality tracking.

Approach

The integration follows a layered approach: (1) Initialize QualityTracker in WorkerAgent.__init__() alongside existing components, (2) Add response count tracking as an instance variable, (3) Record metrics after quality gate execution in complete_task(), (4) Check degradation before allowing task completion, (5) Integrate with checkpoint creation and context reset mechanisms. This approach leverages existing quality gate infrastructure and maintains consistency with the current architecture patterns.

Implementation Steps

1. Add QualityTracker Initialization to WorkerAgent

File: file:codeframe/agents/worker_agent.py

In the WorkerAgent.__init__() method (around line 43-77):

• Add instance variable self.response_count = 0 to track AI conversation length
• Import QualityTracker and QualityMetrics from codeframe.enforcement.quality_tracker
• Initialize self.quality_tracker = None (will be set when project context is established)
• Add a helper method _ensure_quality_tracker() that lazily initializes the tracker when project_path is available from the current task

Initialization pattern:
- Store response_count as instance variable
- Lazy-initialize quality_tracker when project context is available
- Use self._get_project_id() to get project context
- Get project workspace_path from database to initialize tracker

2. Record Quality Metrics After Test Execution

File: file:codeframe/agents/worker_agent.py

In the complete_task() method (around line 812-930), after quality gates run successfully:

• After line 883 where quality_result = await quality_gates.run_all_gates(task) completes
• Before the task status update (line 888-897)
• Extract metrics from quality gate results:
  • Parse test pass rate from quality_result
  • Extract coverage percentage from quality gate coverage results
  • Calculate total_tests, passed_tests, failed_tests from test gate results
• Create QualityMetrics object with:
  • timestamp: Current ISO timestamp
  • response_count: self.response_count
  • test_pass_rate: Calculated from test results (passed/total * 100)
  • coverage_percentage: From coverage gate
  • total_tests, passed_tests, failed_tests: From test gate
  • language: Detect using LanguageDetector from codeframe.enforcement.language_detector
  • framework: From language detection result
• Call self.quality_tracker.record(metrics) to save the checkpoint

Metric extraction logic:
- Access quality_result.failures to find test/coverage gate results
- For test gate: count passed vs failed from gate output
- For coverage gate: extract percentage from gate details
- Use LanguageDetector(project_root).detect() for language/framework

3. Check for Quality Degradation Before Task Completion

File: file:codeframe/agents/worker_agent.py

In the complete_task() method, after recording metrics but before marking task as completed:

• After recording quality metrics (new code from step 2)
• Before line 888 where task status is updated to COMPLETED
• Call degradation = self.quality_tracker.check_degradation(threshold_percent=10.0)
• If degradation["has_degradation"] is True:
  • Create a blocker using existing _create_quality_blocker() method pattern
  • Format blocker message with degradation details from degradation["issues"]
  • Include recommendation from degradation["recommendation"]
  • Return failure response similar to lines 922-929
  • Keep task status as in_progress (don't mark completed)

Degradation check flow:
1. Record metrics first (establishes baseline)
2. Check degradation immediately after
3. If degraded > 10%:
   - Create SYNC blocker with degradation details
   - Return {"success": False, "status": "blocked", "message": "..."}
4. If not degraded:
   - Continue to mark task as completed

4. Increment Response Count on Task Execution

File: file:codeframe/agents/worker_agent.py

In the execute_task() method (around line 219-503):

• After successful LLM call (around line 432-438 where result is returned)
• Increment self.response_count += 1 before returning the result
• This tracks the conversation length for quality degradation detection

5. Integrate with Checkpoint Creation

File: file:codeframe/lib/checkpoint_manager.py

In the create_checkpoint() method (around line 54-143):

• After line 94 where metadata is generated
• Before saving checkpoint to database (line 97)
• Check if quality tracker exists for the project
• If available, get current quality stats using quality_tracker.get_stats()
• Add quality metrics to checkpoint metadata:
  • Include current quality metrics in metadata dict
  • Add peak quality metrics for reference
  • Store quality trend (improving/stable/declining)

Integration approach:
- Import QualityTracker in checkpoint_manager.py
- Initialize tracker with project_root
- Call get_stats() to retrieve current quality state
- Merge quality stats into existing metadata structure

6. Add Context Reset Recommendation

File: file:codeframe/agents/worker_agent.py

Add a new method should_recommend_context_reset() after the should_flash_save() method (around line 644-673):

• Check if quality tracker is initialized
• Call self.quality_tracker.should_reset_context(response_count=self.response_count, max_responses=20, check_degradation=True)
• Return the recommendation dict with should_reset, reasons, and recommendation fields
• This method can be called periodically or before task execution

Usage pattern:
- Call before starting new tasks
- Check during long-running sessions
- Integrate with existing flash_save checks
- Log recommendations for monitoring

7. Dashboard Integration for Quality Trends

File: Create new API endpoint in file:codeframe/ui/routers/ (or extend existing metrics router)

Add endpoint to expose quality history:

• Route: GET /api/projects/{project_id}/quality-history
• Initialize QualityTracker with project workspace path
• Call tracker.get_stats() to get current/peak/average metrics
• Call tracker.load_history() to get full history
• Return JSON with quality trends for dashboard visualization

File: file:web-ui/src/components/Dashboard.tsx

Add quality metrics visualization:

• Fetch quality history from new API endpoint
• Display current vs peak quality metrics
• Show quality trend chart (test pass rate and coverage over time)
• Highlight degradation warnings if present
• Show response count and context reset recommendations

8. Add Integration Tests

File: Create file:tests/integration/test_quality_tracker_integration.py

Test the complete workflow:

• Test 1: Quality metrics recorded after task completion
• Test 2: Degradation detected and blocker created
• Test 3: Context reset recommended when quality degrades
• Test 4: Checkpoint includes quality metrics
• Test 5: Dashboard displays quality trends

Test structure:
1. Create WorkerAgent with database
2. Execute task with quality gates
3. Verify metrics recorded in .codeframe/quality_history.json
4. Simulate degradation by recording lower metrics
5. Verify blocker created and task blocked
6. Verify context reset recommendation triggered

Integration Flow Diagram

sequenceDiagram
    participant WA as WorkerAgent
    participant QG as QualityGates
    participant QT as QualityTracker
    participant DB as Database
    participant CM as CheckpointManager

    Note over WA: Task Execution Flow
    WA->>WA: execute_task(task)
    WA->>WA: response_count += 1
    
    Note over WA: Task Completion Flow
    WA->>QG: complete_task(task)
    QG->>QG: run_all_gates(task)
    QG-->>WA: quality_result
    
    alt Quality Gates Pass
        WA->>QT: record(QualityMetrics)
        QT->>QT: save to quality_history.json
        WA->>QT: check_degradation()
        QT-->>WA: degradation result
        
        alt No Degradation
            WA->>DB: mark task completed
            WA-->>WA: success
        else Degradation > 10%
            WA->>DB: create_blocker()
            WA-->>WA: blocked (task stays in_progress)
        end
    else Quality Gates Fail
        QG->>DB: create_blocker()
        WA-->>WA: blocked
    end
    
    Note over CM: Checkpoint Creation
    CM->>QT: get_stats()
    QT-->>CM: quality metrics
    CM->>DB: save checkpoint with quality data
    
    Note over WA: Context Reset Check
    WA->>QT: should_reset_context(response_count)
    QT-->>WA: recommendation

Loading

Key Files Modified

File	Changes
file:codeframe/agents/worker_agent.py	Add QualityTracker initialization, response_count tracking, metric recording, degradation checks
file:codeframe/lib/checkpoint_manager.py	Integrate quality metrics into checkpoint metadata
file:codeframe/ui/routers/	Add quality history API endpoint
file:web-ui/src/components/Dashboard.tsx	Add quality trends visualization
file:tests/integration/test_quality_tracker_integration.py	Add comprehensive integration tests

Data Flow

Trigger	Source	Metrics Captured	Action
After test run	QualityGates.run_all_gates()	test_pass_rate, coverage_percentage, total_tests	Record to quality_history.json
Before task completion	WorkerAgent.complete_task()	All metrics + response_count	Check degradation, create blocker if >10%
Checkpoint creation	CheckpointManager.create_checkpoint()	Current quality stats	Include in checkpoint metadata
Context reset check	WorkerAgent.should_recommend_context_reset()	response_count + quality trend	Return recommendation

Configuration

Quality tracking behavior can be configured via environment variables:

• QUALITY_DEGRADATION_THRESHOLD: Percentage threshold for degradation detection (default: 10.0)
• MAX_RESPONSE_COUNT: Maximum responses before context reset recommendation (default: 20)
• QUALITY_TRACKING_ENABLED: Enable/disable quality tracking (default: true)

Import In IDE

🤖 Prompt for AI Agents

## Observations

The `QualityTracker` class in `file:codeframe/enforcement/quality_tracker.py` is fully implemented with all required methods (`record()`, `check_degradation()`, `should_reset_context()`). However, it's completely isolated from the `WorkerAgent` workflow. The `WorkerAgent` in `file:codeframe/agents/worker_agent.py` has a comprehensive `complete_task()` method that runs quality gates, but doesn't record quality metrics or check for degradation. The quality gates in `file:codeframe/lib/quality_gates.py` already extract test results and coverage data, providing the necessary inputs for quality tracking.

## Approach

The integration follows a layered approach: (1) Initialize `QualityTracker` in `WorkerAgent.__init__()` alongside existing components, (2) Add response count tracking as an instance variable, (3) Record metrics after quality gate execution in `complete_task()`, (4) Check degradation before allowing task completion, (5) Integrate with checkpoint creation and context reset mechanisms. This approach leverages existing quality gate infrastructure and maintains consistency with the current architecture patterns.

## Implementation Steps

### 1. Add QualityTracker Initialization to WorkerAgent

**File:** `file:codeframe/agents/worker_agent.py`

In the `WorkerAgent.__init__()` method (around line 43-77):

- Add instance variable `self.response_count = 0` to track AI conversation length
- Import `QualityTracker` and `QualityMetrics` from `codeframe.enforcement.quality_tracker`
- Initialize `self.quality_tracker = None` (will be set when project context is established)
- Add a helper method `_ensure_quality_tracker()` that lazily initializes the tracker when `project_path` is available from the current task

```
Initialization pattern:
- Store response_count as instance variable
- Lazy-initialize quality_tracker when project context is available
- Use self._get_project_id() to get project context
- Get project workspace_path from database to initialize tracker
```

### 2. Record Quality Metrics After Test Execution

**File:** `file:codeframe/agents/worker_agent.py`

In the `complete_task()` method (around line 812-930), after quality gates run successfully:

- After line 883 where `quality_result = await quality_gates.run_all_gates(task)` completes
- Before the task status update (line 888-897)
- Extract metrics from quality gate results:
  - Parse test pass rate from `quality_result` 
  - Extract coverage percentage from quality gate coverage results
  - Calculate total_tests, passed_tests, failed_tests from test gate results
- Create `QualityMetrics` object with:
  - `timestamp`: Current ISO timestamp
  - `response_count`: `self.response_count`
  - `test_pass_rate`: Calculated from test results (passed/total * 100)
  - `coverage_percentage`: From coverage gate
  - `total_tests`, `passed_tests`, `failed_tests`: From test gate
  - `language`: Detect using `LanguageDetector` from `codeframe.enforcement.language_detector`
  - `framework`: From language detection result
- Call `self.quality_tracker.record(metrics)` to save the checkpoint

```
Metric extraction logic:
- Access quality_result.failures to find test/coverage gate results
- For test gate: count passed vs failed from gate output
- For coverage gate: extract percentage from gate details
- Use LanguageDetector(project_root).detect() for language/framework
```

### 3. Check for Quality Degradation Before Task Completion

**File:** `file:codeframe/agents/worker_agent.py`

In the `complete_task()` method, after recording metrics but before marking task as completed:

- After recording quality metrics (new code from step 2)
- Before line 888 where task status is updated to COMPLETED
- Call `degradation = self.quality_tracker.check_degradation(threshold_percent=10.0)`
- If `degradation["has_degradation"]` is True:
  - Create a blocker using existing `_create_quality_blocker()` method pattern
  - Format blocker message with degradation details from `degradation["issues"]`
  - Include recommendation from `degradation["recommendation"]`
  - Return failure response similar to lines 922-929
  - Keep task status as `in_progress` (don't mark completed)

```
Degradation check flow:
1. Record metrics first (establishes baseline)
2. Check degradation immediately after
3. If degraded > 10%:
   - Create SYNC blocker with degradation details
   - Return {"success": False, "status": "blocked", "message": "..."}
4. If not degraded:
   - Continue to mark task as completed
```

### 4. Increment Response Count on Task Execution

**File:** `file:codeframe/agents/worker_agent.py`

In the `execute_task()` method (around line 219-503):

- After successful LLM call (around line 432-438 where result is returned)
- Increment `self.response_count += 1` before returning the result
- This tracks the conversation length for quality degradation detection

### 5. Integrate with Checkpoint Creation

**File:** `file:codeframe/lib/checkpoint_manager.py`

In the `create_checkpoint()` method (around line 54-143):

- After line 94 where metadata is generated
- Before saving checkpoint to database (line 97)
- Check if quality tracker exists for the project
- If available, get current quality stats using `quality_tracker.get_stats()`
- Add quality metrics to checkpoint metadata:
  - Include current quality metrics in metadata dict
  - Add peak quality metrics for reference
  - Store quality trend (improving/stable/declining)

```
Integration approach:
- Import QualityTracker in checkpoint_manager.py
- Initialize tracker with project_root
- Call get_stats() to retrieve current quality state
- Merge quality stats into existing metadata structure
```

### 6. Add Context Reset Recommendation

**File:** `file:codeframe/agents/worker_agent.py`

Add a new method `should_recommend_context_reset()` after the `should_flash_save()` method (around line 644-673):

- Check if quality tracker is initialized
- Call `self.quality_tracker.should_reset_context(response_count=self.response_count, max_responses=20, check_degradation=True)`
- Return the recommendation dict with `should_reset`, `reasons`, and `recommendation` fields
- This method can be called periodically or before task execution

```
Usage pattern:
- Call before starting new tasks
- Check during long-running sessions
- Integrate with existing flash_save checks
- Log recommendations for monitoring
```

### 7. Dashboard Integration for Quality Trends

**File:** Create new API endpoint in `file:codeframe/ui/routers/` (or extend existing metrics router)

Add endpoint to expose quality history:

- Route: `GET /api/projects/{project_id}/quality-history`
- Initialize `QualityTracker` with project workspace path
- Call `tracker.get_stats()` to get current/peak/average metrics
- Call `tracker.load_history()` to get full history
- Return JSON with quality trends for dashboard visualization

**File:** `file:web-ui/src/components/Dashboard.tsx`

Add quality metrics visualization:

- Fetch quality history from new API endpoint
- Display current vs peak quality metrics
- Show quality trend chart (test pass rate and coverage over time)
- Highlight degradation warnings if present
- Show response count and context reset recommendations

### 8. Add Integration Tests

**File:** Create `file:tests/integration/test_quality_tracker_integration.py`

Test the complete workflow:

- Test 1: Quality metrics recorded after task completion
- Test 2: Degradation detected and blocker created
- Test 3: Context reset recommended when quality degrades
- Test 4: Checkpoint includes quality metrics
- Test 5: Dashboard displays quality trends

```
Test structure:
1. Create WorkerAgent with database
2. Execute task with quality gates
3. Verify metrics recorded in .codeframe/quality_history.json
4. Simulate degradation by recording lower metrics
5. Verify blocker created and task blocked
6. Verify context reset recommendation triggered
```

## Integration Flow Diagram

```mermaid
sequenceDiagram
    participant WA as WorkerAgent
    participant QG as QualityGates
    participant QT as QualityTracker
    participant DB as Database
    participant CM as CheckpointManager

    Note over WA: Task Execution Flow
    WA->>WA: execute_task(task)
    WA->>WA: response_count += 1
    
    Note over WA: Task Completion Flow
    WA->>QG: complete_task(task)
    QG->>QG: run_all_gates(task)
    QG-->>WA: quality_result
    
    alt Quality Gates Pass
        WA->>QT: record(QualityMetrics)
        QT->>QT: save to quality_history.json
        WA->>QT: check_degradation()
        QT-->>WA: degradation result
        
        alt No Degradation
            WA->>DB: mark task completed
            WA-->>WA: success
        else Degradation > 10%
            WA->>DB: create_blocker()
            WA-->>WA: blocked (task stays in_progress)
        end
    else Quality Gates Fail
        QG->>DB: create_blocker()
        WA-->>WA: blocked
    end
    
    Note over CM: Checkpoint Creation
    CM->>QT: get_stats()
    QT-->>CM: quality metrics
    CM->>DB: save checkpoint with quality data
    
    Note over WA: Context Reset Check
    WA->>QT: should_reset_context(response_count)
    QT-->>WA: recommendation
```

## Key Files Modified

| File | Changes |
|------|---------|
| `file:codeframe/agents/worker_agent.py` | Add QualityTracker initialization, response_count tracking, metric recording, degradation checks |
| `file:codeframe/lib/checkpoint_manager.py` | Integrate quality metrics into checkpoint metadata |
| `file:codeframe/ui/routers/` | Add quality history API endpoint |
| `file:web-ui/src/components/Dashboard.tsx` | Add quality trends visualization |
| `file:tests/integration/test_quality_tracker_integration.py` | Add comprehensive integration tests |

## Data Flow

| Trigger | Source | Metrics Captured | Action |
|---------|--------|------------------|--------|
| After test run | `QualityGates.run_all_gates()` | test_pass_rate, coverage_percentage, total_tests | Record to quality_history.json |
| Before task completion | `WorkerAgent.complete_task()` | All metrics + response_count | Check degradation, create blocker if >10% |
| Checkpoint creation | `CheckpointManager.create_checkpoint()` | Current quality stats | Include in checkpoint metadata |
| Context reset check | `WorkerAgent.should_recommend_context_reset()` | response_count + quality trend | Return recommendation |

## Configuration

Quality tracking behavior can be configured via environment variables:

- `QUALITY_DEGRADATION_THRESHOLD`: Percentage threshold for degradation detection (default: 10.0)
- `MAX_RESPONSE_COUNT`: Maximum responses before context reset recommendation (default: 20)
- `QUALITY_TRACKING_ENABLED`: Enable/disable quality tracking (default: true)

---

Execution Information

Branch: main
Commit: 3fa0e9c