fix(e2e): Improve Playwright test pass rate from 18% to 57%

[frankbria]

Summary

This PR improves E2E Playwright test pass rate from 18% to 57% by adding missing data-testid attributes and fixing API field name mismatches.

Key Improvements

Checkpoint Tests: 8/9 passing (89%)

• ✅ Added all required data-testid attributes to CheckpointList component
• ✅ Implemented validation error handling for empty checkpoint names
• ✅ Added checkpoint-list, create-checkpoint-button, checkpoint-item-*, checkpoint-name-input, etc.
• ❌ 1 test failing: 'should validate checkpoint name input' (test logic issue - tries to click disabled button)

Backend API Fixes

• ✅ Fixed field name mismatches in metrics_tracker.py to match TypeScript interfaces:
  • Changed calls → call_count
  • Changed tokens → total_tokens
• ✅ Updated docstrings to reflect corrected field names
• ✅ API verified working: GET /api/projects/2/metrics/costs returns proper data

Project-Agent Assignments

• ✅ Added project-agent assignments in seed-test-data.py
• ✅ Seeds 5 agents with proper roles (orchestrator, backend, frontend, testing, review)
• ✅ Required for multi-agent architecture (PR feat: Multi-agent per project architecture (Phases 1-5) #37)

Test Results

Phase	Tests Passing	Pass Rate	Improvement
Before	2/11	18%	baseline
After Phase 2	17/37	46%	+28%
After Phase 4	21/37	57%	+39%

Files Changed

1. tests/e2e/seed-test-data.py - Added project-agent assignments
2. web-ui/src/components/checkpoints/CheckpointList.tsx - Added 12+ data-testid attributes
3. web-ui/src/components/metrics/CostDashboard.tsx - Added 10+ data-testid attributes
4. codeframe/lib/metrics_tracker.py - Fixed field names to match TypeScript types
5. claudedocs/SESSION.md - Updated progress tracking

Remaining Issues

Metrics Tests (7/12 failing): CostDashboard component not rendering despite API working correctly. Requires frontend debugging with browser dev tools to identify root cause (likely CORS, API client, or SWR caching issue).

Dashboard Tests (6 failing): Related to metrics panel, review panel, and quality gates panel rendering.

Review Tests (2 failing): Review findings list and score chart not displaying.

Next Steps

1. ✅ Merge this PR to capture 57% improvement
2. 🔍 Debug remaining metrics component issues with browser dev tools
3. 🎯 Target 90-100% pass rate in follow-up PR

Related Issues

• Fixes checkpoint test failures from Sprint 10
• Addresses multi-agent architecture integration
• Improves E2E test stability for CI/CD

---

Note: CI will run tests with fresh backend, which should help validate the metrics_tracker.py field name fixes work correctly in a clean environment.

Summary by CodeRabbit

• Bug Fixes

  • Improved checkpoint name validation with inline error messaging during creation.

• Improvements

  • Renamed cost metric fields for clearer token and call reporting.
  • Updated frontend API base URL sourcing to the NEXT_PUBLIC_API_URL environment key.

• New Features

  • Added a CLI-capable database seeding script to populate comprehensive E2E test data.

• Chores

  • Expanded E2E setup and added UI test hooks to improve Playwright test reliability and coverage.

• Documentation

  • Added an E2E endpoint/seed analysis and phased plan guiding test data seeding and validation.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

Walkthrough

Adds a Python SQLite seeder and Playwright global setup orchestration for E2E tests, renames metrics aggregation fields, adds test IDs and validation in UI components, and adds planning/analysis docs describing the seeding strategy and CI reliability plan.

Changes

Cohort / File(s)	Summary
E2E Test Seeding Infrastructure tests/e2e/global-setup.ts, tests/e2e/seed-test-data.py	Adds a Python script to seed SQLite (agents, tasks, token_usage, code_reviews, project-agent assignments) and updates Playwright global setup to discover state.db, run the seeder, and fall back to API-based seeding for checkpoints/reviews with enhanced logging and non-fatal error handling.
Metrics Field Refactor codeframe/lib/metrics_tracker.py	Renames aggregated return keys to total_tokens and call_count for both agent- and model-level entries; updates internal aggregation to match the new keys.
UI Test Instrumentation web-ui/src/components/checkpoints/CheckpointList.tsx, web-ui/src/components/metrics/CostDashboard.tsx	Adds data-testid attributes across checkpoint and cost dashboard UIs and introduces inline name validation state in CheckpointList; no behavioral changes beyond test hooks.
API Env Var Adjustments web-ui/src/api/... web-ui/src/api/checkpoints.ts, .../context.ts, .../metrics.ts, .../qualityGates.ts, .../review.ts, .../reviews.ts	Switches API base URL source from REACT_APP_API_URL to NEXT_PUBLIC_API_URL with same default fallback (http://localhost:8000); no other logic changes.
Planning & Analysis Documentation PHASE1_API_ENDPOINT_ANALYSIS.md, claudedocs/SESSION.md	Adds API endpoint coverage analysis and replaces the prior session doc with a phased E2E Playwright test reliability plan, listing artifacts, roles, and next steps for seeding and CI validation.

Sequence Diagram(s)

sequenceDiagram
    participant Setup as global-setup.ts
    participant API as Backend API
    participant Python as seed-test-data.py
    participant DB as SQLite DB
    participant Tests as Playwright Tests

    Setup->>Setup: locate state.db
    Setup->>API: create or reuse test project
    Setup->>Python: exec seed-test-data.py (db path, project id)
    Python->>DB: connect & insert agents, tasks, token_usage, reviews, project_agents
    Python-->>Setup: seeding finished / status
    Setup->>API: seed checkpoints via API (fallback if unavailable)
    API->>DB: persist checkpoints
    API-->>Setup: checkpoints seeded
    Setup->>API: seed reviews via API (fallback to DB entries if needed)
    API->>DB: persist reviews
    API-->>Setup: reviews seeded
    Setup->>Setup: export PROJECT_ID env var
    Setup-->>Tests: signal readiness
    Tests->>DB: read seeded data during E2E runs

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

• Areas needing careful review:
  • tests/e2e/seed-test-data.py: schema compatibility, transaction handling, idempotency, and table-existence checks.
  • tests/e2e/global-setup.ts: execSync usage, error handling/fallback logic, timing/race conditions before tests start, and environment propagation.
  • codeframe/lib/metrics_tracker.py: ensure consumers (API responses, frontend, tests) are updated to expect total_tokens/call_count.
  • UI data-testid choices: confirm uniqueness and alignment with Playwright selectors.

Possibly related issues

• E2E Test Failures - 7 tests failing across browsers narrative-modeling-app#35 — CI E2E failures and missing seed data: PR adds global-setup.ts and seed-test-data.py that directly address seeding gaps described in the issue.

Poem

🐰 Hop-hop, I planted seeds tonight,

Agents nested under SQLite light,
Tasks and tokens tucked in row by row,
Checkpoints, reviews — now tests can grow,
CI wakes up, and green lights glow.

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately summarizes the main change: improving the Playwright test pass rate from 18% to 57%, which directly reflects the core objective and measurable outcome of this PR.
Docstring Coverage	✅ Passed	Docstring coverage is 90.91% which is sufficient. The required threshold is 80.00%.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch fix/playwright-e2e-tests-ci

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[claude]

Code Review: E2E Playwright Test Improvements

Overall Assessment: ✅ Approve with minor suggestions

This PR makes solid improvements to the E2E test infrastructure, increasing pass rate from 18% to 57% through data seeding and critical bug fixes. The changes are well-structured and align with the project's architecture.

---

🎯 Key Strengths

1. Critical Backend Bug Fix ✅

The field name corrections in metrics_tracker.py are essential:

• Changed calls → call_count (lines 277, 281, 289, 293)
• Changed tokens → total_tokens (lines 276, 280)
• Correctly aligns with TypeScript interfaces in web-ui/src/types/metrics.ts:76-105

Impact: This fixes API contract violations that would cause frontend parsing errors.

2. Comprehensive Test Data Seeding ✅

tests/e2e/seed-test-data.py properly seeds:

• 5 agents with proper roles (lead, backend, frontend, testing, review)
• Project-agent assignments (lines 64-94) - Critical for multi-agent architecture (PR feat: Multi-agent per project architecture (Phases 1-5) #37)
• 10 tasks with realistic status distribution
• 15 token usage records across 3 models
• Code review findings and checkpoints

Strength: Direct database seeding is the correct approach given missing API endpoints (documented in PHASE1_API_ENDPOINT_ANALYSIS.md).

3. Improved Test Accessibility ✅

Added 20+ data-testid attributes:

• CheckpointList: checkpoint-list, create-checkpoint-button, checkpoint-item-{id}, etc.
• CostDashboard: cost-dashboard, cost-total, agent-cost-table, etc.

Best Practice: Follows React Testing Library conventions for stable, maintainable E2E tests.

4. Input Validation ✅

CheckpointList validation (lines 60-64):

if (\!newCheckpointName.trim()) {
  setNameError('Checkpoint name is required');
  return;
}

Good: Prevents empty checkpoint names, displays user-friendly error message.

---

⚠️ Areas for Improvement

1. Security: SQL Injection Risk (Medium)

File: tests/e2e/seed-test-data.py
Lines: 52-55, 85-88, and throughout

Issue: Direct string interpolation in SQL queries:

cursor.execute("""
    INSERT INTO agents (id, type, provider, ...)
    VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""", agent)

✅ Actually CORRECT - Using parameterized queries with ? placeholders.

Recommendation: No change needed - already following best practices! I initially flagged this but on closer inspection, the code is secure.

2. Error Handling: Silent Failures

File: tests/e2e/seed-test-data.py
Lines: 56-57, 89-90

Issue: Individual insert failures are logged but don't fail the seeding process:

except sqlite3.Error as e:
    print(f"⚠️  Failed to insert agent {agent[0]}: {e}")

Risk: Tests might pass with incomplete data (e.g., 3/5 agents seeded).

Recommendation:

failed_inserts = []
for agent in agents:
    try:
        cursor.execute(...)
    except sqlite3.Error as e:
        failed_inserts.append((agent[0], str(e)))

if failed_inserts:
    raise Exception(f"Failed to seed {len(failed_inserts)} agents: {failed_inserts}")

3. Type Safety: Missing Total Fields

File: codeframe/lib/metrics_tracker.py
Lines: 246-251

Issue: result dict initialization includes total_tokens and total_calls, but TypeScript CostBreakdown interface only expects:

interface CostBreakdown {
  total_cost_usd: number;
  by_agent: AgentCostBreakdown[];
  by_model: ModelCostBreakdown[];
}

Impact: Extra fields in API response (not breaking, but inconsistent).

Recommendation: Remove total_tokens and total_calls from result dict (lines 248-249) OR update TypeScript interface to include them.

4. Performance: Database Connection Management

File: tests/e2e/seed-test-data.py
Line: 14

Issue: Single connection for all operations without connection pooling.

Current:

conn = sqlite3.connect(db_path)
cursor = conn.cursor()

Recommendation: Use context manager for automatic cleanup:

with sqlite3.connect(db_path) as conn:
    cursor = conn.cursor()
    # ... seeding operations

5. Testing: Missing Test Coverage

Gap: No unit tests for seed-test-data.py script.

Risk: Seeding logic bugs could break CI without local detection.

Recommendation: Add tests/e2e/test_seed_data.py:

def test_seed_agents_creates_five_agents(tmp_db_path):
    seed_test_data(tmp_db_path, project_id=1)
    conn = sqlite3.connect(tmp_db_path)
    cursor = conn.cursor()
    cursor.execute("SELECT COUNT(*) FROM agents")
    assert cursor.fetchone()[0] == 5

6. Documentation: Field Name Change Migration

Missing: No mention of breaking API change in PR description or CHANGELOG.

Impact: External API consumers (if any) would break silently.

Recommendation: Add migration notes:

## Breaking Changes

### Metrics API (`GET /api/projects/{id}/metrics/costs`)
- `by_agent[].calls` → `by_agent[].call_count`
- `by_agent[].tokens` → `by_agent[].total_tokens`
- `by_model[].total_calls` → `by_model[].call_count`

---

🧪 Testing Recommendations

1. Validate metrics_tracker changes:

   pytest tests/lib/test_metrics_tracker.py -v
   pytest tests/api/test_api_metrics.py -v

2. Test checkpoint validation:

   cd tests/e2e
   npx playwright test test_checkpoint_ui.spec.ts --project=chromium

3. Verify seeding script:

   python3 tests/e2e/seed-test-data.py state.db 2
   sqlite3 state.db "SELECT COUNT(*) FROM agents"  # Should return 5

---

📊 Code Quality Metrics

Metric	Score	Notes
Code Style	✅ 9/10	Follows project conventions, good formatting
Type Safety	⚠️ 7/10	TypeScript alignment good, but extra fields in Python
Error Handling	⚠️ 6/10	Silent failures in seeding script
Security	✅ 9/10	Parameterized queries used correctly
Test Coverage	⚠️ 5/10	Seeding script lacks unit tests
Documentation	⚠️ 7/10	Good analysis docs, missing breaking change notes

Overall: 7.5/10

---

✅ Approval Decision

Approved with the following conditions:

Must Fix Before Merge:

1. ❌ None - All critical issues resolved

Should Fix Soon (Follow-up PR):

1. ⚠️ Add unit tests for seed-test-data.py
2. ⚠️ Document API field name changes as breaking
3. ⚠️ Fail fast on seeding errors (don't continue with partial data)

Nice to Have:

1. 💡 Use context manager for DB connection in seeding script
2. 💡 Align total_tokens/total_calls between Python and TypeScript

---

🚀 Impact Assessment

• Risk Level: Low (test infrastructure only, no prod impact)
• Breaking Changes: Yes (API field names, but aligns with types)
• Rollback Plan: Revert commit, field names easily reversible
• Performance Impact: None (E2E tests only)

---

📝 Summary

Excellent work improving E2E test stability! The 39% improvement in pass rate (18% → 57%) demonstrates significant progress. The backend field name fix is critical and correctly resolves type mismatches.

Next Steps:

1. Merge this PR to capture improvements ✅
2. Address remaining 16 test failures in follow-up PR (as planned in PR description)
3. Add unit test coverage for seeding script

Recommendation: APPROVE AND MERGE 🎉

Great job on the comprehensive analysis documents (PHASE1_API_ENDPOINT_ANALYSIS.md) - this will help future contributors understand the E2E test architecture!

---

Reviewed by: Claude Code (Sonnet 4.5)
Review Date: 2025-12-03

[coderabbitai]

Actionable comments posted: 3

🧹 Nitpick comments (6)

web-ui/src/components/metrics/CostDashboard.tsx (1)

199-199: Consider sanitizing model names in data-testid attributes.

Model names like claude-sonnet-4-5 work fine, but if model names ever contain special characters (spaces, dots), they could cause issues in test selectors.

-                  <tr key={model.model_name} className="hover:bg-gray-50" data-testid={`model-cost-${model.model_name}`}>
+                  <tr key={model.model_name} className="hover:bg-gray-50" data-testid={`model-cost-${model.model_name.replace(/[^a-zA-Z0-9-]/g, '-')}`}>

codeframe/lib/metrics_tracker.py (1)

366-373: Inconsistent field naming in get_agent_costs method.

The get_project_costs method uses call_count (lines 277, 281, 289, 293), but get_agent_costs uses calls (line 370, 373). For API consistency, consider aligning the field names.

             if call_type not in call_type_stats:
                 call_type_stats[call_type] = {
                     "call_type": call_type,
                     "cost_usd": 0.0,
-                    "calls": 0
+                    "call_count": 0
                 }
             call_type_stats[call_type]["cost_usd"] += cost
-            call_type_stats[call_type]["calls"] += 1
+            call_type_stats[call_type]["call_count"] += 1

Also update the docstring at lines 321-323 to reflect the field name change.

claudedocs/SESSION.md (1)

84-88: Add language specifier to fenced code blocks.

Per static analysis (markdownlint MD040), fenced code blocks should have a language specified. These appear to be file path listings.

-```
+```text
 tests/e2e/
 ├── global-setup.ts        (MODIFIED) - Add seeding orchestration
 └── seed-test-data.py      (NEW) - Python seeding script

Apply similarly to the code blocks at lines 147-154 and 182-186.


Also applies to: 147-154, 182-186

</blockquote></details>
<details>
<summary>tests/e2e/seed-test-data.py (1)</summary><blockquote>

`47-48`: **Clearing all agents may affect other projects in shared environments.**

Line 48 deletes all agents (`DELETE FROM agents`) rather than scoping to a specific project. If multiple test runs share a database, this could cause data conflicts.



Consider adding a comment explaining this is intentional for isolated E2E test databases, or scope deletion if the agents table has project association:

```diff
             # Clear existing agents (no project_id in agents table)
+            # NOTE: This clears ALL agents - safe for isolated E2E test DBs only
             cursor.execute("DELETE FROM agents")

tests/e2e/global-setup.ts (2)

55-59: Silently continuing after seeding failure may hide critical issues.

The catch block logs the error but allows tests to proceed. While this prevents total failure, it may lead to confusing test failures when data is missing.

Consider setting an environment variable to indicate seeding status, so tests can skip gracefully:

   } catch (error) {
     console.error('❌ Failed to seed database:', error);
     console.warn('⚠️  Tests may fail due to missing test data');
+    process.env.E2E_SEEDING_FAILED = 'true';
     // Don't throw - allow tests to run even if seeding fails
   }

---

49-52: Hardcoded python3 may not exist on all CI environments.

Some systems use python instead of python3, which could cause seeding to fail.

-    const command = `python3 "${scriptPath}" "${dbPath}" ${projectId}`;
+    // Try python3 first, fall back to python
+    const pythonCmd = process.platform === 'win32' ? 'python' : 'python3';
+    const command = `${pythonCmd} "${scriptPath}" "${dbPath}" ${projectId}`;

Or use environment variable:

+    const pythonCmd = process.env.PYTHON_CMD || 'python3';
-    const command = `python3 "${scriptPath}" "${dbPath}" ${projectId}`;
+    const command = `${pythonCmd} "${scriptPath}" "${dbPath}" ${projectId}`;

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7f7e895 and a4b099b.

📒 Files selected for processing (7)

• PHASE1_API_ENDPOINT_ANALYSIS.md (1 hunks)
• claudedocs/SESSION.md (1 hunks)
• codeframe/lib/metrics_tracker.py (2 hunks)
• tests/e2e/global-setup.ts (3 hunks)
• tests/e2e/seed-test-data.py (1 hunks)
• web-ui/src/components/checkpoints/CheckpointList.tsx (11 hunks)
• web-ui/src/components/metrics/CostDashboard.tsx (4 hunks)

🧰 Additional context used

📓 Path-based instructions (8)

tests/**/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

tests/**/*.py: Run all tests with pytest
Maintain 88%+ test coverage for Sprint 10 components and 100% pass rate

Files:

• tests/e2e/seed-test-data.py

tests/e2e/**/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

Use TestSprite MCP for E2E test generation and Playwright for frontend E2E testing

Files:

• tests/e2e/seed-test-data.py

web-ui/src/**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

web-ui/src/**/*.{ts,tsx}: Use TypeScript 5.3+ with React, strict mode, and maintain 85%+ test coverage for frontend code
Use React 18 with Tailwind CSS for frontend styling
Use Context + Reducer pattern (React Context with useReducer) for centralized state management in frontend

Files:

• web-ui/src/components/checkpoints/CheckpointList.tsx
• web-ui/src/components/metrics/CostDashboard.tsx

web-ui/**/*.{ts,tsx,test.ts,test.tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

Run frontend tests with npm test from web-ui directory

Files:

• web-ui/src/components/checkpoints/CheckpointList.tsx
• web-ui/src/components/metrics/CostDashboard.tsx

web-ui/src/components/**/*.tsx

📄 CodeRabbit inference engine (CLAUDE.md)

Use React.memo on all Dashboard sub-components for performance optimization

Files:

• web-ui/src/components/checkpoints/CheckpointList.tsx
• web-ui/src/components/metrics/CostDashboard.tsx

**/*.md

📄 CodeRabbit inference engine (AGENTS.md)

Documentation files must be sized to fit in a single agent context window (spec.md ~200-400 lines, plan.md ~300-600 lines, tasks.md ~400-800 lines)

Files:

• claudedocs/SESSION.md
• PHASE1_API_ENDPOINT_ANALYSIS.md

codeframe/**/*.py

📄 CodeRabbit inference engine (CLAUDE.md)

codeframe/**/*.py: Use Python 3.11+ with async/await pattern, type hints, and comprehensive tests for backend code
Use ruff for linting Python code
Use async/await pattern for promises in Python async code
Use FastAPI for backend API implementation
Use SQLite with async support (aiosqlite) for database operations
Use WebSocket for real-time updates between backend and frontend
API endpoints should accept project_id query parameter for multi-project support

Files:

• codeframe/lib/metrics_tracker.py

codeframe/lib/metrics_tracker.py

📄 CodeRabbit inference engine (CLAUDE.md)

codeframe/lib/metrics_tracker.py: Record token usage automatically after every LLM API call with model name, input/output tokens, and call type
Calculate model costs using pricing table: Sonnet 4.5 (3.00/15.00), Opus 4 (15.00/75.00), Haiku 4 (0.80/4.00) USD per million tokens

Files:

• codeframe/lib/metrics_tracker.py

🧠 Learnings (7)

📓 Common learnings

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to web-ui/src/**/*.{ts,tsx} : Use TypeScript 5.3+ with React, strict mode, and maintain 85%+ test coverage for frontend code

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to tests/**/*.py : Maintain 88%+ test coverage for Sprint 10 components and 100% pass rate

📚 Learning: 2025-11-25T19:08:54.154Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: AGENTS.md:0-0
Timestamp: 2025-11-25T19:08:54.154Z
Learning: Applies to {README.md,CODEFRAME_SPEC.md,CHANGELOG.md,SPRINTS.md,CLAUDE.md,AGENTS.md,TESTING.md,CONTRIBUTING.md} : Root-level documentation must include: README.md (project intro), CODEFRAME_SPEC.md (architecture, ~800 lines), CHANGELOG.md (user-facing changes), SPRINTS.md (timeline index), CLAUDE.md (coding standards), AGENTS.md (navigation guide), TESTING.md (test standards), and CONTRIBUTING.md (contribution guidelines)

Applied to files:

• claudedocs/SESSION.md

📚 Learning: 2025-11-25T19:08:20.036Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to tests/e2e/**/*.py : Use TestSprite MCP for E2E test generation and Playwright for frontend E2E testing

Applied to files:

• tests/e2e/global-setup.ts

📚 Learning: 2025-11-25T19:08:20.036Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to web-ui/src/components/**/*.tsx : Use React.memo on all Dashboard sub-components for performance optimization

Applied to files:

• web-ui/src/components/metrics/CostDashboard.tsx

📚 Learning: 2025-11-25T19:08:20.036Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to web-ui/src/**/*.{ts,tsx} : Use TypeScript 5.3+ with React, strict mode, and maintain 85%+ test coverage for frontend code

Applied to files:

• web-ui/src/components/metrics/CostDashboard.tsx

📚 Learning: 2025-11-25T19:08:20.036Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to codeframe/lib/metrics_tracker.py : Calculate model costs using pricing table: Sonnet 4.5 (3.00/15.00), Opus 4 (15.00/75.00), Haiku 4 (0.80/4.00) USD per million tokens

Applied to files:

• codeframe/lib/metrics_tracker.py

📚 Learning: 2025-11-25T19:08:20.036Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to codeframe/lib/metrics_tracker.py : Record token usage automatically after every LLM API call with model name, input/output tokens, and call type

Applied to files:

• codeframe/lib/metrics_tracker.py

🧬 Code graph analysis (1)

tests/e2e/seed-test-data.py (2)

codeframe/cli.py (1)

• agents (164-169)

tests/agents/test_review_worker_agent.py (1)

• agent (49-56)

🪛 GitHub Actions: Test Suite (Unit + E2E)

tests/e2e/seed-test-data.py

[error] 10-10: Ruff: F401 imported but unused ('pathlib.Path'). Remove unused import. Found 1 error. 1 fixable with the --fix option.

🪛 markdownlint-cli2 (0.18.1)

claudedocs/SESSION.md

84-84: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

147-147: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

---

182-182: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: claude-review

🔇 Additional comments (7)

web-ui/src/components/metrics/CostDashboard.tsx (1)

115-228: LGTM! Data-testid attributes added correctly for E2E testing.

The component properly uses React.memo for performance optimization as per coding guidelines, and all data-testid attributes are appropriately placed for testing key UI elements.

codeframe/lib/metrics_tracker.py (1)

226-231: Field names correctly updated to match TypeScript interfaces.

The change from tokens/calls to total_tokens/call_count aligns the backend API response with the frontend's expected data structure.

web-ui/src/components/checkpoints/CheckpointList.tsx (2)

27-27: Good UX improvement with per-field validation.

The nameError state provides immediate feedback when the checkpoint name is empty, with proper clearing on valid input. This improves user experience over the previous generic error handling.

Also applies to: 60-68, 187-202

---

154-265: Comprehensive data-testid coverage for E2E testing.

All key UI elements are properly tagged: list container, create button, modal, inputs, action buttons, empty state, and individual checkpoint items. This enables reliable Playwright test selectors.

tests/e2e/global-setup.ts (1)

16-31: Good defensive approach for finding the database.

The findDatabasePath() function checks multiple common locations, which improves reliability across different CI environments and local setups.

PHASE1_API_ENDPOINT_ANALYSIS.md (2)

206-217: Documentation is well-structured and appropriately sized.

The Phase 1 analysis is comprehensive, fits within a single context window (~217 lines), and logically progresses through findings → strategy → next steps. The executive summary and validation checklist make it easy to navigate.

---

1-217: Verify database method line numbers and confirm Phase 2 implementation follows Phase 1 recommendations.

This Phase 1 analysis document provides foundational findings for test data seeding strategy, but verification is needed for:

1. Line number accuracy in database.py: The document references specific line numbers for methods like create_agent (line 1169), create_task (line 653), save_token_usage (line 3424), and others. These may have shifted during development and should be validated against the actual codebase.

2. Phase 2 seed-test-data.py implementation: The document states (line 174) that tests/e2e/seed-test-data.py doesn't exist yet. Verify whether Phase 2 created this script and whether the implementation aligns with the Phase 1 recommendations (seeding agents, tasks, token usage, reviews, quality gates, and project-agent assignments).

3. Pseudo-code imports (lines 124-129): Confirm that imports like from codeframe.persistence.database import Database and from codeframe.core.models import Task, Agent reflect the actual module structure.

4. Pseudo-code method calls: Verify that methods shown in the code example (create_agent(), create_task(), save_token_usage()) match the actual database method signatures.

[coderabbitai]

🛠️ Refactor suggestion | 🟠 Major

Large block of unused code: seedAgents, seedTasks, seedTokenUsage, and seedReviews functions are never called.

These functions (lines 65-768) are defined but never invoked. The actual seeding is done via seedDatabaseDirectly() (Python script) at line 822 and seedCheckpoints() at line 828.

Either:

1. Remove these unused functions to reduce maintenance burden, or
2. Call them as a fallback when Python seeding is unavailable

If keeping as documentation/reference, add a comment:

+/**
+ * NOTE: The following seed functions are kept for reference but not used.
+ * Actual seeding is done via Python script (seedDatabaseDirectly).
+ * These can be enabled if API endpoints become available.
+ */
+
 /**
  * Seed 5 agents with mixed statuses (working, idle, blocked).
  */
 async function seedAgents(page: Page, projectId: number): Promise<void> {

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	async function seedAgents(page: Page, projectId: number): Promise<void> {
	console.log('👥 Seeding agents...');
	const agents = [
	{
	id: 'lead-001',
	type: 'lead',
	status: 'working',
	provider: 'anthropic',
	maturity: 'delegating',
	current_task: { id: 1, title: 'Orchestrate project' },
	context_tokens: 25000,
	tasks_completed: 12,
	timestamp: Date.now()
	},
	{
	id: 'backend-worker-001',
	type: 'backend-worker',
	status: 'working',
	provider: 'anthropic',
	maturity: 'delegating',
	current_task: { id: 2, title: 'Implement API endpoints' },
	context_tokens: 45000,
	tasks_completed: 8,
	timestamp: Date.now()
	},
	{
	id: 'frontend-specialist-001',
	type: 'frontend-specialist',
	status: 'idle',
	provider: 'anthropic',
	maturity: 'supporting',
	context_tokens: 12000,
	tasks_completed: 5,
	timestamp: Date.now()
	},
	{
	id: 'test-engineer-001',
	type: 'test-engineer',
	status: 'working',
	provider: 'anthropic',
	maturity: 'delegating',
	current_task: { id: 3, title: 'Write E2E tests' },
	context_tokens: 30000,
	tasks_completed: 15,
	timestamp: Date.now()
	},
	{
	id: 'review-agent-001',
	type: 'review',
	status: 'blocked',
	provider: 'anthropic',
	maturity: 'delegating',
	blocker: 'Waiting for code review completion',
	context_tokens: 18000,
	tasks_completed: 20,
	timestamp: Date.now()
	}
	];
	let createdCount = 0;
	for (const agent of agents) {
	try {
	// Note: The backend may not have a direct POST /api/agents endpoint.
	// Agents are typically created internally by the system.
	// We'll try the endpoint, but expect it may not exist.
	const response = await page.request.post(`${BACKEND_URL}/api/agents`, {
	data: agent,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	} else {
	console.warn(`⚠️ Failed to create agent ${agent.id}: ${response.statusText()}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create agent ${agent.id}:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${agents.length} agents`);
	} else {
	console.log('⚠️ No agents created (endpoint may not exist or agents created internally)');
	}
	}
	/**
	* Seed 10 tasks with mixed statuses (completed, in_progress, blocked, pending).
	*/
	async function seedTasks(page: Page, projectId: number): Promise<void> {
	console.log('📋 Seeding tasks...');
	const tasks = [
	// Completed tasks
	{
	id: 1,
	project_id: projectId,
	title: 'Setup project structure',
	description: 'Initialize project repository and workspace',
	status: 'completed',
	assigned_to: 'lead-001',
	priority: 1,
	workflow_step: 1,
	timestamp: Date.now() - 86400000 * 2 // 2 days ago
	},
	{
	id: 2,
	project_id: projectId,
	title: 'Implement authentication API',
	description: 'Build JWT-based authentication endpoints',
	status: 'completed',
	assigned_to: 'backend-worker-001',
	priority: 1,
	workflow_step: 2,
	timestamp: Date.now() - 86400000 * 1 // 1 day ago
	},
	{
	id: 3,
	project_id: projectId,
	title: 'Write unit tests for auth',
	description: 'Comprehensive test coverage for authentication',
	status: 'completed',
	assigned_to: 'test-engineer-001',
	priority: 1,
	workflow_step: 3,
	timestamp: Date.now() - 43200000 // 12 hours ago
	},
	// In-progress tasks
	{
	id: 4,
	project_id: projectId,
	title: 'Build dashboard UI',
	description: 'Create React dashboard with real-time updates',
	status: 'in_progress',
	assigned_to: 'frontend-specialist-001',
	priority: 2,
	workflow_step: 4,
	timestamp: Date.now() - 7200000 // 2 hours ago
	},
	{
	id: 5,
	project_id: projectId,
	title: 'Add token usage tracking',
	description: 'Implement token counting and cost analytics',
	status: 'in_progress',
	assigned_to: 'backend-worker-001',
	priority: 2,
	workflow_step: 4,
	timestamp: Date.now() - 3600000 // 1 hour ago
	},
	// Blocked tasks
	{
	id: 6,
	project_id: projectId,
	title: 'Deploy to production',
	description: 'Set up production deployment pipeline',
	status: 'blocked',
	depends_on: '7,8',
	priority: 3,
	workflow_step: 6,
	timestamp: Date.now() - 1800000 // 30 minutes ago
	},
	{
	id: 7,
	project_id: projectId,
	title: 'Security audit',
	description: 'Comprehensive security review and penetration testing',
	status: 'blocked',
	depends_on: '4',
	priority: 2,
	workflow_step: 5,
	timestamp: Date.now() - 1800000 // 30 minutes ago
	},
	// Pending tasks
	{
	id: 8,
	project_id: projectId,
	title: 'Write API documentation',
	description: 'OpenAPI/Swagger documentation for all endpoints',
	status: 'pending',
	priority: 3,
	workflow_step: 5,
	timestamp: Date.now()
	},
	{
	id: 9,
	project_id: projectId,
	title: 'Optimize database queries',
	description: 'Add indexes and optimize slow queries',
	status: 'pending',
	priority: 2,
	workflow_step: 5,
	timestamp: Date.now()
	},
	{
	id: 10,
	project_id: projectId,
	title: 'Add logging middleware',
	description: 'Structured logging with request/response tracking',
	status: 'pending',
	priority: 2,
	workflow_step: 5,
	timestamp: Date.now()
	}
	];
	let createdCount = 0;
	for (const task of tasks) {
	try {
	const response = await page.request.post(`${BACKEND_URL}/api/tasks`, {
	data: task,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	} else {
	console.warn(`⚠️ Failed to create task ${task.id}: ${response.statusText()}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create task ${task.id}:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${tasks.length} tasks`);
	} else {
	console.log('⚠️ No tasks created (endpoint may not exist)');
	}
	}
	/**
	* Seed 15 token usage records across 3 models (Sonnet, Opus, Haiku) and 3 days.
	* Total cost: ~$4.46 USD
	*/
	async function seedTokenUsage(page: Page, projectId: number): Promise<void> {
	console.log('💰 Seeding token usage records...');
	const now = Date.now();
	const dayMs = 86400000; // 24 hours in milliseconds
	const tokenRecords = [
	// Backend agent usage (Sonnet)
	{
	task_id: 2,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 12500,
	output_tokens: 4800,
	estimated_cost_usd: 0.11,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2).toISOString()
	},
	{
	task_id: 2,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 8900,
	output_tokens: 3200,
	estimated_cost_usd: 0.075,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2 + 5400000).toISOString() // +1.5h
	},
	// Frontend agent usage (Haiku for smaller tasks)
	{
	task_id: 4,
	agent_id: 'frontend-specialist-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 5000,
	output_tokens: 2000,
	estimated_cost_usd: 0.012,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2 + 14400000).toISOString() // +4h
	},
	{
	task_id: 4,
	agent_id: 'frontend-specialist-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 6200,
	output_tokens: 2500,
	estimated_cost_usd: 0.015,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 1 + 3600000).toISOString() // Day 2 +1h
	},
	// Test engineer usage (Sonnet)
	{
	task_id: 3,
	agent_id: 'test-engineer-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 15000,
	output_tokens: 6000,
	estimated_cost_usd: 0.135,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2 + 21600000).toISOString() // +6h
	},
	// Review agent usage (Opus for code review)
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 25000,
	output_tokens: 8000,
	estimated_cost_usd: 0.975,
	call_type: 'code_review',
	timestamp: new Date(now - dayMs * 1 + 10800000).toISOString() // Day 2 +3h
	},
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 18000,
	output_tokens: 5500,
	estimated_cost_usd: 0.6825,
	call_type: 'code_review',
	timestamp: new Date(now - dayMs * 1 + 18000000).toISOString() // Day 2 +5h
	},
	// Lead agent coordination (Sonnet)
	{
	agent_id: 'lead-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 8000,
	output_tokens: 3000,
	estimated_cost_usd: 0.069,
	call_type: 'coordination',
	timestamp: new Date(now - 3600000).toISOString() // Today -1h
	},
	// Additional records for time-series (Day 3 - today)
	{
	task_id: 5,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 10000,
	output_tokens: 4000,
	estimated_cost_usd: 0.09,
	call_type: 'task_execution',
	timestamp: new Date(now - 1800000).toISOString() // Today -30min
	},
	{
	task_id: 4,
	agent_id: 'frontend-specialist-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 7000,
	output_tokens: 2800,
	estimated_cost_usd: 0.017,
	call_type: 'task_execution',
	timestamp: new Date(now - 900000).toISOString() // Today -15min
	},
	// More Opus usage for higher costs
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 30000,
	output_tokens: 10000,
	estimated_cost_usd: 1.2,
	call_type: 'code_review',
	timestamp: new Date(now - 7200000).toISOString() // Today -2h
	},
	// Haiku for quick coordination
	{
	agent_id: 'lead-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 3000,
	output_tokens: 1200,
	estimated_cost_usd: 0.0072,
	call_type: 'coordination',
	timestamp: new Date(now - 5400000).toISOString() // Today -1.5h
	},
	// Additional Sonnet usage
	{
	task_id: 5,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 14000,
	output_tokens: 5500,
	estimated_cost_usd: 0.1245,
	call_type: 'task_execution',
	timestamp: new Date(now - 10800000).toISOString() // Today -3h
	},
	{
	task_id: 3,
	agent_id: 'test-engineer-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 11000,
	output_tokens: 4200,
	estimated_cost_usd: 0.096,
	call_type: 'task_execution',
	timestamp: new Date(now - 14400000).toISOString() // Today -4h
	},
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 22000,
	output_tokens: 7000,
	estimated_cost_usd: 0.855,
	call_type: 'code_review',
	timestamp: new Date(now - 18000000).toISOString() // Today -5h
	}
	];
	let createdCount = 0;
	for (const record of tokenRecords) {
	try {
	// Try the most likely endpoints
	const endpoints = [
	`/api/projects/${projectId}/metrics/tokens`,
	`/api/token-usage`
	];
	let success = false;
	for (const endpoint of endpoints) {
	try {
	const response = await page.request.post(`${BACKEND_URL}${endpoint}`, {
	data: record,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	success = true;
	break;
	}
	} catch (error) {
	// Try next endpoint
	continue;
	}
	}
	if (!success) {
	console.warn(`⚠️ Failed to create token usage record for agent ${record.agent_id}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create token usage record:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${tokenRecords.length} token usage records (~$4.46 total)`);
	} else {
	console.log('⚠️ No token usage records created (endpoint may not exist)');
	}
	}
	/**
	* Seed 3 checkpoints with Git commit SHAs and metadata.
	*/
	async function seedCheckpoints(page: Page, projectId: number): Promise<void> {
	console.log('💾 Seeding checkpoints...');
	const now = Date.now();
	const dayMs = 86400000;
	const checkpoints = [
	{
	project_id: projectId,
	name: 'Initial setup complete',
	description: 'Project structure and authentication working',
	trigger: 'phase_transition',
	git_commit: 'a1b2c3d4e5f6',
	database_backup_path: '.codeframe/checkpoints/checkpoint-001-db.sqlite',
	context_snapshot_path: '.codeframe/checkpoints/checkpoint-001-context.json',
	metadata: {
	project_id: projectId,
	phase: 'setup',
	tasks_completed: 3,
	tasks_total: 10,
	agents_active: ['lead-001', 'backend-worker-001', 'test-engineer-001'],
	last_task_completed: 'Write unit tests for auth',
	context_items_count: 45,
	total_cost_usd: 1.2
	},
	created_at: new Date(now - dayMs * 2 + 64800000).toISOString() // 2 days ago + 18h
	},
	{
	project_id: projectId,
	name: 'UI development milestone',
	description: 'Dashboard UI 50% complete',
	trigger: 'manual',
	git_commit: 'f6e5d4c3b2a1',
	database_backup_path: '.codeframe/checkpoints/checkpoint-002-db.sqlite',
	context_snapshot_path: '.codeframe/checkpoints/checkpoint-002-context.json',
	metadata: {
	project_id: projectId,
	phase: 'ui-development',
	tasks_completed: 4,
	tasks_total: 10,
	agents_active: ['lead-001', 'frontend-specialist-001'],
	last_task_completed: 'Build dashboard UI',
	context_items_count: 78,
	total_cost_usd: 2.8
	},
	created_at: new Date(now - dayMs * 1 + 72000000).toISOString() // 1 day ago + 20h
	},
	{
	project_id: projectId,
	name: 'Pre-review snapshot',
	description: 'Before code review process',
	trigger: 'auto',
	git_commit: '9876543210ab',
	database_backup_path: '.codeframe/checkpoints/checkpoint-003-db.sqlite',
	context_snapshot_path: '.codeframe/checkpoints/checkpoint-003-context.json',
	metadata: {
	project_id: projectId,
	phase: 'review',
	tasks_completed: 5,
	tasks_total: 10,
	agents_active: ['lead-001', 'review-agent-001'],
	last_task_completed: 'Add token usage tracking',
	context_items_count: 120,
	total_cost_usd: 4.46
	},
	created_at: new Date(now - 3600000).toISOString() // Today -1h
	}
	];
	let createdCount = 0;
	for (const checkpoint of checkpoints) {
	try {
	const response = await page.request.post(
	`${BACKEND_URL}/api/projects/${projectId}/checkpoints`,
	{
	data: checkpoint,
	timeout: 10000
	}
	);
	if (response.ok()) {
	createdCount++;
	} else {
	console.warn(`⚠️ Failed to create checkpoint "${checkpoint.name}": ${response.statusText()}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create checkpoint "${checkpoint.name}":`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${checkpoints.length} checkpoints`);
	} else {
	console.log('⚠️ No checkpoints created (endpoint may not exist)');
	}
	}
	/**
	* Seed 2 review reports: 1 approved, 1 changes_requested.
	*/
	async function seedReviews(page: Page, projectId: number): Promise<void> {
	console.log('🔍 Seeding review reports...');
	const now = Date.now();
	const reviews = [
	{
	task_id: 2,
	reviewer_agent_id: 'review-agent-001',
	overall_score: 85,
	complexity_score: 80,
	security_score: 90,
	style_score: 85,
	status: 'approved',
	findings: [
	{
	file_path: 'codeframe/api/auth.py',
	line_number: 45,
	category: 'security',
	severity: 'medium',
	message: 'Consider adding rate limiting to login endpoint to prevent brute force attacks',
	suggestion: "Use FastAPI's limiter middleware with 5 requests per minute limit"
	},
	{
	file_path: 'codeframe/api/auth.py',
	line_number: 78,
	category: 'style',
	severity: 'low',
	message: "Function 'validate_token' exceeds 50 lines, consider extracting helper functions",
	suggestion: 'Extract JWT decoding logic into separate function'
	},
	{
	file_path: 'codeframe/api/auth.py',
	line_number: 120,
	category: 'coverage',
	severity: 'medium',
	message: 'Error handling path not covered by tests (line 120-125)',
	suggestion: 'Add test case for expired token scenario'
	}
	],
	summary: 'Good implementation overall. Authentication logic is solid with proper JWT handling. Main concerns are rate limiting and test coverage for error paths. Approved with suggested improvements.',
	created_at: new Date(now - 86400000 * 1 + 43200000).toISOString() // 1 day ago + 12h
	},
	{
	task_id: 4,
	reviewer_agent_id: 'review-agent-001',
	overall_score: 65,
	complexity_score: 60,
	security_score: 75,
	style_score: 70,
	status: 'changes_requested',
	findings: [
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 125,
	category: 'security',
	severity: 'critical',
	message: 'User input not sanitized before rendering, potential XSS vulnerability',
	suggestion: 'Use DOMPurify to sanitize user-generated content before rendering'
	},
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 200,
	category: 'complexity',
	severity: 'high',
	message: 'Component exceeds 300 lines, violating single responsibility principle',
	suggestion: 'Extract AgentStatusPanel, TaskList, and MetricsChart into separate components'
	},
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 45,
	category: 'style',
	severity: 'medium',
	message: 'useState hooks not grouped at top of component',
	suggestion: 'Move all useState declarations to top of component for better readability'
	},
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 180,
	category: 'owasp',
	severity: 'critical',
	message: 'Sensitive data (API tokens) logged to console in production build',
	suggestion: 'Remove console.log statements or gate with NODE_ENV check'
	}
	],
	summary: 'Component needs refactoring before approval. Critical security issues found: XSS vulnerability and token exposure in logs. Component is too complex (300+ lines) and violates separation of concerns. Please address critical findings before re-review.',
	created_at: new Date(now - 7200000).toISOString() // Today -2h
	}
	];
	let createdCount = 0;
	for (const review of reviews) {
	try {
	// Try multiple possible endpoints
	const endpoints = [
	`/api/reviews`,
	`/api/projects/${projectId}/reviews`,
	`/api/agents/${review.reviewer_agent_id}/review`
	];
	let success = false;
	for (const endpoint of endpoints) {
	try {
	const response = await page.request.post(`${BACKEND_URL}${endpoint}`, {
	data: review,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	success = true;
	break;
	}
	} catch (error) {
	// Try next endpoint
	continue;
	}
	}
	if (!success) {
	console.warn(`⚠️ Failed to create review for task ${review.task_id}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create review for task ${review.task_id}:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${reviews.length} review reports`);
	} else {
	console.log('⚠️ No review reports created (endpoint may not exist)');
	}
	}
	/**
	* NOTE: The following seed functions are kept for reference but not used.
	* Actual seeding is done via Python script (seedDatabaseDirectly).
	* These can be enabled if API endpoints become available.
	*/
	/**
	* Seed 5 agents with mixed statuses (working, idle, blocked).
	*/
	async function seedAgents(page: Page, projectId: number): Promise<void> {
	console.log('👥 Seeding agents...');
	const agents = [
	{
	id: 'lead-001',
	type: 'lead',
	status: 'working',
	provider: 'anthropic',
	maturity: 'delegating',
	current_task: { id: 1, title: 'Orchestrate project' },
	context_tokens: 25000,
	tasks_completed: 12,
	timestamp: Date.now()
	},
	{
	id: 'backend-worker-001',
	type: 'backend-worker',
	status: 'working',
	provider: 'anthropic',
	maturity: 'delegating',
	current_task: { id: 2, title: 'Implement API endpoints' },
	context_tokens: 45000,
	tasks_completed: 8,
	timestamp: Date.now()
	},
	{
	id: 'frontend-specialist-001',
	type: 'frontend-specialist',
	status: 'idle',
	provider: 'anthropic',
	maturity: 'supporting',
	context_tokens: 12000,
	tasks_completed: 5,
	timestamp: Date.now()
	},
	{
	id: 'test-engineer-001',
	type: 'test-engineer',
	status: 'working',
	provider: 'anthropic',
	maturity: 'delegating',
	current_task: { id: 3, title: 'Write E2E tests' },
	context_tokens: 30000,
	tasks_completed: 15,
	timestamp: Date.now()
	},
	{
	id: 'review-agent-001',
	type: 'review',
	status: 'blocked',
	provider: 'anthropic',
	maturity: 'delegating',
	blocker: 'Waiting for code review completion',
	context_tokens: 18000,
	tasks_completed: 20,
	timestamp: Date.now()
	}
	];
	let createdCount = 0;
	for (const agent of agents) {
	try {
	// Note: The backend may not have a direct POST /api/agents endpoint.
	// Agents are typically created internally by the system.
	// We'll try the endpoint, but expect it may not exist.
	const response = await page.request.post(`${BACKEND_URL}/api/agents`, {
	data: agent,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	} else {
	console.warn(`⚠️ Failed to create agent ${agent.id}: ${response.statusText()}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create agent ${agent.id}:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${agents.length} agents`);
	} else {
	console.log('⚠️ No agents created (endpoint may not exist or agents created internally)');
	}
	}
	/**
	* Seed 10 tasks with mixed statuses (completed, in_progress, blocked, pending).
	*/
	async function seedTasks(page: Page, projectId: number): Promise<void> {
	console.log('📋 Seeding tasks...');
	const tasks = [
	// Completed tasks
	{
	id: 1,
	project_id: projectId,
	title: 'Setup project structure',
	description: 'Initialize project repository and workspace',
	status: 'completed',
	assigned_to: 'lead-001',
	priority: 1,
	workflow_step: 1,
	timestamp: Date.now() - 86400000 * 2 // 2 days ago
	},
	{
	id: 2,
	project_id: projectId,
	title: 'Implement authentication API',
	description: 'Build JWT-based authentication endpoints',
	status: 'completed',
	assigned_to: 'backend-worker-001',
	priority: 1,
	workflow_step: 2,
	timestamp: Date.now() - 86400000 * 1 // 1 day ago
	},
	{
	id: 3,
	project_id: projectId,
	title: 'Write unit tests for auth',
	description: 'Comprehensive test coverage for authentication',
	status: 'completed',
	assigned_to: 'test-engineer-001',
	priority: 1,
	workflow_step: 3,
	timestamp: Date.now() - 43200000 // 12 hours ago
	},
	// In-progress tasks
	{
	id: 4,
	project_id: projectId,
	title: 'Build dashboard UI',
	description: 'Create React dashboard with real-time updates',
	status: 'in_progress',
	assigned_to: 'frontend-specialist-001',
	priority: 2,
	workflow_step: 4,
	timestamp: Date.now() - 7200000 // 2 hours ago
	},
	{
	id: 5,
	project_id: projectId,
	title: 'Add token usage tracking',
	description: 'Implement token counting and cost analytics',
	status: 'in_progress',
	assigned_to: 'backend-worker-001',
	priority: 2,
	workflow_step: 4,
	timestamp: Date.now() - 3600000 // 1 hour ago
	},
	// Blocked tasks
	{
	id: 6,
	project_id: projectId,
	title: 'Deploy to production',
	description: 'Set up production deployment pipeline',
	status: 'blocked',
	depends_on: '7,8',
	priority: 3,
	workflow_step: 6,
	timestamp: Date.now() - 1800000 // 30 minutes ago
	},
	{
	id: 7,
	project_id: projectId,
	title: 'Security audit',
	description: 'Comprehensive security review and penetration testing',
	status: 'blocked',
	depends_on: '4',
	priority: 2,
	workflow_step: 5,
	timestamp: Date.now() - 1800000 // 30 minutes ago
	},
	// Pending tasks
	{
	id: 8,
	project_id: projectId,
	title: 'Write API documentation',
	description: 'OpenAPI/Swagger documentation for all endpoints',
	status: 'pending',
	priority: 3,
	workflow_step: 5,
	timestamp: Date.now()
	},
	{
	id: 9,
	project_id: projectId,
	title: 'Optimize database queries',
	description: 'Add indexes and optimize slow queries',
	status: 'pending',
	priority: 2,
	workflow_step: 5,
	timestamp: Date.now()
	},
	{
	id: 10,
	project_id: projectId,
	title: 'Add logging middleware',
	description: 'Structured logging with request/response tracking',
	status: 'pending',
	priority: 2,
	workflow_step: 5,
	timestamp: Date.now()
	}
	];
	let createdCount = 0;
	for (const task of tasks) {
	try {
	const response = await page.request.post(`${BACKEND_URL}/api/tasks`, {
	data: task,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	} else {
	console.warn(`⚠️ Failed to create task ${task.id}: ${response.statusText()}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create task ${task.id}:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${tasks.length} tasks`);
	} else {
	console.log('⚠️ No tasks created (endpoint may not exist)');
	}
	}
	/**
	* Seed 15 token usage records across 3 models (Sonnet, Opus, Haiku) and 3 days.
	* Total cost: ~$4.46 USD
	*/
	async function seedTokenUsage(page: Page, projectId: number): Promise<void> {
	console.log('💰 Seeding token usage records...');
	const now = Date.now();
	const dayMs = 86400000; // 24 hours in milliseconds
	const tokenRecords = [
	// Backend agent usage (Sonnet)
	{
	task_id: 2,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 12500,
	output_tokens: 4800,
	estimated_cost_usd: 0.11,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2).toISOString()
	},
	{
	task_id: 2,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 8900,
	output_tokens: 3200,
	estimated_cost_usd: 0.075,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2 + 5400000).toISOString() // +1.5h
	},
	// Frontend agent usage (Haiku for smaller tasks)
	{
	task_id: 4,
	agent_id: 'frontend-specialist-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 5000,
	output_tokens: 2000,
	estimated_cost_usd: 0.012,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2 + 14400000).toISOString() // +4h
	},
	{
	task_id: 4,
	agent_id: 'frontend-specialist-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 6200,
	output_tokens: 2500,
	estimated_cost_usd: 0.015,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 1 + 3600000).toISOString() // Day 2 +1h
	},
	// Test engineer usage (Sonnet)
	{
	task_id: 3,
	agent_id: 'test-engineer-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 15000,
	output_tokens: 6000,
	estimated_cost_usd: 0.135,
	call_type: 'task_execution',
	timestamp: new Date(now - dayMs * 2 + 21600000).toISOString() // +6h
	},
	// Review agent usage (Opus for code review)
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 25000,
	output_tokens: 8000,
	estimated_cost_usd: 0.975,
	call_type: 'code_review',
	timestamp: new Date(now - dayMs * 1 + 10800000).toISOString() // Day 2 +3h
	},
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 18000,
	output_tokens: 5500,
	estimated_cost_usd: 0.6825,
	call_type: 'code_review',
	timestamp: new Date(now - dayMs * 1 + 18000000).toISOString() // Day 2 +5h
	},
	// Lead agent coordination (Sonnet)
	{
	agent_id: 'lead-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 8000,
	output_tokens: 3000,
	estimated_cost_usd: 0.069,
	call_type: 'coordination',
	timestamp: new Date(now - 3600000).toISOString() // Today -1h
	},
	// Additional records for time-series (Day 3 - today)
	{
	task_id: 5,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 10000,
	output_tokens: 4000,
	estimated_cost_usd: 0.09,
	call_type: 'task_execution',
	timestamp: new Date(now - 1800000).toISOString() // Today -30min
	},
	{
	task_id: 4,
	agent_id: 'frontend-specialist-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 7000,
	output_tokens: 2800,
	estimated_cost_usd: 0.017,
	call_type: 'task_execution',
	timestamp: new Date(now - 900000).toISOString() // Today -15min
	},
	// More Opus usage for higher costs
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 30000,
	output_tokens: 10000,
	estimated_cost_usd: 1.2,
	call_type: 'code_review',
	timestamp: new Date(now - 7200000).toISOString() // Today -2h
	},
	// Haiku for quick coordination
	{
	agent_id: 'lead-001',
	project_id: projectId,
	model_name: 'claude-haiku-4-20250929',
	input_tokens: 3000,
	output_tokens: 1200,
	estimated_cost_usd: 0.0072,
	call_type: 'coordination',
	timestamp: new Date(now - 5400000).toISOString() // Today -1.5h
	},
	// Additional Sonnet usage
	{
	task_id: 5,
	agent_id: 'backend-worker-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 14000,
	output_tokens: 5500,
	estimated_cost_usd: 0.1245,
	call_type: 'task_execution',
	timestamp: new Date(now - 10800000).toISOString() // Today -3h
	},
	{
	task_id: 3,
	agent_id: 'test-engineer-001',
	project_id: projectId,
	model_name: 'claude-sonnet-4-5-20250929',
	input_tokens: 11000,
	output_tokens: 4200,
	estimated_cost_usd: 0.096,
	call_type: 'task_execution',
	timestamp: new Date(now - 14400000).toISOString() // Today -4h
	},
	{
	agent_id: 'review-agent-001',
	project_id: projectId,
	model_name: 'claude-opus-4-20250929',
	input_tokens: 22000,
	output_tokens: 7000,
	estimated_cost_usd: 0.855,
	call_type: 'code_review',
	timestamp: new Date(now - 18000000).toISOString() // Today -5h
	}
	];
	let createdCount = 0;
	for (const record of tokenRecords) {
	try {
	// Try the most likely endpoints
	const endpoints = [
	`/api/projects/${projectId}/metrics/tokens`,
	`/api/token-usage`
	];
	let success = false;
	for (const endpoint of endpoints) {
	try {
	const response = await page.request.post(`${BACKEND_URL}${endpoint}`, {
	data: record,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	success = true;
	break;
	}
	} catch (error) {
	// Try next endpoint
	continue;
	}
	}
	if (!success) {
	console.warn(`⚠️ Failed to create token usage record for agent ${record.agent_id}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create token usage record:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${tokenRecords.length} token usage records (~$4.46 total)`);
	} else {
	console.log('⚠️ No token usage records created (endpoint may not exist)');
	}
	}
	/**
	* Seed 3 checkpoints with Git commit SHAs and metadata.
	*/
	async function seedCheckpoints(page: Page, projectId: number): Promise<void> {
	console.log('💾 Seeding checkpoints...');
	const now = Date.now();
	const dayMs = 86400000;
	const checkpoints = [
	{
	project_id: projectId,
	name: 'Initial setup complete',
	description: 'Project structure and authentication working',
	trigger: 'phase_transition',
	git_commit: 'a1b2c3d4e5f6',
	database_backup_path: '.codeframe/checkpoints/checkpoint-001-db.sqlite',
	context_snapshot_path: '.codeframe/checkpoints/checkpoint-001-context.json',
	metadata: {
	project_id: projectId,
	phase: 'setup',
	tasks_completed: 3,
	tasks_total: 10,
	agents_active: ['lead-001', 'backend-worker-001', 'test-engineer-001'],
	last_task_completed: 'Write unit tests for auth',
	context_items_count: 45,
	total_cost_usd: 1.2
	},
	created_at: new Date(now - dayMs * 2 + 64800000).toISOString() // 2 days ago + 18h
	},
	{
	project_id: projectId,
	name: 'UI development milestone',
	description: 'Dashboard UI 50% complete',
	trigger: 'manual',
	git_commit: 'f6e5d4c3b2a1',
	database_backup_path: '.codeframe/checkpoints/checkpoint-002-db.sqlite',
	context_snapshot_path: '.codeframe/checkpoints/checkpoint-002-context.json',
	metadata: {
	project_id: projectId,
	phase: 'ui-development',
	tasks_completed: 4,
	tasks_total: 10,
	agents_active: ['lead-001', 'frontend-specialist-001'],
	last_task_completed: 'Build dashboard UI',
	context_items_count: 78,
	total_cost_usd: 2.8
	},
	created_at: new Date(now - dayMs * 1 + 72000000).toISOString() // 1 day ago + 20h
	},
	{
	project_id: projectId,
	name: 'Pre-review snapshot',
	description: 'Before code review process',
	trigger: 'auto',
	git_commit: '9876543210ab',
	database_backup_path: '.codeframe/checkpoints/checkpoint-003-db.sqlite',
	context_snapshot_path: '.codeframe/checkpoints/checkpoint-003-context.json',
	metadata: {
	project_id: projectId,
	phase: 'review',
	tasks_completed: 5,
	tasks_total: 10,
	agents_active: ['lead-001', 'review-agent-001'],
	last_task_completed: 'Add token usage tracking',
	context_items_count: 120,
	total_cost_usd: 4.46
	},
	created_at: new Date(now - 3600000).toISOString() // Today -1h
	}
	];
	let createdCount = 0;
	for (const checkpoint of checkpoints) {
	try {
	const response = await page.request.post(
	`${BACKEND_URL}/api/projects/${projectId}/checkpoints`,
	{
	data: checkpoint,
	timeout: 10000
	}
	);
	if (response.ok()) {
	createdCount++;
	} else {
	console.warn(`⚠️ Failed to create checkpoint "${checkpoint.name}": ${response.statusText()}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create checkpoint "${checkpoint.name}":`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${checkpoints.length} checkpoints`);
	} else {
	console.log('⚠️ No checkpoints created (endpoint may not exist)');
	}
	}
	/**
	* Seed 2 review reports: 1 approved, 1 changes_requested.
	*/
	async function seedReviews(page: Page, projectId: number): Promise<void> {
	console.log('🔍 Seeding review reports...');
	const now = Date.now();
	const reviews = [
	{
	task_id: 2,
	reviewer_agent_id: 'review-agent-001',
	overall_score: 85,
	complexity_score: 80,
	security_score: 90,
	style_score: 85,
	status: 'approved',
	findings: [
	{
	file_path: 'codeframe/api/auth.py',
	line_number: 45,
	category: 'security',
	severity: 'medium',
	message: 'Consider adding rate limiting to login endpoint to prevent brute force attacks',
	suggestion: "Use FastAPI's limiter middleware with 5 requests per minute limit"
	},
	{
	file_path: 'codeframe/api/auth.py',
	line_number: 78,
	category: 'style',
	severity: 'low',
	message: "Function 'validate_token' exceeds 50 lines, consider extracting helper functions",
	suggestion: 'Extract JWT decoding logic into separate function'
	},
	{
	file_path: 'codeframe/api/auth.py',
	line_number: 120,
	category: 'coverage',
	severity: 'medium',
	message: 'Error handling path not covered by tests (line 120-125)',
	suggestion: 'Add test case for expired token scenario'
	}
	],
	summary: 'Good implementation overall. Authentication logic is solid with proper JWT handling. Main concerns are rate limiting and test coverage for error paths. Approved with suggested improvements.',
	created_at: new Date(now - 86400000 * 1 + 43200000).toISOString() // 1 day ago + 12h
	},
	{
	task_id: 4,
	reviewer_agent_id: 'review-agent-001',
	overall_score: 65,
	complexity_score: 60,
	security_score: 75,
	style_score: 70,
	status: 'changes_requested',
	findings: [
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 125,
	category: 'security',
	severity: 'critical',
	message: 'User input not sanitized before rendering, potential XSS vulnerability',
	suggestion: 'Use DOMPurify to sanitize user-generated content before rendering'
	},
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 200,
	category: 'complexity',
	severity: 'high',
	message: 'Component exceeds 300 lines, violating single responsibility principle',
	suggestion: 'Extract AgentStatusPanel, TaskList, and MetricsChart into separate components'
	},
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 45,
	category: 'style',
	severity: 'medium',
	message: 'useState hooks not grouped at top of component',
	suggestion: 'Move all useState declarations to top of component for better readability'
	},
	{
	file_path: 'web-ui/src/components/Dashboard.tsx',
	line_number: 180,
	category: 'owasp',
	severity: 'critical',
	message: 'Sensitive data (API tokens) logged to console in production build',
	suggestion: 'Remove console.log statements or gate with NODE_ENV check'
	}
	],
	summary: 'Component needs refactoring before approval. Critical security issues found: XSS vulnerability and token exposure in logs. Component is too complex (300+ lines) and violates separation of concerns. Please address critical findings before re-review.',
	created_at: new Date(now - 7200000).toISOString() // Today -2h
	}
	];
	let createdCount = 0;
	for (const review of reviews) {
	try {
	// Try multiple possible endpoints
	const endpoints = [
	`/api/reviews`,
	`/api/projects/${projectId}/reviews`,
	`/api/agents/${review.reviewer_agent_id}/review`
	];
	let success = false;
	for (const endpoint of endpoints) {
	try {
	const response = await page.request.post(`${BACKEND_URL}${endpoint}`, {
	data: review,
	timeout: 10000
	});
	if (response.ok()) {
	createdCount++;
	success = true;
	break;
	}
	} catch (error) {
	// Try next endpoint
	continue;
	}
	}
	if (!success) {
	console.warn(`⚠️ Failed to create review for task ${review.task_id}`);
	}
	} catch (error) {
	console.warn(`⚠️ Failed to create review for task ${review.task_id}:`, error);
	}
	}
	if (createdCount > 0) {
	console.log(`✅ Seeded ${createdCount}/${reviews.length} review reports`);
	} else {
	console.log('⚠️ No review reports created (endpoint may not exist)');
	}
	}

🤖 Prompt for AI Agents

In tests/e2e/global-setup.ts around lines 65-768 there are four seeding
functions (seedAgents, seedTasks, seedTokenUsage, seedReviews) defined but never
used; remove them if you want to drop unused code, otherwise wire them up as a
fallback by invoking them from the main setup flow when the primary Python
seeding fails (wrap the Python seed call in try/catch and call these JS seeders
on error), or retain them as documentation by adding a clear top-of-file comment
stating they are optional fallback helpers and why they remain; implement one of
these three actions to eliminate the dead-code warning.

[coderabbitai]

⚠️ Potential issue | 🟡 Minor

Remove unused import.

The pipeline reports pathlib.Path is imported but unused.

 from datetime import datetime, timedelta
-from pathlib import Path

🧰 Tools

🪛 GitHub Actions: Test Suite (Unit + E2E)

[error] 10-10: Ruff: F401 imported but unused ('pathlib.Path'). Remove unused import. Found 1 error. 1 fixable with the --fix option.

🤖 Prompt for AI Agents

In tests/e2e/seed-test-data.py around line 10, the import "from pathlib import
Path" is unused; remove this import line to eliminate the unused-import warning
and keep the file imports minimal. If Path was intended to be used later, either
use it where needed or replace with a direct alternative; otherwise delete the
import statement.

[claude]

Code Review: E2E Test Improvements (PR #38)

Overall Assessment: ✅ Approve with Minor Recommendations

This PR makes solid progress on E2E test infrastructure, improving the pass rate from 18% to 57%. The code quality is generally good.

🎯 Strengths

1. Backend API Fixes - Excellent Work ✅

Fixed field name mismatches in metrics_tracker.py (lines 276-293):

• Changed calls → call_count and tokens → total_tokens
• Now consistent with TypeScript interfaces
• Docstrings properly updated

2. Comprehensive Test Data Seeding ✅

• Well-structured seed-test-data.py script
• Proper error handling and table existence checks
• Critical addition: project-agent assignments for multi-agent architecture

3. UI Accessibility Improvements ✅

• Added comprehensive data-testid attributes to CheckpointList.tsx
• All interactive elements properly tagged

⚠️ Issues & Recommendations

1. Error Handling: Silent Failures (High Priority)

File: tests/e2e/seed-test-data.py:56-57

Individual insertion failures are logged but ignored. If critical data fails to seed, tests will run but fail mysteriously.

Recommend: Track failures and fail-fast if more than 50% fail.

2. Security: Input Validation (Medium Priority)

File: tests/e2e/seed-test-data.py

Add validation for project_id and db_path parameters.

3. Performance Optimization (Low Priority)

Use executemany for batch inserts instead of individual INSERT statements.
Expected 30-50% speed improvement.

📈 Impact Assessment

Metric	Before	After	Change
Test Pass Rate	18%	57%	+39% ✅
API Consistency	Broken	Fixed	✅
Multi-Agent Support	Missing	Present	✅

🎯 Verdict

✅ APPROVED - This PR delivers significant value:

• Fixes critical API contract issues
• Establishes robust E2E test infrastructure
• Enables multi-agent testing architecture
• Improves test pass rate by 39%

The identified issues are minor and can be addressed in follow-up PRs. Great work! 🚀

Reviewed by: Claude Code | Date: 2025-12-04

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

web-ui/src/api/checkpoints.ts (1)

13-13: Consistent API base URL definition

Switching to NEXT_PUBLIC_API_URL with the same fallback keeps this client aligned with the rest of the API layer and should help E2E runs use the correct backend URL.

You might eventually centralize API_BASE_URL into a shared utility to avoid drift across multiple API modules.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 2f6670e and 1cb7045.

📒 Files selected for processing (6)

• web-ui/src/api/checkpoints.ts (1 hunks)
• web-ui/src/api/context.ts (1 hunks)
• web-ui/src/api/metrics.ts (1 hunks)
• web-ui/src/api/qualityGates.ts (1 hunks)
• web-ui/src/api/review.ts (1 hunks)
• web-ui/src/api/reviews.ts (1 hunks)

🧰 Additional context used

📓 Path-based instructions (2)

web-ui/src/**/*.{ts,tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

web-ui/src/**/*.{ts,tsx}: Use TypeScript 5.3+ with React, strict mode, and maintain 85%+ test coverage for frontend code
Use React 18 with Tailwind CSS for frontend styling
Use Context + Reducer pattern (React Context with useReducer) for centralized state management in frontend

Files:

• web-ui/src/api/context.ts
• web-ui/src/api/reviews.ts
• web-ui/src/api/qualityGates.ts
• web-ui/src/api/review.ts
• web-ui/src/api/checkpoints.ts
• web-ui/src/api/metrics.ts

web-ui/**/*.{ts,tsx,test.ts,test.tsx}

📄 CodeRabbit inference engine (CLAUDE.md)

Run frontend tests with npm test from web-ui directory

Files:

• web-ui/src/api/context.ts
• web-ui/src/api/reviews.ts
• web-ui/src/api/qualityGates.ts
• web-ui/src/api/review.ts
• web-ui/src/api/checkpoints.ts
• web-ui/src/api/metrics.ts

🧠 Learnings (3)

📓 Common learnings

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to tests/**/*.py : Maintain 88%+ test coverage for Sprint 10 components and 100% pass rate

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to web-ui/src/**/*.{ts,tsx} : Use TypeScript 5.3+ with React, strict mode, and maintain 85%+ test coverage for frontend code

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to tests/e2e/**/*.py : Use TestSprite MCP for E2E test generation and Playwright for frontend E2E testing

📚 Learning: 2025-11-25T19:08:37.203Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: docs/CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:37.203Z
Learning: Applies to docs/web-ui/**/*.{ts,tsx} : Use Next.js 14 with React 18 App Router for the frontend

Applied to files:

• web-ui/src/api/context.ts
• web-ui/src/api/reviews.ts
• web-ui/src/api/checkpoints.ts
• web-ui/src/api/metrics.ts

📚 Learning: 2025-11-25T19:08:20.036Z

Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to web-ui/src/**/*.{ts,tsx} : Use Context + Reducer pattern (React Context with useReducer) for centralized state management in frontend

Applied to files:

• web-ui/src/api/context.ts

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: claude-review

🔇 Additional comments (5)

web-ui/src/api/review.ts (1)

17-17: Base URL env var switch looks good

Using process.env.NEXT_PUBLIC_API_URL with the same localhost fallback is consistent with other API modules and Next.js public env conventions; no issues from this change alone.

web-ui/src/api/metrics.ts (1)

18-18: Metrics client now aligned with shared base URL pattern

The updated API_BASE_URL definition matches other API modules and preserves the same fallback; no additional issues spotted here.

web-ui/src/api/qualityGates.ts (1)

16-16: Quality gates API using shared NEXT_PUBLIC_API_URL

The base URL now follows the same NEXT_PUBLIC_API_URL convention as other API clients, with the existing localhost default preserved; this looks correct.

web-ui/src/api/context.ts (1)

17-17: Context API base URL change is consistent with Next.js setup

Using NEXT_PUBLIC_API_URL here keeps the context API client in line with the rest of the web‑ui and Next.js public env naming, with identical behavior when the var is unset.

web-ui/src/api/reviews.ts (1)

12-12: Review(s) API now using shared NEXT_PUBLIC_API_URL

The base URL definition is now consistent with other API clients and keeps the same default; no functional concerns from this update.

[claude]

Code Review - PR #38: E2E Test Improvements

Summary

This PR makes meaningful progress on E2E test stability, improving pass rate from 18% to 57% (+39%). The approach is solid: fixing field name mismatches between backend and frontend, adding missing test IDs, and seeding comprehensive test data.

---

✅ Strengths

1. Excellent Test Data Seeding Strategy

The seed-test-data.py script is well-designed:

• ✅ Comprehensive coverage: agents, tasks, token usage, reviews, checkpoints
• ✅ Realistic data distribution across multiple days for time-series testing
• ✅ Proper error handling with graceful degradation
• ✅ Good separation of concerns (Python for DB, TypeScript for orchestration)
• ✅ Critical fix: Added project-agent assignments (lines 64-94) - essential for multi-agent architecture

2. API Field Name Fixes

The metrics_tracker.py changes correctly align with TypeScript interfaces:

• ✅ calls → call_count (more descriptive)
• ✅ tokens → total_tokens (clearer aggregation intent)
• ✅ Updated docstrings to match new field names
• ✅ Consistent naming across agent and model breakdowns

3. Frontend Improvements

CheckpointList.tsx and CostDashboard.tsx:

• ✅ Added 20+ data-testid attributes for Playwright selectors
• ✅ Proper validation error handling with inline error messages
• ✅ Good UX: disabled button state when name is empty

4. Environment Variable Fix

Switching from REACT_APP_API_URL to NEXT_PUBLIC_API_URL across 6 API client files:

• ✅ Correct Next.js convention
• ✅ Should fix 404 errors in dashboard components
• ✅ Consistent application across all API clients

---

⚠️ Issues & Concerns

1. Security: Hardcoded Model Names

Location: seed-test-data.py lines 175-194

The model names were changed from versioned (claude-sonnet-4-5-20250929) to canonical (claude-sonnet-4-5), but this creates a maintenance risk:

# Hard to maintain if pricing dictionary changes
(1, 2, 'backend-worker-001', project_id, 'claude-sonnet-4-5', ...)

Recommendation: Define model names as constants at the top of the file:

# Model name constants (must match metrics_tracker.py pricing dictionary)
MODEL_SONNET = 'claude-sonnet-4-5'
MODEL_OPUS = 'claude-opus-4'
MODEL_HAIKU = 'claude-haiku-4'

# Usage:
(1, 2, 'backend-worker-001', project_id, MODEL_SONNET, ...)

2. Database Schema Assumptions

Location: seed-test-data.py lines 42-48, 68-70, 144-146

The script checks if tables exist but doesn't validate column schemas:

cursor.execute("SELECT name FROM sqlite_master WHERE type='table' AND name='agents'")
if not cursor.fetchone():
    print("⚠️  Warning: agents table doesn't exist, skipping agents")

Risk: If table exists but columns have changed (e.g., renamed, reordered), inserts will fail silently.

Recommendation: Add column validation or use INSERT OR REPLACE with explicit column names (already doing the latter, which is good).

3. Incomplete Test Coverage in PR Description

The PR states:

Metrics Tests (7/12 failing): CostDashboard component not rendering

Question: After the NEXT_PUBLIC_API_URL fix, were metrics tests re-run? The PR description suggests these may now pass, but this isn't confirmed.

Recommendation:

• Re-run full test suite after all changes
• Update PR description with final pass rates
• Document which specific tests are still failing

4. Validation Error Display Logic

Location: CheckpointList.tsx lines 60-64, 187-191

The validation logic clears the error when typing, but the error persists if user types then deletes:

onChange={(e) => {
  setNewCheckpointName(e.target.value);
  if (e.target.value.trim()) {  // Only clears if value is non-empty
    setNameError(null);
  }
}}

Edge case: User enters name → error clears → user deletes name → error doesn't re-appear until "Create" is clicked.

Recommendation: Either:

• Accept current behavior (validation on submit is fine)
• Add real-time validation: setNameError(e.target.value.trim() ? null : 'Required')

5. Missing Type Safety in Token Calculation

Location: seed-test-data.py lines 173-195

The estimated cost values are hardcoded without validation:

(1, 2, 'backend-worker-001', project_id, 'claude-sonnet-4-5', 12500, 4800, 0.11, ...)

Risk: If someone updates input/output tokens but forgets to recalculate cost, data becomes inconsistent.

Recommendation: Calculate costs programmatically:

def calculate_cost(model: str, input_tokens: int, output_tokens: int) -> float:
    pricing = {
        'claude-sonnet-4-5': {'input': 3.00, 'output': 15.00},
        'claude-opus-4': {'input': 15.00, 'output': 75.00},
        'claude-haiku-4': {'input': 0.80, 'output': 4.00}
    }
    p = pricing[model]
    return (input_tokens * p['input'] + output_tokens * p['output']) / 1_000_000

# Usage:
cost = calculate_cost('claude-sonnet-4-5', 12500, 4800)
(1, 2, 'backend-worker-001', project_id, 'claude-sonnet-4-5', 12500, 4800, cost, ...)

---

🎯 Best Practices Observed

1. Proper error handling: All database operations wrapped in try/except
2. Graceful degradation: Script continues even if some tables don't exist
3. Clear progress feedback: Print statements with emojis for readability
4. Consistent naming: Following TypeScript interface conventions
5. Good documentation: PR description is detailed and includes test results

---

🔒 Security Notes

No security vulnerabilities detected in this PR. The changes are test-infrastructure focused and don't introduce:

• SQL injection risks (using parameterized queries ✅)
• XSS vulnerabilities (no unsanitized user input rendering)
• Authentication bypasses
• Credential leaks

---

📊 Performance Considerations

1. Database seeding performance: The current approach seeds ~40 records synchronously

   • ✅ Fast enough for E2E tests (<2 seconds)
   • ⚠️ Consider batch inserts if seeding grows >100 records

2. API URL defaulting: NEXT_PUBLIC_API_URL || 'http://localhost:8000'

   • ⚠️ Wrong default port (should be 8080 per .env.local)
   • Recommendation: Update default to 8080 or throw error if env var missing

---

📝 Recommendations Summary

Must Fix (Blocking)

1. ✅ None - PR is safe to merge as-is

Should Fix (Follow-up PR)

1. Extract model names to constants in seed-test-data.py
2. Update API URL default from 8000 to 8080 in all API client files
3. Add programmatic cost calculation instead of hardcoded values
4. Re-run full test suite and update PR description with final results

Nice to Have

1. Add column schema validation in seeding script
2. Improve real-time validation UX in CheckpointList component

---

✅ Recommendation

APPROVE with suggestions for follow-up improvements.

This PR makes solid progress toward test stability (18% → 57%) and fixes critical issues:

• ✅ Field name mismatches resolved
• ✅ Environment variable conventions corrected
• ✅ Project-agent assignments now seeded
• ✅ Comprehensive test data in place

The remaining 43% of failing tests appear to be component-specific rendering issues that require debugging with browser dev tools (as noted in PR description). This PR establishes a strong foundation for achieving 90-100% pass rate.

Merge when ready and continue with Phase 5 debugging in a follow-up PR.

---

Reviewed by: Claude Code
Review Date: 2025-12-04
Files Reviewed: 13 files (+1631 -140 lines)