[Phase 5] Add performance and load testing benchmarks

[frankbria]

Overview

No performance benchmarks or load tests exist, making it impossible to detect regressions or plan capacity.

Missing Performance Tests

Agent Operations

• Task execution latency (target: <30s per task)
• Context loading time (target: <200ms for 1000 items)
• Flash save duration (target: <2s)
• Checkpoint creation (target: <10s)

Database Operations

• Query performance with 10,000+ tasks
• Concurrent read/write performance
• Index efficiency
• Migration time for large databases

API Endpoints

• Response time under load (100 req/s)
• WebSocket broadcast latency (target: <100ms)
• Concurrent client handling (1000+ connections)
• Memory usage under sustained load

Quality Gates

• Lint analysis time (target: <2min)
• Test execution overhead (target: <10% of test time)
• Coverage calculation time
• Review agent analysis (target: <30s per file)

Implementation Requirements

1. Use pytest-benchmark for Python performance tests
2. Use locust or k6 for load testing
3. Set performance baselines
4. Detect regressions automatically in CI
5. Generate performance reports

Example Benchmark Test

# Test: context loading performance
def test_context_load_performance(benchmark):
    db = Database(":memory:")
    
    # Create 1000 context items
    for i in range(1000):
        db.save_context_item(ContextItem(...))
    
    # Benchmark loading
    result = benchmark(db.load_context, agent_id="test-001")
    
    assert result.stats.mean < 0.2  # <200ms
    assert result.stats.max < 0.5   # <500ms worst case

Performance Targets

Operation	Target	Critical Threshold
Task execution	<30s	<60s
Context load (1000 items)	<200ms	<500ms
Flash save	<2s	<5s
Checkpoint create	<10s	<30s
Checkpoint restore	<30s	<60s
API response (95th %ile)	<500ms	<2s
WebSocket broadcast	<100ms	<500ms
Review agent analysis	<30s/file	<2min/file

Acceptance Criteria

• Performance benchmarks for all core operations
• Load tests for API and database
• Baselines established for regression detection
• CI fails on performance regressions >20%
• Performance reports generated per commit
• Dashboard shows performance trends

Priority

P2 - Essential: Performance issues become critical at scale.

References

• Issues Analysis: CODEFRAME_ISSUES_ANALYSIS.md lines 184
• Performance characteristics: CLAUDE.md lines 165-174

[traycerai]

Plan

Observations

The codebase has a comprehensive test infrastructure with 111+ tests organized in the tests/ directory, using pytest with pytest-asyncio. However, no performance benchmarks or load tests exist. The system uses a repository pattern with 17 domain-specific repositories, FastAPI for the API layer, and has well-defined operations for context management, checkpoints, quality gates, and agent operations. The CI/CD pipeline (file:.github/workflows/test.yml) runs functional tests but lacks performance regression detection. Key operations like context loading, flash saves, checkpoint creation, database queries, API endpoints, and quality gate execution are prime candidates for performance benchmarking.

Approach

This is a Phase 5 enhancement that establishes a phased, modular performance testing infrastructure. The implementation follows a four-tier benchmarking strategy: (1) Unit-level performance tests using pytest-benchmark for core operations (context loading, database queries, checkpoint operations), (2) Integration-level load tests using locust for API endpoints and WebSocket broadcasts, (3) Quality gate benchmarks for orchestration and analysis overhead, and (4) CI integration with automated regression detection. This approach aligns with the ticket's requirements by establishing baselines, detecting regressions automatically, and generating performance reports. The plan prioritizes operations by criticality (agent operations → database → API → quality gates) and sets up infrastructure before implementing specific benchmarks. Each phase can be implemented independently and integrated into the development workflow as resources allow.

---

Phase 5: Performance and Load Testing Benchmarks

This phase establishes comprehensive performance testing infrastructure to detect regressions, establish baselines, and plan capacity for the codeframe system.

Implementation Steps

1. Add Performance Testing Dependencies

Objective: Install pytest-benchmark and locust for performance and load testing.

Actions:

• Update file:pyproject.toml to add performance testing dependencies:
  • Add pytest-benchmark>=4.0.0 to [project.optional-dependencies].perf section (new optional group)
  • Add locust>=2.20.0 to [project.optional-dependencies].perf section
  • Add pytest-timeout>=2.2.0 to [project.optional-dependencies].perf section
  • This allows developers to opt-in to performance testing without bloating the base dependencies
• Run uv sync --extra perf to install performance testing dependencies
• Verify installation by running pytest --benchmark-help and locust --version
• Document in file:TESTING.md how to install and use performance testing tools

Files to modify:

• file:pyproject.toml
• file:TESTING.md

---

2. Create Performance Test Infrastructure

Objective: Set up directory structure and base fixtures for performance tests.

Actions:

• Create new directory tests/performance/ for all performance tests
• Create tests/performance/conftest.py with shared fixtures:
  • benchmark_db fixture: In-memory database with pre-populated data (1000 context items, 100 tasks, 10 agents)
  • benchmark_project_root fixture: Temporary project directory for file-based operations
  • benchmark_context_items fixture: Factory for generating test context items
  • performance_baseline fixture: Loads baseline metrics from JSON file for regression detection
  • benchmark_skip_if_disabled fixture: Allows skipping performance tests if --benchmark-disable flag is used
• Create tests/performance/baselines.json to store performance baselines (initially empty, populated after first run)
• Create tests/performance/README.md documenting:
  • How to run performance tests: pytest tests/performance/ --benchmark-only
  • How to update baselines: pytest tests/performance/ --benchmark-save=baseline
  • How to compare against baselines: pytest tests/performance/ --benchmark-compare=baseline
  • How to skip performance tests during regular test runs: pytest --benchmark-disable
  • Performance testing is optional and can be run separately from functional tests

New files:

• tests/performance/conftest.py
• tests/performance/baselines.json
• tests/performance/README.md

---

3. Implement Agent Operation Benchmarks (Phase 1)

Objective: Benchmark task execution, context loading, flash save, and checkpoint operations.

Actions:

• Create tests/performance/test_agent_operations.py with benchmark tests:

  Test: Context Loading Performance (Target: <200ms for 1000 items)

  • Use ContextManager.recalculate_scores_for_agent() from file:codeframe/lib/context_manager.py
  • Pre-populate database with 1000 context items across HOT/WARM/COLD tiers
  • Benchmark loading and score recalculation
  • Assert mean < 200ms, max < 500ms

  Test: Flash Save Duration (Target: <2s)

  • Use ContextManager.flash_save() from file:codeframe/lib/context_manager.py
  • Pre-populate with 1000 context items (300 COLD items to archive)
  • Benchmark full flash save workflow (checkpoint creation + archival)
  • Assert mean < 2s, max < 5s

  Test: Checkpoint Creation (Target: <10s)

  • Use CheckpointManager.create_checkpoint() from file:codeframe/lib/checkpoint_manager.py
  • Set up git repository with 50 files, database with 100 tasks
  • Benchmark checkpoint creation (git commit + DB backup + context snapshot)
  • Assert mean < 10s, max < 30s

  Test: Checkpoint Restoration (Target: <30s)

  • Use CheckpointManager.restore_checkpoint() from file:codeframe/lib/checkpoint_manager.py
  • Pre-create checkpoint with 100 tasks, 1000 context items
  • Benchmark full restoration workflow
  • Assert mean < 30s, max < 60s

  Note: This phase establishes the foundation for performance testing. Subsequent phases (database, API, quality gates) can be implemented independently as needed.

New file:

• tests/performance/test_agent_operations.py

Reference implementations:

• file:codeframe/lib/context_manager.py (lines 38-286)
• file:codeframe/lib/checkpoint_manager.py (lines 54-649)

---

4. Implement Database Operation Benchmarks (Phase 2)

Objective: Benchmark database queries with large datasets and concurrent operations.

Actions:

• Create tests/performance/test_database_operations.py with benchmark tests:

  Test: Query Performance with 10,000+ Tasks

  • Use TaskRepository methods from file:codeframe/persistence/database.py
  • Pre-populate database with 10,000 tasks across multiple projects
  • Benchmark queries:
    • get_pending_tasks() with filters
    • get_tasks_by_issue() with pagination
    • get_recently_completed_tasks() with date ranges
  • Assert query time < 100ms for indexed queries, < 500ms for complex joins

  Test: Concurrent Read/Write Performance

  • Use async database operations from file:codeframe/persistence/database.py
  • Simulate 10 concurrent agents reading/writing context items
  • Use asyncio.gather() to run operations in parallel
  • Benchmark throughput (operations/second)
  • Assert no deadlocks, consistent results

  Test: Context Item Queries (1000+ items)

  • Use ContextRepository.list_context_items() from file:codeframe/persistence/database.py
  • Pre-populate with 10,000 context items across 10 agents
  • Benchmark filtered queries by tier, agent_id, importance_score
  • Assert query time < 50ms with proper indexes

  Test: Checkpoint Queries

  • Use CheckpointRepository methods from file:codeframe/persistence/database.py
  • Pre-populate with 100 checkpoints
  • Benchmark get_checkpoints() and get_checkpoint_by_id()
  • Assert query time < 20ms

  Note: This phase can be implemented after Phase 1 is complete and integrated. Database performance is critical for scale, so this phase should be prioritized after agent operations are established.

New file:

• tests/performance/test_database_operations.py

Reference implementations:

• file:codeframe/persistence/database.py (lines 260-677)
• Repository pattern documentation: docs/architecture/database-repository-pattern.md

---

5. Implement API Endpoint Load Tests (Phase 3)

Objective: Load test API endpoints and WebSocket broadcasts using locust.

Actions:

• Create tests/performance/locustfile.py for load testing:

  User Class: APIUser

  • Simulates typical API usage patterns
  • Tasks:
    • GET /api/projects (weight: 10)
    • GET /api/agents (weight: 8)
    • GET /api/tasks?project_id=1 (weight: 15)
    • GET /api/context?agent_id=test-001 (weight: 12)
    • POST /api/checkpoints (weight: 2)
    • GET /health (weight: 5)
  • Wait time: 1-3 seconds between requests

  User Class: WebSocketUser

  • Simulates WebSocket connections for real-time updates
  • Connects to /ws endpoint from file:codeframe/ui/routers/websocket.py
  • Subscribes to project updates
  • Measures broadcast latency (time from server event to client receipt)
  • Target: <100ms broadcast latency

  Load Test Scenarios:

  • Scenario 1: 100 concurrent users, 10 req/s sustained load (5 minutes)
  • Scenario 2: 1000 concurrent WebSocket connections
  • Scenario 3: Spike test (0→500 users in 30s, hold 2 min, ramp down)

• Create tests/performance/run_load_tests.sh script:

  • Starts FastAPI server in background
  • Waits for server health check
  • Runs locust in headless mode with different scenarios
  • Captures results to tests/performance/load_test_results/
  • Stops server after tests

• Create tests/performance/test_api_performance.py for pytest-based API benchmarks:

  • Use TestClient from FastAPI for synchronous benchmarks
  • Benchmark individual endpoint response times
  • Assert 95th percentile < 500ms, 99th percentile < 2s

  Note: This phase focuses on API and WebSocket performance. Can be implemented after Phase 1 and 2 are established, or in parallel if API performance is a priority.

New files:

• tests/performance/locustfile.py
• tests/performance/run_load_tests.sh
• tests/performance/test_api_performance.py

Reference implementations:

• file:codeframe/ui/server.py (lines 1-371)
• file:codeframe/ui/routers/websocket.py

---

6. Implement Quality Gate Benchmarks (Phase 4)

Objective: Benchmark quality gate execution times.

Actions:

• Create tests/performance/test_quality_gates.py with benchmark tests:

  Test: Lint Analysis Time (Target: <2min)

  • Use QualityGates.run_linting_gate() from file:codeframe/lib/quality_gates.py
  • Set up test project with 100 Python files (10,000 LOC)
  • Benchmark ruff execution time
  • Assert mean < 120s

  Test: Test Execution Overhead (Target: <10% of test time)

  • Use TestRunner.run_tests() from file:codeframe/testing/test_runner.py
  • Measure baseline test execution time (direct pytest)
  • Measure test execution through TestRunner
  • Calculate overhead percentage
  • Assert overhead < 10%

  Test: Coverage Calculation Time

  • Use QualityGates.run_coverage_gate() from file:codeframe/lib/quality_gates.py
  • Benchmark pytest with coverage on 100-file project
  • Assert mean < 180s (3 minutes)

  Test: Review Agent Analysis (Target: <30s per file)

  • Use QualityGates.run_review_gate() from file:codeframe/lib/quality_gates.py
  • Mock Review Agent to avoid LLM calls (focus on orchestration overhead)
  • Benchmark review workflow for 10 files
  • Assert mean < 30s per file

  Test: All Gates Orchestration

  • Use QualityGates.run_all_gates() from file:codeframe/lib/quality_gates.py
  • Benchmark full quality gate pipeline
  • Measure individual gate times and total time
  • Assert total time < 5 minutes for typical task

  Note: This phase is optional and can be deferred if quality gate performance is not a current bottleneck. Implement after Phases 1-3 are complete.

New file:

• tests/performance/test_quality_gates.py

Reference implementations:

• file:codeframe/lib/quality_gates.py (lines 107-1085)
• file:codeframe/testing/test_runner.py (lines 1-187)

---

7. Establish Performance Baselines

Objective: Run all benchmarks to establish initial baselines.

Actions:

• After each phase is implemented, run performance tests with baseline capture:

  pytest tests/performance/ --benchmark-only --benchmark-save=baseline --benchmark-json=tests/performance/baseline_results.json

• Review baseline results and adjust targets if needed
• Document baseline environment:
  • Hardware specs (CPU, RAM, disk type)
  • Python version
  • Database size
  • OS and kernel version
• Store baselines in tests/performance/baselines.json with metadata:

  {
    "environment": {
      "python_version": "3.11.x",
      "cpu": "...",
      "ram": "...",
      "date": "2025-12-26"
    },
    "baselines": {
      "context_load_1000_items": {"mean": 0.15, "max": 0.35},
      "flash_save": {"mean": 1.5, "max": 3.2},
      ...
    }
  }

• Create tests/performance/BASELINES.md documenting:
  • Baseline values for each operation
  • Target thresholds (from ticket)
  • Critical thresholds (20% above target)
  • Instructions for updating baselines
• Baselines should be committed to version control to track performance over time

New files:

• tests/performance/baseline_results.json
• tests/performance/BASELINES.md

---

8. Integrate Performance Tests into CI (Optional)

Objective: Add performance regression detection to GitHub Actions workflow.

Actions:

• Update file:.github/workflows/test.yml to add optional performance testing job:

  performance-tests:
    name: Performance Benchmarks
    runs-on: ubuntu-latest
    needs: backend-tests
    # Only run on main branch or when performance tests are modified
    if: github.ref == 'refs/heads/main' || contains(github.event.head_commit.modified, 'tests/performance/')
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
      
      - name: Install uv
        uses: astral-sh/setup-uv@v4
        with:
          enable-cache: true
      
      - name: Install dependencies
        run: |
          uv venv
          uv sync --extra perf
      
      - name: Run performance benchmarks
        run: |
          uv run pytest tests/performance/ \
            --benchmark-only \
            --benchmark-compare=baseline \
            --benchmark-compare-fail=mean:20% \
            --benchmark-json=performance_results.json
      
      - name: Check for performance regressions
        run: |
          # Fail if any benchmark regressed by >20%
          python scripts/check_performance_regression.py performance_results.json
      
      - name: Upload performance results
        uses: actions/upload-artifact@v4
        with:
          name: performance-results
          path: performance_results.json
          retention-days: 90
      
      - name: Comment PR with performance results
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('performance_results.json'));
            // Format and post comment with performance comparison

• Create scripts/check_performance_regression.py:

  • Parses pytest-benchmark JSON output
  • Compares against baselines
  • Fails CI if any benchmark regressed >20%
  • Generates markdown report for PR comments

• Update pytest.ini to add performance test markers:

  markers =
      performance: marks tests as performance benchmarks
      load_test: marks tests as load tests (requires running server)

• Note: This CI integration is optional and can be deferred. Performance tests can be run manually during development and on-demand in CI until the infrastructure is fully mature.

Files to modify:

• file:.github/workflows/test.yml
• file:pytest.ini

New files:

• scripts/check_performance_regression.py

---

9. Create Performance Dashboard (Optional)

Objective: Generate HTML reports showing performance trends over time.

Actions:

• Create scripts/generate_performance_report.py:

  • Reads historical performance results from CI artifacts
  • Generates HTML dashboard with charts:
    • Line charts showing performance trends over time
    • Bar charts comparing current vs baseline
    • Heatmap showing which operations are slowest
    • Table with all benchmark results and thresholds
  • Uses matplotlib or plotly for visualizations
  • Outputs to tests/performance/reports/index.html

• Create tests/performance/reports/.gitkeep to track directory

• Add GitHub Actions step to publish performance reports:

  - name: Generate performance report
    run: python scripts/generate_performance_report.py
  
  - name: Deploy to GitHub Pages
    if: github.ref == 'refs/heads/main'
    uses: peaceiris/actions-gh-pages@v3
    with:
      github_token: ${{ secrets.GITHUB_TOKEN }}
      publish_dir: tests/performance/reports

• Note: This dashboard is a nice-to-have feature that can be added after the core performance testing infrastructure is established and baselines are being tracked.

New files:

• scripts/generate_performance_report.py
• tests/performance/reports/.gitkeep

---

10. Document Performance Testing

Objective: Create comprehensive documentation for performance testing.

Actions:

• Update file:TESTING.md to add "Performance Testing" section:

  • Overview of performance testing strategy and phased approach
  • How to run performance tests locally: pytest tests/performance/ --benchmark-only
  • How to skip performance tests during regular runs: pytest --benchmark-disable
  • How to interpret benchmark results
  • How to update baselines
  • Performance targets and thresholds
  • Link to performance dashboard (when available)
  • Note that performance testing is optional and can be installed with uv sync --extra perf

• Create docs/performance-testing.md with detailed guide:

  • Architecture of performance testing infrastructure
  • Phased implementation approach (Phase 1-4)
  • Benchmark test patterns and best practices
  • Load testing scenarios and how to add new ones
  • CI integration and regression detection (optional)
  • Troubleshooting performance issues
  • Performance optimization tips
  • How to contribute new performance tests

• Add performance testing to file:README.md:

  ## Performance Testing (Optional)
  
  Install performance testing tools:
  ```bash
  uv sync --extra perf

  Run performance benchmarks:

  pytest tests/performance/ --benchmark-only

  Run load tests:

  ./tests/performance/run_load_tests.sh

  See Performance Testing Guide for more details.

Files to modify:

• file:TESTING.md
• file:README.md

New files:

• docs/performance-testing.md

---

Performance Targets Summary

Operation	Target	Critical Threshold	Test Location	Phase
Task execution	<30s	<60s	test_agent_operations.py	1
Context load (1000 items)	<200ms	<500ms	test_agent_operations.py	1
Flash save	<2s	<5s	test_agent_operations.py	1
Checkpoint create	<10s	<30s	test_agent_operations.py	1
Checkpoint restore	<30s	<60s	test_agent_operations.py	1
Database query (10k tasks)	<100ms	<500ms	test_database_operations.py	2
Concurrent read/write	N/A	N/A	test_database_operations.py	2
API response (95th %ile)	<500ms	<2s	test_api_performance.py	3
WebSocket broadcast	<100ms	<500ms	locustfile.py	3
Lint analysis	<2min	<5min	test_quality_gates.py	4
Test execution overhead	<10%	<20%	test_quality_gates.py	4
Review agent analysis	<30s/file	<2min/file	test_quality_gates.py	4

---

Phased Implementation Approach (Phase 5)

Phase 5.1 (Foundation): Agent operation benchmarks establish the core performance testing infrastructure.

Phase 5.2 (Database): Database operation benchmarks ensure query performance at scale.

Phase 5.3 (API): API and WebSocket load tests validate endpoint performance under concurrent load.

Phase 5.4 (Quality Gates): Quality gate benchmarks measure orchestration and analysis overhead (optional, can be deferred).

Phase 5.5 (CI Integration & Dashboard): Optional enhancements that can be added after core phases are complete.

---

Architecture Diagram

graph TD
    A[Phase 5: Performance Testing Infrastructure] --> B[Phase 5.1: Agent Operations]
    A --> C[Phase 5.2: Database Operations]
    A --> D[Phase 5.3: API Load Tests]
    A --> E[Phase 5.4: Quality Gates]
    
    B --> B1[Context Loading<br/>Flash Save<br/>Checkpoints]
    C --> C1[Query Performance<br/>Concurrent Ops<br/>Index Efficiency]
    D --> D1[API Endpoints<br/>WebSocket Broadcasts<br/>Concurrent Clients]
    E --> E1[Lint Analysis<br/>Test Overhead<br/>Review Agent]
    
    B1 --> F[Baseline Storage<br/>baselines.json]
    C1 --> F
    D1 --> F
    E1 --> F
    
    F --> G[Phase 5.5: CI Integration]
    F --> H[Phase 5.5: Performance Dashboard]
    
    G --> I[Regression Detection<br/>PR Comments]
    H --> J[Trend Visualization<br/>GitHub Pages]
    
    style A fill:#e1f5ff
    style B fill:#fff4e1
    style C fill:#fff4e1
    style D fill:#ffe1f5
    style E fill:#ffe1f5
    style G fill:#e1ffe1
    style H fill:#e1ffe1

Loading

---

Acceptance Criteria Checklist (Phase 5)

• Phase 5.1: Performance benchmarks for agent operations (context loading, flash save, checkpoints)
• Phase 5.2: Performance benchmarks for database operations (queries, concurrent access)
• Phase 5.3: Load tests for API endpoints and WebSocket broadcasts
• Phase 5.4 (Optional): Performance benchmarks for quality gates
• Baselines established for regression detection (stored in baselines.json)
• Phase 5.5: CI integration with regression detection (>20% threshold)
• Phase 5.5: Performance dashboard with trend visualization
• Documentation updated (TESTING.md, README.md, performance-testing.md)
• Performance testing infrastructure is modular and can be extended

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a comprehensive test infrastructure with 111+ tests organized in the `tests/` directory, using pytest with pytest-asyncio. However, **no performance benchmarks or load tests exist**. The system uses a repository pattern with 17 domain-specific repositories, FastAPI for the API layer, and has well-defined operations for context management, checkpoints, quality gates, and agent operations. The CI/CD pipeline (file:.github/workflows/test.yml) runs functional tests but lacks performance regression detection. Key operations like context loading, flash saves, checkpoint creation, database queries, API endpoints, and quality gate execution are prime candidates for performance benchmarking.

## Approach

This is a **Phase 5 enhancement** that establishes a phased, modular performance testing infrastructure. The implementation follows a **four-tier benchmarking strategy**: (1) **Unit-level performance tests** using pytest-benchmark for core operations (context loading, database queries, checkpoint operations), (2) **Integration-level load tests** using locust for API endpoints and WebSocket broadcasts, (3) **Quality gate benchmarks** for orchestration and analysis overhead, and (4) **CI integration** with automated regression detection. This approach aligns with the ticket's requirements by establishing baselines, detecting regressions automatically, and generating performance reports. The plan prioritizes operations by criticality (agent operations → database → API → quality gates) and sets up infrastructure before implementing specific benchmarks. Each phase can be implemented independently and integrated into the development workflow as resources allow.

---

## Phase 5: Performance and Load Testing Benchmarks

This phase establishes comprehensive performance testing infrastructure to detect regressions, establish baselines, and plan capacity for the codeframe system.

## Implementation Steps

### 1. Add Performance Testing Dependencies

**Objective:** Install pytest-benchmark and locust for performance and load testing.

**Actions:**
- Update file:pyproject.toml to add performance testing dependencies:
  - Add `pytest-benchmark>=4.0.0` to `[project.optional-dependencies].perf` section (new optional group)
  - Add `locust>=2.20.0` to `[project.optional-dependencies].perf` section
  - Add `pytest-timeout>=2.2.0` to `[project.optional-dependencies].perf` section
  - This allows developers to opt-in to performance testing without bloating the base dependencies
- Run `uv sync --extra perf` to install performance testing dependencies
- Verify installation by running `pytest --benchmark-help` and `locust --version`
- Document in file:TESTING.md how to install and use performance testing tools

**Files to modify:**
- file:pyproject.toml
- file:TESTING.md

---

### 2. Create Performance Test Infrastructure

**Objective:** Set up directory structure and base fixtures for performance tests.

**Actions:**
- Create new directory `tests/performance/` for all performance tests
- Create `tests/performance/conftest.py` with shared fixtures:
  - `benchmark_db` fixture: In-memory database with pre-populated data (1000 context items, 100 tasks, 10 agents)
  - `benchmark_project_root` fixture: Temporary project directory for file-based operations
  - `benchmark_context_items` fixture: Factory for generating test context items
  - `performance_baseline` fixture: Loads baseline metrics from JSON file for regression detection
  - `benchmark_skip_if_disabled` fixture: Allows skipping performance tests if `--benchmark-disable` flag is used
- Create `tests/performance/baselines.json` to store performance baselines (initially empty, populated after first run)
- Create `tests/performance/README.md` documenting:
  - How to run performance tests: `pytest tests/performance/ --benchmark-only`
  - How to update baselines: `pytest tests/performance/ --benchmark-save=baseline`
  - How to compare against baselines: `pytest tests/performance/ --benchmark-compare=baseline`
  - How to skip performance tests during regular test runs: `pytest --benchmark-disable`
  - Performance testing is optional and can be run separately from functional tests

**New files:**
- `tests/performance/conftest.py`
- `tests/performance/baselines.json`
- `tests/performance/README.md`

---

### 3. Implement Agent Operation Benchmarks (Phase 1)

**Objective:** Benchmark task execution, context loading, flash save, and checkpoint operations.

**Actions:**
- Create `tests/performance/test_agent_operations.py` with benchmark tests:
  
  **Test: Context Loading Performance (Target: <200ms for 1000 items)**
  - Use `ContextManager.recalculate_scores_for_agent()` from file:codeframe/lib/context_manager.py
  - Pre-populate database with 1000 context items across HOT/WARM/COLD tiers
  - Benchmark loading and score recalculation
  - Assert mean < 200ms, max < 500ms
  
  **Test: Flash Save Duration (Target: <2s)**
  - Use `ContextManager.flash_save()` from file:codeframe/lib/context_manager.py
  - Pre-populate with 1000 context items (300 COLD items to archive)
  - Benchmark full flash save workflow (checkpoint creation + archival)
  - Assert mean < 2s, max < 5s
  
  **Test: Checkpoint Creation (Target: <10s)**
  - Use `CheckpointManager.create_checkpoint()` from file:codeframe/lib/checkpoint_manager.py
  - Set up git repository with 50 files, database with 100 tasks
  - Benchmark checkpoint creation (git commit + DB backup + context snapshot)
  - Assert mean < 10s, max < 30s
  
  **Test: Checkpoint Restoration (Target: <30s)**
  - Use `CheckpointManager.restore_checkpoint()` from file:codeframe/lib/checkpoint_manager.py
  - Pre-create checkpoint with 100 tasks, 1000 context items
  - Benchmark full restoration workflow
  - Assert mean < 30s, max < 60s
  
  **Note:** This phase establishes the foundation for performance testing. Subsequent phases (database, API, quality gates) can be implemented independently as needed.

**New file:**
- `tests/performance/test_agent_operations.py`

**Reference implementations:**
- file:codeframe/lib/context_manager.py (lines 38-286)
- file:codeframe/lib/checkpoint_manager.py (lines 54-649)

---

### 4. Implement Database Operation Benchmarks (Phase 2)

**Objective:** Benchmark database queries with large datasets and concurrent operations.

**Actions:**
- Create `tests/performance/test_database_operations.py` with benchmark tests:
  
  **Test: Query Performance with 10,000+ Tasks**
  - Use `TaskRepository` methods from file:codeframe/persistence/database.py
  - Pre-populate database with 10,000 tasks across multiple projects
  - Benchmark queries:
    - `get_pending_tasks()` with filters
    - `get_tasks_by_issue()` with pagination
    - `get_recently_completed_tasks()` with date ranges
  - Assert query time < 100ms for indexed queries, < 500ms for complex joins
  
  **Test: Concurrent Read/Write Performance**
  - Use async database operations from file:codeframe/persistence/database.py
  - Simulate 10 concurrent agents reading/writing context items
  - Use `asyncio.gather()` to run operations in parallel
  - Benchmark throughput (operations/second)
  - Assert no deadlocks, consistent results
  
  **Test: Context Item Queries (1000+ items)**
  - Use `ContextRepository.list_context_items()` from file:codeframe/persistence/database.py
  - Pre-populate with 10,000 context items across 10 agents
  - Benchmark filtered queries by tier, agent_id, importance_score
  - Assert query time < 50ms with proper indexes
  
  **Test: Checkpoint Queries**
  - Use `CheckpointRepository` methods from file:codeframe/persistence/database.py
  - Pre-populate with 100 checkpoints
  - Benchmark `get_checkpoints()` and `get_checkpoint_by_id()`
  - Assert query time < 20ms
  
  **Note:** This phase can be implemented after Phase 1 is complete and integrated. Database performance is critical for scale, so this phase should be prioritized after agent operations are established.

**New file:**
- `tests/performance/test_database_operations.py`

**Reference implementations:**
- file:codeframe/persistence/database.py (lines 260-677)
- Repository pattern documentation: `docs/architecture/database-repository-pattern.md`

---

### 5. Implement API Endpoint Load Tests (Phase 3)

**Objective:** Load test API endpoints and WebSocket broadcasts using locust.

**Actions:**
- Create `tests/performance/locustfile.py` for load testing:
  
  **User Class: APIUser**
  - Simulates typical API usage patterns
  - Tasks:
    - GET `/api/projects` (weight: 10)
    - GET `/api/agents` (weight: 8)
    - GET `/api/tasks?project_id=1` (weight: 15)
    - GET `/api/context?agent_id=test-001` (weight: 12)
    - POST `/api/checkpoints` (weight: 2)
    - GET `/health` (weight: 5)
  - Wait time: 1-3 seconds between requests
  
  **User Class: WebSocketUser**
  - Simulates WebSocket connections for real-time updates
  - Connects to `/ws` endpoint from file:codeframe/ui/routers/websocket.py
  - Subscribes to project updates
  - Measures broadcast latency (time from server event to client receipt)
  - Target: <100ms broadcast latency
  
  **Load Test Scenarios:**
  - Scenario 1: 100 concurrent users, 10 req/s sustained load (5 minutes)
  - Scenario 2: 1000 concurrent WebSocket connections
  - Scenario 3: Spike test (0→500 users in 30s, hold 2 min, ramp down)
  
- Create `tests/performance/run_load_tests.sh` script:
  - Starts FastAPI server in background
  - Waits for server health check
  - Runs locust in headless mode with different scenarios
  - Captures results to `tests/performance/load_test_results/`
  - Stops server after tests
  
- Create `tests/performance/test_api_performance.py` for pytest-based API benchmarks:
  - Use `TestClient` from FastAPI for synchronous benchmarks
  - Benchmark individual endpoint response times
  - Assert 95th percentile < 500ms, 99th percentile < 2s
  
  **Note:** This phase focuses on API and WebSocket performance. Can be implemented after Phase 1 and 2 are established, or in parallel if API performance is a priority.

**New files:**
- `tests/performance/locustfile.py`
- `tests/performance/run_load_tests.sh`
- `tests/performance/test_api_performance.py`

**Reference implementations:**
- file:codeframe/ui/server.py (lines 1-371)
- file:codeframe/ui/routers/websocket.py

---

### 6. Implement Quality Gate Benchmarks (Phase 4)

**Objective:** Benchmark quality gate execution times.

**Actions:**
- Create `tests/performance/test_quality_gates.py` with benchmark tests:
  
  **Test: Lint Analysis Time (Target: <2min)**
  - Use `QualityGates.run_linting_gate()` from file:codeframe/lib/quality_gates.py
  - Set up test project with 100 Python files (10,000 LOC)
  - Benchmark ruff execution time
  - Assert mean < 120s
  
  **Test: Test Execution Overhead (Target: <10% of test time)**
  - Use `TestRunner.run_tests()` from file:codeframe/testing/test_runner.py
  - Measure baseline test execution time (direct pytest)
  - Measure test execution through TestRunner
  - Calculate overhead percentage
  - Assert overhead < 10%
  
  **Test: Coverage Calculation Time**
  - Use `QualityGates.run_coverage_gate()` from file:codeframe/lib/quality_gates.py
  - Benchmark pytest with coverage on 100-file project
  - Assert mean < 180s (3 minutes)
  
  **Test: Review Agent Analysis (Target: <30s per file)**
  - Use `QualityGates.run_review_gate()` from file:codeframe/lib/quality_gates.py
  - Mock Review Agent to avoid LLM calls (focus on orchestration overhead)
  - Benchmark review workflow for 10 files
  - Assert mean < 30s per file
  
  **Test: All Gates Orchestration**
  - Use `QualityGates.run_all_gates()` from file:codeframe/lib/quality_gates.py
  - Benchmark full quality gate pipeline
  - Measure individual gate times and total time
  - Assert total time < 5 minutes for typical task
  
  **Note:** This phase is optional and can be deferred if quality gate performance is not a current bottleneck. Implement after Phases 1-3 are complete.

**New file:**
- `tests/performance/test_quality_gates.py`

**Reference implementations:**
- file:codeframe/lib/quality_gates.py (lines 107-1085)
- file:codeframe/testing/test_runner.py (lines 1-187)

---

### 7. Establish Performance Baselines

**Objective:** Run all benchmarks to establish initial baselines.

**Actions:**
- After each phase is implemented, run performance tests with baseline capture:
  ```bash
  pytest tests/performance/ --benchmark-only --benchmark-save=baseline --benchmark-json=tests/performance/baseline_results.json
  ```
- Review baseline results and adjust targets if needed
- Document baseline environment:
  - Hardware specs (CPU, RAM, disk type)
  - Python version
  - Database size
  - OS and kernel version
- Store baselines in `tests/performance/baselines.json` with metadata:
  ```json
  {
    "environment": {
      "python_version": "3.11.x",
      "cpu": "...",
      "ram": "...",
      "date": "2025-12-26"
    },
    "baselines": {
      "context_load_1000_items": {"mean": 0.15, "max": 0.35},
      "flash_save": {"mean": 1.5, "max": 3.2},
      ...
    }
  }
  ```
- Create `tests/performance/BASELINES.md` documenting:
  - Baseline values for each operation
  - Target thresholds (from ticket)
  - Critical thresholds (20% above target)
  - Instructions for updating baselines
- Baselines should be committed to version control to track performance over time

**New files:**
- `tests/performance/baseline_results.json`
- `tests/performance/BASELINES.md`

---

### 8. Integrate Performance Tests into CI (Optional)

**Objective:** Add performance regression detection to GitHub Actions workflow.

**Actions:**
- Update file:.github/workflows/test.yml to add optional performance testing job:
  
  ```yaml
  performance-tests:
    name: Performance Benchmarks
    runs-on: ubuntu-latest
    needs: backend-tests
    # Only run on main branch or when performance tests are modified
    if: github.ref == 'refs/heads/main' || contains(github.event.head_commit.modified, 'tests/performance/')
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
      
      - name: Install uv
        uses: astral-sh/setup-uv@v4
        with:
          enable-cache: true
      
      - name: Install dependencies
        run: |
          uv venv
          uv sync --extra perf
      
      - name: Run performance benchmarks
        run: |
          uv run pytest tests/performance/ \
            --benchmark-only \
            --benchmark-compare=baseline \
            --benchmark-compare-fail=mean:20% \
            --benchmark-json=performance_results.json
      
      - name: Check for performance regressions
        run: |
          # Fail if any benchmark regressed by >20%
          python scripts/check_performance_regression.py performance_results.json
      
      - name: Upload performance results
        uses: actions/upload-artifact@v4
        with:
          name: performance-results
          path: performance_results.json
          retention-days: 90
      
      - name: Comment PR with performance results
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('performance_results.json'));
            // Format and post comment with performance comparison
  ```

- Create `scripts/check_performance_regression.py`:
  - Parses pytest-benchmark JSON output
  - Compares against baselines
  - Fails CI if any benchmark regressed >20%
  - Generates markdown report for PR comments
  
- Update `pytest.ini` to add performance test markers:
  ```ini
  markers =
      performance: marks tests as performance benchmarks
      load_test: marks tests as load tests (requires running server)
  ```
  
- **Note:** This CI integration is optional and can be deferred. Performance tests can be run manually during development and on-demand in CI until the infrastructure is fully mature.

**Files to modify:**
- file:.github/workflows/test.yml
- file:pytest.ini

**New files:**
- `scripts/check_performance_regression.py`

---

### 9. Create Performance Dashboard (Optional)

**Objective:** Generate HTML reports showing performance trends over time.

**Actions:**
- Create `scripts/generate_performance_report.py`:
  - Reads historical performance results from CI artifacts
  - Generates HTML dashboard with charts:
    - Line charts showing performance trends over time
    - Bar charts comparing current vs baseline
    - Heatmap showing which operations are slowest
    - Table with all benchmark results and thresholds
  - Uses matplotlib or plotly for visualizations
  - Outputs to `tests/performance/reports/index.html`
  
- Create `tests/performance/reports/.gitkeep` to track directory
  
- Add GitHub Actions step to publish performance reports:
  ```yaml
  - name: Generate performance report
    run: python scripts/generate_performance_report.py
  
  - name: Deploy to GitHub Pages
    if: github.ref == 'refs/heads/main'
    uses: peaceiris/actions-gh-pages@v3
    with:
      github_token: ${{ secrets.GITHUB_TOKEN }}
      publish_dir: tests/performance/reports
  ```
  
- **Note:** This dashboard is a nice-to-have feature that can be added after the core performance testing infrastructure is established and baselines are being tracked.

**New files:**
- `scripts/generate_performance_report.py`
- `tests/performance/reports/.gitkeep`

---

### 10. Document Performance Testing

**Objective:** Create comprehensive documentation for performance testing.

**Actions:**
- Update file:TESTING.md to add "Performance Testing" section:
  - Overview of performance testing strategy and phased approach
  - How to run performance tests locally: `pytest tests/performance/ --benchmark-only`
  - How to skip performance tests during regular runs: `pytest --benchmark-disable`
  - How to interpret benchmark results
  - How to update baselines
  - Performance targets and thresholds
  - Link to performance dashboard (when available)
  - Note that performance testing is optional and can be installed with `uv sync --extra perf`
  
- Create `docs/performance-testing.md` with detailed guide:
  - Architecture of performance testing infrastructure
  - Phased implementation approach (Phase 1-4)
  - Benchmark test patterns and best practices
  - Load testing scenarios and how to add new ones
  - CI integration and regression detection (optional)
  - Troubleshooting performance issues
  - Performance optimization tips
  - How to contribute new performance tests
  
- Add performance testing to file:README.md:
  ```markdown
  ## Performance Testing (Optional)
  
  Install performance testing tools:
  ```bash
  uv sync --extra perf
  ```
  
  Run performance benchmarks:
  ```bash
  pytest tests/performance/ --benchmark-only
  ```
  
  Run load tests:
  ```bash
  ./tests/performance/run_load_tests.sh
  ```
  
  See [Performance Testing Guide](docs/performance-testing.md) for more details.
  ```

**Files to modify:**
- file:TESTING.md
- file:README.md

**New files:**
- `docs/performance-testing.md`

---

## Performance Targets Summary

| Operation | Target | Critical Threshold | Test Location | Phase |
|-----------|--------|-------------------|---------------|-------|
| Task execution | <30s | <60s | `test_agent_operations.py` | 1 |
| Context load (1000 items) | <200ms | <500ms | `test_agent_operations.py` | 1 |
| Flash save | <2s | <5s | `test_agent_operations.py` | 1 |
| Checkpoint create | <10s | <30s | `test_agent_operations.py` | 1 |
| Checkpoint restore | <30s | <60s | `test_agent_operations.py` | 1 |
| Database query (10k tasks) | <100ms | <500ms | `test_database_operations.py` | 2 |
| Concurrent read/write | N/A | N/A | `test_database_operations.py` | 2 |
| API response (95th %ile) | <500ms | <2s | `test_api_performance.py` | 3 |
| WebSocket broadcast | <100ms | <500ms | `locustfile.py` | 3 |
| Lint analysis | <2min | <5min | `test_quality_gates.py` | 4 |
| Test execution overhead | <10% | <20% | `test_quality_gates.py` | 4 |
| Review agent analysis | <30s/file | <2min/file | `test_quality_gates.py` | 4 |

---

## Phased Implementation Approach (Phase 5)

**Phase 5.1 (Foundation):** Agent operation benchmarks establish the core performance testing infrastructure.

**Phase 5.2 (Database):** Database operation benchmarks ensure query performance at scale.

**Phase 5.3 (API):** API and WebSocket load tests validate endpoint performance under concurrent load.

**Phase 5.4 (Quality Gates):** Quality gate benchmarks measure orchestration and analysis overhead (optional, can be deferred).

**Phase 5.5 (CI Integration & Dashboard):** Optional enhancements that can be added after core phases are complete.

---

## Architecture Diagram

```mermaid
graph TD
    A[Phase 5: Performance Testing Infrastructure] --> B[Phase 5.1: Agent Operations]
    A --> C[Phase 5.2: Database Operations]
    A --> D[Phase 5.3: API Load Tests]
    A --> E[Phase 5.4: Quality Gates]
    
    B --> B1[Context Loading<br/>Flash Save<br/>Checkpoints]
    C --> C1[Query Performance<br/>Concurrent Ops<br/>Index Efficiency]
    D --> D1[API Endpoints<br/>WebSocket Broadcasts<br/>Concurrent Clients]
    E --> E1[Lint Analysis<br/>Test Overhead<br/>Review Agent]
    
    B1 --> F[Baseline Storage<br/>baselines.json]
    C1 --> F
    D1 --> F
    E1 --> F
    
    F --> G[Phase 5.5: CI Integration]
    F --> H[Phase 5.5: Performance Dashboard]
    
    G --> I[Regression Detection<br/>PR Comments]
    H --> J[Trend Visualization<br/>GitHub Pages]
    
    style A fill:#e1f5ff
    style B fill:#fff4e1
    style C fill:#fff4e1
    style D fill:#ffe1f5
    style E fill:#ffe1f5
    style G fill:#e1ffe1
    style H fill:#e1ffe1
```

---

## Acceptance Criteria Checklist (Phase 5)

- [ ] Phase 5.1: Performance benchmarks for agent operations (context loading, flash save, checkpoints)
- [ ] Phase 5.2: Performance benchmarks for database operations (queries, concurrent access)
- [ ] Phase 5.3: Load tests for API endpoints and WebSocket broadcasts
- [ ] Phase 5.4 (Optional): Performance benchmarks for quality gates
- [ ] Baselines established for regression detection (stored in `baselines.json`)
- [ ] Phase 5.5: CI integration with regression detection (>20% threshold)
- [ ] Phase 5.5: Performance dashboard with trend visualization
- [ ] Documentation updated (TESTING.md, README.md, performance-testing.md)
- [ ] Performance testing infrastructure is modular and can be extended

---

Execution Information

Branch: main
Commit: 4d894a6

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

📝 CodeRabbit Plan Mode

Generate an implementation plan and prompts that you can use with your favorite coding agent.

• Create Plan

Examples

• Example 1
• Example 2

---

🔗 Related PRs

#125 - Implement WorkerAgent execute_task method [merged]
#128 - Implement bottleneck detection for workflow monitoring [merged]
#151 - Integrate QualityTracker with workflow system [merged]
#152 - Implement agent maturity level system [merged]
#246 - feat(agents): trigger multi-agent execution after task approval [merged]

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!

[traycerai]

I have updated the Implementation Plan to reflect the recent changes in the ticket.

New Plan

Observations

The codebase has a comprehensive test infrastructure with 111+ tests organized in the tests/ directory, using pytest with pytest-asyncio. However, no performance benchmarks or load tests exist. The system uses a repository pattern with 17 domain-specific repositories, FastAPI for the API layer, and has well-defined operations for context management, checkpoints, quality gates, and agent operations. The CI/CD pipeline (file:.github/workflows/test.yml) runs functional tests but lacks performance regression detection. Key operations like context loading, flash saves, checkpoint creation, database queries, API endpoints, and quality gate execution are prime candidates for performance benchmarking.

Approach

The implementation will follow a three-tier benchmarking strategy: (1) Unit-level performance tests using pytest-benchmark for core operations (context loading, database queries, checkpoint operations), (2) Integration-level load tests using locust for API endpoints and WebSocket broadcasts, and (3) CI integration with automated regression detection. This approach aligns with the ticket's requirements by establishing baselines, detecting regressions automatically, and generating performance reports. The plan prioritizes operations by criticality (agent operations → database → API → quality gates) and sets up infrastructure before implementing specific benchmarks.

---

Implementation Steps

1. Add Performance Testing Dependencies

Objective: Install pytest-benchmark and locust for performance and load testing.

Actions:

• Update file:pyproject.toml to add performance testing dependencies:
  • Add pytest-benchmark>=4.0.0 to dependencies section
  • Add locust>=2.20.0 to [project.optional-dependencies].dev section
  • Add pytest-timeout>=2.2.0 for benchmark timeouts
• Run uv sync --extra dev to install new dependencies
• Verify installation by running pytest --benchmark-help and locust --version

Files to modify:

• file:pyproject.toml

---

2. Create Performance Test Infrastructure

Objective: Set up directory structure and base fixtures for performance tests.

Actions:

• Create new directory tests/performance/ for all performance tests
• Create tests/performance/conftest.py with shared fixtures:
  • benchmark_db fixture: In-memory database with pre-populated data (1000 context items, 100 tasks, 10 agents)
  • benchmark_project_root fixture: Temporary project directory for file-based operations
  • benchmark_context_items fixture: Factory for generating test context items
  • performance_baseline fixture: Loads baseline metrics from JSON file for regression detection
• Create tests/performance/baselines.json to store performance baselines (initially empty, populated after first run)
• Create tests/performance/README.md documenting:
  • How to run performance tests: pytest tests/performance/ --benchmark-only
  • How to update baselines: pytest tests/performance/ --benchmark-save=baseline
  • How to compare against baselines: pytest tests/performance/ --benchmark-compare=baseline

New files:

• tests/performance/conftest.py
• tests/performance/baselines.json
• tests/performance/README.md

---

3. Implement Agent Operation Benchmarks

Objective: Benchmark task execution, context loading, flash save, and checkpoint operations.

Actions:

• Create tests/performance/test_agent_operations.py with benchmark tests:

  Test: Context Loading Performance (Target: <200ms for 1000 items)

  • Use ContextManager.recalculate_scores_for_agent() from file:codeframe/lib/context_manager.py
  • Pre-populate database with 1000 context items across HOT/WARM/COLD tiers
  • Benchmark loading and score recalculation
  • Assert mean < 200ms, max < 500ms

  Test: Flash Save Duration (Target: <2s)

  • Use ContextManager.flash_save() from file:codeframe/lib/context_manager.py
  • Pre-populate with 1000 context items (300 COLD items to archive)
  • Benchmark full flash save workflow (checkpoint creation + archival)
  • Assert mean < 2s, max < 5s

  Test: Checkpoint Creation (Target: <10s)

  • Use CheckpointManager.create_checkpoint() from file:codeframe/lib/checkpoint_manager.py
  • Set up git repository with 50 files, database with 100 tasks
  • Benchmark checkpoint creation (git commit + DB backup + context snapshot)
  • Assert mean < 10s, max < 30s

  Test: Checkpoint Restoration (Target: <30s)

  • Use CheckpointManager.restore_checkpoint() from file:codeframe/lib/checkpoint_manager.py
  • Pre-create checkpoint with 100 tasks, 1000 context items
  • Benchmark full restoration workflow
  • Assert mean < 30s, max < 60s

New file:

• tests/performance/test_agent_operations.py

Reference implementations:

• file:codeframe/lib/context_manager.py (lines 38-286)
• file:codeframe/lib/checkpoint_manager.py (lines 54-649)

---

4. Implement Database Operation Benchmarks

Objective: Benchmark database queries with large datasets and concurrent operations.

Actions:

• Create tests/performance/test_database_operations.py with benchmark tests:

  Test: Query Performance with 10,000+ Tasks

  • Use TaskRepository methods from file:codeframe/persistence/database.py
  • Pre-populate database with 10,000 tasks across multiple projects
  • Benchmark queries:
    • get_pending_tasks() with filters
    • get_tasks_by_issue() with pagination
    • get_recently_completed_tasks() with date ranges
  • Assert query time < 100ms for indexed queries, < 500ms for complex joins

  Test: Concurrent Read/Write Performance

  • Use async database operations from file:codeframe/persistence/database.py
  • Simulate 10 concurrent agents reading/writing context items
  • Use asyncio.gather() to run operations in parallel
  • Benchmark throughput (operations/second)
  • Assert no deadlocks, consistent results

  Test: Context Item Queries (1000+ items)

  • Use ContextRepository.list_context_items() from file:codeframe/persistence/database.py
  • Pre-populate with 10,000 context items across 10 agents
  • Benchmark filtered queries by tier, agent_id, importance_score
  • Assert query time < 50ms with proper indexes

  Test: Checkpoint Queries

  • Use CheckpointRepository methods from file:codeframe/persistence/database.py
  • Pre-populate with 100 checkpoints
  • Benchmark get_checkpoints() and get_checkpoint_by_id()
  • Assert query time < 20ms

New file:

• tests/performance/test_database_operations.py

Reference implementations:

• file:codeframe/persistence/database.py (lines 260-677)
• Repository pattern documentation: docs/architecture/database-repository-pattern.md

---

5. Implement API Endpoint Load Tests

Objective: Load test API endpoints and WebSocket broadcasts using locust.

Actions:

• Create tests/performance/locustfile.py for load testing:

  User Class: APIUser

  • Simulates typical API usage patterns
  • Tasks:
    • GET /api/projects (weight: 10)
    • GET /api/agents (weight: 8)
    • GET /api/tasks?project_id=1 (weight: 15)
    • GET /api/context?agent_id=test-001 (weight: 12)
    • POST /api/checkpoints (weight: 2)
    • GET /health (weight: 5)
  • Wait time: 1-3 seconds between requests

  User Class: WebSocketUser

  • Simulates WebSocket connections for real-time updates
  • Connects to /ws endpoint from file:codeframe/ui/routers/websocket.py
  • Subscribes to project updates
  • Measures broadcast latency (time from server event to client receipt)
  • Target: <100ms broadcast latency

  Load Test Scenarios:

  • Scenario 1: 100 concurrent users, 10 req/s sustained load (5 minutes)
  • Scenario 2: 1000 concurrent WebSocket connections
  • Scenario 3: Spike test (0→500 users in 30s, hold 2 min, ramp down)

• Create tests/performance/run_load_tests.sh script:

  • Starts FastAPI server in background
  • Waits for server health check
  • Runs locust in headless mode with different scenarios
  • Captures results to tests/performance/load_test_results/
  • Stops server after tests

• Create tests/performance/test_api_performance.py for pytest-based API benchmarks:

  • Use TestClient from FastAPI for synchronous benchmarks
  • Benchmark individual endpoint response times
  • Assert 95th percentile < 500ms, 99th percentile < 2s

New files:

• tests/performance/locustfile.py
• tests/performance/run_load_tests.sh
• tests/performance/test_api_performance.py

Reference implementations:

• file:codeframe/ui/server.py (lines 1-371)
• file:codeframe/ui/routers/websocket.py

---

6. Implement Quality Gate Benchmarks

Objective: Benchmark quality gate execution times.

Actions:

• Create tests/performance/test_quality_gates.py with benchmark tests:

  Test: Lint Analysis Time (Target: <2min)

  • Use QualityGates.run_linting_gate() from file:codeframe/lib/quality_gates.py
  • Set up test project with 100 Python files (10,000 LOC)
  • Benchmark ruff execution time
  • Assert mean < 120s

  Test: Test Execution Overhead (Target: <10% of test time)

  • Use TestRunner.run_tests() from file:codeframe/testing/test_runner.py
  • Measure baseline test execution time (direct pytest)
  • Measure test execution through TestRunner
  • Calculate overhead percentage
  • Assert overhead < 10%

  Test: Coverage Calculation Time

  • Use QualityGates.run_coverage_gate() from file:codeframe/lib/quality_gates.py
  • Benchmark pytest with coverage on 100-file project
  • Assert mean < 180s (3 minutes)

  Test: Review Agent Analysis (Target: <30s per file)

  • Use QualityGates.run_review_gate() from file:codeframe/lib/quality_gates.py
  • Mock Review Agent to avoid LLM calls (focus on orchestration overhead)
  • Benchmark review workflow for 10 files
  • Assert mean < 30s per file

  Test: All Gates Orchestration

  • Use QualityGates.run_all_gates() from file:codeframe/lib/quality_gates.py
  • Benchmark full quality gate pipeline
  • Measure individual gate times and total time
  • Assert total time < 5 minutes for typical task

New file:

• tests/performance/test_quality_gates.py

Reference implementations:

• file:codeframe/lib/quality_gates.py (lines 107-1085)
• file:codeframe/testing/test_runner.py (lines 1-187)

---

7. Establish Performance Baselines

Objective: Run all benchmarks to establish initial baselines.

Actions:

• Run all performance tests with baseline capture:

  pytest tests/performance/ --benchmark-only --benchmark-save=baseline --benchmark-json=tests/performance/baseline_results.json

• Review baseline results and adjust targets if needed
• Document baseline environment:
  • Hardware specs (CPU, RAM, disk type)
  • Python version
  • Database size
  • OS and kernel version
• Store baselines in tests/performance/baselines.json with metadata:

  {
    "environment": {
      "python_version": "3.11.x",
      "cpu": "...",
      "ram": "...",
      "date": "2025-12-26"
    },
    "baselines": {
      "context_load_1000_items": {"mean": 0.15, "max": 0.35},
      "flash_save": {"mean": 1.5, "max": 3.2},
      ...
    }
  }

• Create tests/performance/BASELINES.md documenting:
  • Baseline values for each operation
  • Target thresholds (from ticket)
  • Critical thresholds (20% above target)
  • Instructions for updating baselines

New files:

• tests/performance/baseline_results.json
• tests/performance/BASELINES.md

---

8. Integrate Performance Tests into CI

Objective: Add performance regression detection to GitHub Actions workflow.

Actions:

• Update file:.github/workflows/test.yml to add new job:

  performance-tests:
    name: Performance Benchmarks
    runs-on: ubuntu-latest
    needs: backend-tests
    # Only run on main branch or when performance tests are modified
    if: github.ref == 'refs/heads/main' || contains(github.event.head_commit.modified, 'tests/performance/')
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
      
      - name: Install uv
        uses: astral-sh/setup-uv@v4
        with:
          enable-cache: true
      
      - name: Install dependencies
        run: |
          uv venv
          uv sync --extra dev
      
      - name: Run performance benchmarks
        run: |
          uv run pytest tests/performance/ \
            --benchmark-only \
            --benchmark-compare=baseline \
            --benchmark-compare-fail=mean:20% \
            --benchmark-json=performance_results.json
      
      - name: Check for performance regressions
        run: |
          # Fail if any benchmark regressed by >20%
          python scripts/check_performance_regression.py performance_results.json
      
      - name: Upload performance results
        uses: actions/upload-artifact@v4
        with:
          name: performance-results
          path: performance_results.json
          retention-days: 90
      
      - name: Comment PR with performance results
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('performance_results.json'));
            // Format and post comment with performance comparison

• Create scripts/check_performance_regression.py:

  • Parses pytest-benchmark JSON output
  • Compares against baselines
  • Fails CI if any benchmark regressed >20%
  • Generates markdown report for PR comments

• Update pytest.ini to add performance test markers:

  markers =
      performance: marks tests as performance benchmarks
      load_test: marks tests as load tests (requires running server)

Files to modify:

• file:.github/workflows/test.yml
• file:pytest.ini

New files:

• scripts/check_performance_regression.py

---

9. Create Performance Dashboard

Objective: Generate HTML reports showing performance trends over time.

Actions:

• Create scripts/generate_performance_report.py:

  • Reads historical performance results from CI artifacts
  • Generates HTML dashboard with charts:
    • Line charts showing performance trends over time
    • Bar charts comparing current vs baseline
    • Heatmap showing which operations are slowest
    • Table with all benchmark results and thresholds
  • Uses matplotlib or plotly for visualizations
  • Outputs to tests/performance/reports/index.html

• Create tests/performance/reports/.gitkeep to track directory

• Add GitHub Actions step to publish performance reports:

  - name: Generate performance report
    run: python scripts/generate_performance_report.py
  
  - name: Deploy to GitHub Pages
    if: github.ref == 'refs/heads/main'
    uses: peaceiris/actions-gh-pages@v3
    with:
      github_token: ${{ secrets.GITHUB_TOKEN }}
      publish_dir: tests/performance/reports

New files:

• scripts/generate_performance_report.py
• tests/performance/reports/.gitkeep

---

10. Document Performance Testing

Objective: Create comprehensive documentation for performance testing.

Actions:

• Update file:TESTING.md to add "Performance Testing" section:

  • Overview of performance testing strategy
  • How to run performance tests locally
  • How to interpret benchmark results
  • How to update baselines
  • Performance targets and thresholds
  • Link to performance dashboard

• Create docs/performance-testing.md with detailed guide:

  • Architecture of performance testing infrastructure
  • Benchmark test patterns and best practices
  • Load testing scenarios and how to add new ones
  • CI integration and regression detection
  • Troubleshooting performance issues
  • Performance optimization tips

• Add performance testing to file:README.md:

  ## Performance Testing
  
  Run performance benchmarks:
  ```bash
  pytest tests/performance/ --benchmark-only

  Run load tests:

  ./tests/performance/run_load_tests.sh

  View performance dashboard: Performance Reports

Files to modify:

• file:TESTING.md
• file:README.md

New files:

• docs/performance-testing.md

---

Performance Targets Summary

Operation	Target	Critical Threshold	Test Location
Task execution	<30s	<60s	test_agent_operations.py
Context load (1000 items)	<200ms	<500ms	test_agent_operations.py
Flash save	<2s	<5s	test_agent_operations.py
Checkpoint create	<10s	<30s	test_agent_operations.py
Checkpoint restore	<30s	<60s	test_agent_operations.py
Database query (10k tasks)	<100ms	<500ms	test_database_operations.py
API response (95th %ile)	<500ms	<2s	test_api_performance.py
WebSocket broadcast	<100ms	<500ms	locustfile.py
Lint analysis	<2min	<5min	test_quality_gates.py
Test execution overhead	<10%	<20%	test_quality_gates.py
Review agent analysis	<30s/file	<2min/file	test_quality_gates.py

---

Architecture Diagram

graph TD
    A[Performance Testing Infrastructure] --> B[Unit Benchmarks]
    A --> C[Load Tests]
    A --> D[CI Integration]
    
    B --> B1[Agent Operations<br/>pytest-benchmark]
    B --> B2[Database Operations<br/>pytest-benchmark]
    B --> B3[Quality Gates<br/>pytest-benchmark]
    
    C --> C1[API Endpoints<br/>locust]
    C --> C2[WebSocket Broadcasts<br/>locust]
    
    D --> D1[GitHub Actions<br/>Regression Detection]
    D --> D2[Performance Dashboard<br/>Trend Visualization]
    D --> D3[PR Comments<br/>Performance Comparison]
    
    B1 --> E[Baseline Storage<br/>baselines.json]
    B2 --> E
    B3 --> E
    C1 --> E
    C2 --> E
    
    E --> D1
    D1 --> D2
    D1 --> D3
    
    style A fill:#e1f5ff
    style B fill:#fff4e1
    style C fill:#ffe1f5
    style D fill:#e1ffe1

Loading

---

Acceptance Criteria Checklist

• Performance benchmarks for all core operations (agent, database, API, quality gates)
• Load tests for API endpoints and WebSocket broadcasts using locust
• Baselines established for regression detection (stored in baselines.json)
• CI fails on performance regressions >20% (GitHub Actions job)
• Performance reports generated per commit (HTML dashboard)
• Dashboard shows performance trends (GitHub Pages deployment)
• All targets from ticket met or documented if not achievable
• Documentation updated (TESTING.md, README.md, performance-testing.md)

Import In IDE

🤖 Prompt for AI Agents

## Observations

The codebase has a comprehensive test infrastructure with 111+ tests organized in the `tests/` directory, using pytest with pytest-asyncio. However, **no performance benchmarks or load tests exist**. The system uses a repository pattern with 17 domain-specific repositories, FastAPI for the API layer, and has well-defined operations for context management, checkpoints, quality gates, and agent operations. The CI/CD pipeline (file:.github/workflows/test.yml) runs functional tests but lacks performance regression detection. Key operations like context loading, flash saves, checkpoint creation, database queries, API endpoints, and quality gate execution are prime candidates for performance benchmarking.

## Approach

The implementation will follow a **three-tier benchmarking strategy**: (1) **Unit-level performance tests** using pytest-benchmark for core operations (context loading, database queries, checkpoint operations), (2) **Integration-level load tests** using locust for API endpoints and WebSocket broadcasts, and (3) **CI integration** with automated regression detection. This approach aligns with the ticket's requirements by establishing baselines, detecting regressions automatically, and generating performance reports. The plan prioritizes operations by criticality (agent operations → database → API → quality gates) and sets up infrastructure before implementing specific benchmarks.

---

## Implementation Steps

### 1. Add Performance Testing Dependencies

**Objective:** Install pytest-benchmark and locust for performance and load testing.

**Actions:**
- Update file:pyproject.toml to add performance testing dependencies:
  - Add `pytest-benchmark>=4.0.0` to `dependencies` section
  - Add `locust>=2.20.0` to `[project.optional-dependencies].dev` section
  - Add `pytest-timeout>=2.2.0` for benchmark timeouts
- Run `uv sync --extra dev` to install new dependencies
- Verify installation by running `pytest --benchmark-help` and `locust --version`

**Files to modify:**
- file:pyproject.toml

---

### 2. Create Performance Test Infrastructure

**Objective:** Set up directory structure and base fixtures for performance tests.

**Actions:**
- Create new directory `tests/performance/` for all performance tests
- Create `tests/performance/conftest.py` with shared fixtures:
  - `benchmark_db` fixture: In-memory database with pre-populated data (1000 context items, 100 tasks, 10 agents)
  - `benchmark_project_root` fixture: Temporary project directory for file-based operations
  - `benchmark_context_items` fixture: Factory for generating test context items
  - `performance_baseline` fixture: Loads baseline metrics from JSON file for regression detection
- Create `tests/performance/baselines.json` to store performance baselines (initially empty, populated after first run)
- Create `tests/performance/README.md` documenting:
  - How to run performance tests: `pytest tests/performance/ --benchmark-only`
  - How to update baselines: `pytest tests/performance/ --benchmark-save=baseline`
  - How to compare against baselines: `pytest tests/performance/ --benchmark-compare=baseline`

**New files:**
- `tests/performance/conftest.py`
- `tests/performance/baselines.json`
- `tests/performance/README.md`

---

### 3. Implement Agent Operation Benchmarks

**Objective:** Benchmark task execution, context loading, flash save, and checkpoint operations.

**Actions:**
- Create `tests/performance/test_agent_operations.py` with benchmark tests:
  
  **Test: Context Loading Performance (Target: <200ms for 1000 items)**
  - Use `ContextManager.recalculate_scores_for_agent()` from file:codeframe/lib/context_manager.py
  - Pre-populate database with 1000 context items across HOT/WARM/COLD tiers
  - Benchmark loading and score recalculation
  - Assert mean < 200ms, max < 500ms
  
  **Test: Flash Save Duration (Target: <2s)**
  - Use `ContextManager.flash_save()` from file:codeframe/lib/context_manager.py
  - Pre-populate with 1000 context items (300 COLD items to archive)
  - Benchmark full flash save workflow (checkpoint creation + archival)
  - Assert mean < 2s, max < 5s
  
  **Test: Checkpoint Creation (Target: <10s)**
  - Use `CheckpointManager.create_checkpoint()` from file:codeframe/lib/checkpoint_manager.py
  - Set up git repository with 50 files, database with 100 tasks
  - Benchmark checkpoint creation (git commit + DB backup + context snapshot)
  - Assert mean < 10s, max < 30s
  
  **Test: Checkpoint Restoration (Target: <30s)**
  - Use `CheckpointManager.restore_checkpoint()` from file:codeframe/lib/checkpoint_manager.py
  - Pre-create checkpoint with 100 tasks, 1000 context items
  - Benchmark full restoration workflow
  - Assert mean < 30s, max < 60s

**New file:**
- `tests/performance/test_agent_operations.py`

**Reference implementations:**
- file:codeframe/lib/context_manager.py (lines 38-286)
- file:codeframe/lib/checkpoint_manager.py (lines 54-649)

---

### 4. Implement Database Operation Benchmarks

**Objective:** Benchmark database queries with large datasets and concurrent operations.

**Actions:**
- Create `tests/performance/test_database_operations.py` with benchmark tests:
  
  **Test: Query Performance with 10,000+ Tasks**
  - Use `TaskRepository` methods from file:codeframe/persistence/database.py
  - Pre-populate database with 10,000 tasks across multiple projects
  - Benchmark queries:
    - `get_pending_tasks()` with filters
    - `get_tasks_by_issue()` with pagination
    - `get_recently_completed_tasks()` with date ranges
  - Assert query time < 100ms for indexed queries, < 500ms for complex joins
  
  **Test: Concurrent Read/Write Performance**
  - Use async database operations from file:codeframe/persistence/database.py
  - Simulate 10 concurrent agents reading/writing context items
  - Use `asyncio.gather()` to run operations in parallel
  - Benchmark throughput (operations/second)
  - Assert no deadlocks, consistent results
  
  **Test: Context Item Queries (1000+ items)**
  - Use `ContextRepository.list_context_items()` from file:codeframe/persistence/database.py
  - Pre-populate with 10,000 context items across 10 agents
  - Benchmark filtered queries by tier, agent_id, importance_score
  - Assert query time < 50ms with proper indexes
  
  **Test: Checkpoint Queries**
  - Use `CheckpointRepository` methods from file:codeframe/persistence/database.py
  - Pre-populate with 100 checkpoints
  - Benchmark `get_checkpoints()` and `get_checkpoint_by_id()`
  - Assert query time < 20ms

**New file:**
- `tests/performance/test_database_operations.py`

**Reference implementations:**
- file:codeframe/persistence/database.py (lines 260-677)
- Repository pattern documentation: `docs/architecture/database-repository-pattern.md`

---

### 5. Implement API Endpoint Load Tests

**Objective:** Load test API endpoints and WebSocket broadcasts using locust.

**Actions:**
- Create `tests/performance/locustfile.py` for load testing:
  
  **User Class: APIUser**
  - Simulates typical API usage patterns
  - Tasks:
    - GET `/api/projects` (weight: 10)
    - GET `/api/agents` (weight: 8)
    - GET `/api/tasks?project_id=1` (weight: 15)
    - GET `/api/context?agent_id=test-001` (weight: 12)
    - POST `/api/checkpoints` (weight: 2)
    - GET `/health` (weight: 5)
  - Wait time: 1-3 seconds between requests
  
  **User Class: WebSocketUser**
  - Simulates WebSocket connections for real-time updates
  - Connects to `/ws` endpoint from file:codeframe/ui/routers/websocket.py
  - Subscribes to project updates
  - Measures broadcast latency (time from server event to client receipt)
  - Target: <100ms broadcast latency
  
  **Load Test Scenarios:**
  - Scenario 1: 100 concurrent users, 10 req/s sustained load (5 minutes)
  - Scenario 2: 1000 concurrent WebSocket connections
  - Scenario 3: Spike test (0→500 users in 30s, hold 2 min, ramp down)
  
- Create `tests/performance/run_load_tests.sh` script:
  - Starts FastAPI server in background
  - Waits for server health check
  - Runs locust in headless mode with different scenarios
  - Captures results to `tests/performance/load_test_results/`
  - Stops server after tests
  
- Create `tests/performance/test_api_performance.py` for pytest-based API benchmarks:
  - Use `TestClient` from FastAPI for synchronous benchmarks
  - Benchmark individual endpoint response times
  - Assert 95th percentile < 500ms, 99th percentile < 2s

**New files:**
- `tests/performance/locustfile.py`
- `tests/performance/run_load_tests.sh`
- `tests/performance/test_api_performance.py`

**Reference implementations:**
- file:codeframe/ui/server.py (lines 1-371)
- file:codeframe/ui/routers/websocket.py

---

### 6. Implement Quality Gate Benchmarks

**Objective:** Benchmark quality gate execution times.

**Actions:**
- Create `tests/performance/test_quality_gates.py` with benchmark tests:
  
  **Test: Lint Analysis Time (Target: <2min)**
  - Use `QualityGates.run_linting_gate()` from file:codeframe/lib/quality_gates.py
  - Set up test project with 100 Python files (10,000 LOC)
  - Benchmark ruff execution time
  - Assert mean < 120s
  
  **Test: Test Execution Overhead (Target: <10% of test time)**
  - Use `TestRunner.run_tests()` from file:codeframe/testing/test_runner.py
  - Measure baseline test execution time (direct pytest)
  - Measure test execution through TestRunner
  - Calculate overhead percentage
  - Assert overhead < 10%
  
  **Test: Coverage Calculation Time**
  - Use `QualityGates.run_coverage_gate()` from file:codeframe/lib/quality_gates.py
  - Benchmark pytest with coverage on 100-file project
  - Assert mean < 180s (3 minutes)
  
  **Test: Review Agent Analysis (Target: <30s per file)**
  - Use `QualityGates.run_review_gate()` from file:codeframe/lib/quality_gates.py
  - Mock Review Agent to avoid LLM calls (focus on orchestration overhead)
  - Benchmark review workflow for 10 files
  - Assert mean < 30s per file
  
  **Test: All Gates Orchestration**
  - Use `QualityGates.run_all_gates()` from file:codeframe/lib/quality_gates.py
  - Benchmark full quality gate pipeline
  - Measure individual gate times and total time
  - Assert total time < 5 minutes for typical task

**New file:**
- `tests/performance/test_quality_gates.py`

**Reference implementations:**
- file:codeframe/lib/quality_gates.py (lines 107-1085)
- file:codeframe/testing/test_runner.py (lines 1-187)

---

### 7. Establish Performance Baselines

**Objective:** Run all benchmarks to establish initial baselines.

**Actions:**
- Run all performance tests with baseline capture:
  ```bash
  pytest tests/performance/ --benchmark-only --benchmark-save=baseline --benchmark-json=tests/performance/baseline_results.json
  ```
- Review baseline results and adjust targets if needed
- Document baseline environment:
  - Hardware specs (CPU, RAM, disk type)
  - Python version
  - Database size
  - OS and kernel version
- Store baselines in `tests/performance/baselines.json` with metadata:
  ```json
  {
    "environment": {
      "python_version": "3.11.x",
      "cpu": "...",
      "ram": "...",
      "date": "2025-12-26"
    },
    "baselines": {
      "context_load_1000_items": {"mean": 0.15, "max": 0.35},
      "flash_save": {"mean": 1.5, "max": 3.2},
      ...
    }
  }
  ```
- Create `tests/performance/BASELINES.md` documenting:
  - Baseline values for each operation
  - Target thresholds (from ticket)
  - Critical thresholds (20% above target)
  - Instructions for updating baselines

**New files:**
- `tests/performance/baseline_results.json`
- `tests/performance/BASELINES.md`

---

### 8. Integrate Performance Tests into CI

**Objective:** Add performance regression detection to GitHub Actions workflow.

**Actions:**
- Update file:.github/workflows/test.yml to add new job:
  
  ```yaml
  performance-tests:
    name: Performance Benchmarks
    runs-on: ubuntu-latest
    needs: backend-tests
    # Only run on main branch or when performance tests are modified
    if: github.ref == 'refs/heads/main' || contains(github.event.head_commit.modified, 'tests/performance/')
    
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: ${{ env.PYTHON_VERSION }}
      
      - name: Install uv
        uses: astral-sh/setup-uv@v4
        with:
          enable-cache: true
      
      - name: Install dependencies
        run: |
          uv venv
          uv sync --extra dev
      
      - name: Run performance benchmarks
        run: |
          uv run pytest tests/performance/ \
            --benchmark-only \
            --benchmark-compare=baseline \
            --benchmark-compare-fail=mean:20% \
            --benchmark-json=performance_results.json
      
      - name: Check for performance regressions
        run: |
          # Fail if any benchmark regressed by >20%
          python scripts/check_performance_regression.py performance_results.json
      
      - name: Upload performance results
        uses: actions/upload-artifact@v4
        with:
          name: performance-results
          path: performance_results.json
          retention-days: 90
      
      - name: Comment PR with performance results
        if: github.event_name == 'pull_request'
        uses: actions/github-script@v7
        with:
          script: |
            const fs = require('fs');
            const results = JSON.parse(fs.readFileSync('performance_results.json'));
            // Format and post comment with performance comparison
  ```

- Create `scripts/check_performance_regression.py`:
  - Parses pytest-benchmark JSON output
  - Compares against baselines
  - Fails CI if any benchmark regressed >20%
  - Generates markdown report for PR comments
  
- Update `pytest.ini` to add performance test markers:
  ```ini
  markers =
      performance: marks tests as performance benchmarks
      load_test: marks tests as load tests (requires running server)
  ```

**Files to modify:**
- file:.github/workflows/test.yml
- file:pytest.ini

**New files:**
- `scripts/check_performance_regression.py`

---

### 9. Create Performance Dashboard

**Objective:** Generate HTML reports showing performance trends over time.

**Actions:**
- Create `scripts/generate_performance_report.py`:
  - Reads historical performance results from CI artifacts
  - Generates HTML dashboard with charts:
    - Line charts showing performance trends over time
    - Bar charts comparing current vs baseline
    - Heatmap showing which operations are slowest
    - Table with all benchmark results and thresholds
  - Uses matplotlib or plotly for visualizations
  - Outputs to `tests/performance/reports/index.html`
  
- Create `tests/performance/reports/.gitkeep` to track directory
  
- Add GitHub Actions step to publish performance reports:
  ```yaml
  - name: Generate performance report
    run: python scripts/generate_performance_report.py
  
  - name: Deploy to GitHub Pages
    if: github.ref == 'refs/heads/main'
    uses: peaceiris/actions-gh-pages@v3
    with:
      github_token: ${{ secrets.GITHUB_TOKEN }}
      publish_dir: tests/performance/reports
  ```

**New files:**
- `scripts/generate_performance_report.py`
- `tests/performance/reports/.gitkeep`

---

### 10. Document Performance Testing

**Objective:** Create comprehensive documentation for performance testing.

**Actions:**
- Update file:TESTING.md to add "Performance Testing" section:
  - Overview of performance testing strategy
  - How to run performance tests locally
  - How to interpret benchmark results
  - How to update baselines
  - Performance targets and thresholds
  - Link to performance dashboard
  
- Create `docs/performance-testing.md` with detailed guide:
  - Architecture of performance testing infrastructure
  - Benchmark test patterns and best practices
  - Load testing scenarios and how to add new ones
  - CI integration and regression detection
  - Troubleshooting performance issues
  - Performance optimization tips
  
- Add performance testing to file:README.md:
  ```markdown
  ## Performance Testing
  
  Run performance benchmarks:
  ```bash
  pytest tests/performance/ --benchmark-only
  ```
  
  Run load tests:
  ```bash
  ./tests/performance/run_load_tests.sh
  ```
  
  View performance dashboard: [Performance Reports](https://frankbria.github.io/codeframe/performance/)
  ```

**Files to modify:**
- file:TESTING.md
- file:README.md

**New files:**
- `docs/performance-testing.md`

---

## Performance Targets Summary

| Operation | Target | Critical Threshold | Test Location |
|-----------|--------|-------------------|---------------|
| Task execution | <30s | <60s | `test_agent_operations.py` |
| Context load (1000 items) | <200ms | <500ms | `test_agent_operations.py` |
| Flash save | <2s | <5s | `test_agent_operations.py` |
| Checkpoint create | <10s | <30s | `test_agent_operations.py` |
| Checkpoint restore | <30s | <60s | `test_agent_operations.py` |
| Database query (10k tasks) | <100ms | <500ms | `test_database_operations.py` |
| API response (95th %ile) | <500ms | <2s | `test_api_performance.py` |
| WebSocket broadcast | <100ms | <500ms | `locustfile.py` |
| Lint analysis | <2min | <5min | `test_quality_gates.py` |
| Test execution overhead | <10% | <20% | `test_quality_gates.py` |
| Review agent analysis | <30s/file | <2min/file | `test_quality_gates.py` |

---

## Architecture Diagram

```mermaid
graph TD
    A[Performance Testing Infrastructure] --> B[Unit Benchmarks]
    A --> C[Load Tests]
    A --> D[CI Integration]
    
    B --> B1[Agent Operations<br/>pytest-benchmark]
    B --> B2[Database Operations<br/>pytest-benchmark]
    B --> B3[Quality Gates<br/>pytest-benchmark]
    
    C --> C1[API Endpoints<br/>locust]
    C --> C2[WebSocket Broadcasts<br/>locust]
    
    D --> D1[GitHub Actions<br/>Regression Detection]
    D --> D2[Performance Dashboard<br/>Trend Visualization]
    D --> D3[PR Comments<br/>Performance Comparison]
    
    B1 --> E[Baseline Storage<br/>baselines.json]
    B2 --> E
    B3 --> E
    C1 --> E
    C2 --> E
    
    E --> D1
    D1 --> D2
    D1 --> D3
    
    style A fill:#e1f5ff
    style B fill:#fff4e1
    style C fill:#ffe1f5
    style D fill:#e1ffe1
```

---

## Acceptance Criteria Checklist

- [ ] Performance benchmarks for all core operations (agent, database, API, quality gates)
- [ ] Load tests for API endpoints and WebSocket broadcasts using locust
- [ ] Baselines established for regression detection (stored in `baselines.json`)
- [ ] CI fails on performance regressions >20% (GitHub Actions job)
- [ ] Performance reports generated per commit (HTML dashboard)
- [ ] Dashboard shows performance trends (GitHub Pages deployment)
- [ ] All targets from ticket met or documented if not achievable
- [ ] Documentation updated (TESTING.md, README.md, performance-testing.md)

---

Execution Information

Branch: main
Commit: 1169cb6