fix: Resolve E2E test timeout failures in GitHub Actions - #35
Conversation
Implements comprehensive fixes for E2E backend and frontend test failures caused by server startup timeouts. **Root Cause:** - `uv run uvicorn` triggers package rebuild on every invocation (20-30s overhead) - 60-second timeout insufficient with rebuild time - Poor error visibility and process verification **Changes:** Backend E2E Tests: - Pre-install package with `uv pip install -e .` to eliminate runtime rebuild - Initialize database explicitly before server startup - Replace `uv run` with direct venv python invocation - Increase health check timeout from 60s to 120s - Add retry loop with progress feedback (120 attempts @ 1s) - Verify server process didn't die immediately (5s check) - Capture server logs to /tmp/server.log for debugging - Upload server logs as artifacts (7-day retention) - Use curl instead of npx wait-on for simpler dependencies Frontend E2E Tests: - Apply same backend server startup fixes - Sequential server startup (backend → frontend) - Separate health checks with proper timeouts (120s backend, 60s frontend) - Capture both backend and frontend logs separately - Upload both log files as artifacts - Better error messages showing which server failed Environment Configuration: - Set DATABASE_PATH explicitly to workspace location - Set WORKSPACE_ROOT to github.workspace - Set CODEFRAME_DEPLOYMENT_MODE=self_hosted **Expected Impact:** - Reduces server startup time by 20-30 seconds - Provides 120-second total timeout (was 60s) - Better diagnostics with server logs on failure - Immediate crash detection before timeout - Clearer error messages and progress feedback **Testing:** - Will validate in CI on next push - Local server startup confirmed working (<5s after package install) - Health endpoint confirmed functional Closes issues with E2E test timeouts in GitHub Actions.
WalkthroughThe changes enhance GitHub Actions E2E workflows by introducing explicit server lifecycle management: editable package installation, structured startup sequences with background process verification, health-check polling loops with timeouts, and comprehensive server log artifact uploads. These improvements replace simpler implicit startup flows across backend, frontend, and Playwright test jobs. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes
Poem
Pre-merge checks and finishing touches✅ Passed checks (3 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
.github/workflows/test.yml (2)
243-258: Inconsistent readiness-check exit patterns.Backend uses
exit 0on success followed by a fallthrough failure case, while frontend backend usesbreak+ inlineif [ $i -eq 120 ]check. Both are functionally correct, but the backend pattern is clearer.Consider unifying to the backend pattern (lines 243–258) for consistency:
- name: Wait for backend to be ready run: | echo "Waiting for backend to start..." - for i in {1..120}; do + for i in {1..120}; do if curl -s http://localhost:8080/health > /dev/null; then echo "✅ Backend is ready!" curl -s http://localhost:8080/health | jq . - break + exit 0 fi echo "Attempt $i/120: Backend not ready yet..." sleep 1 - if [ $i -eq 120 ]; then - echo "❌ Backend failed to start within 120 seconds" - echo "=== Backend Logs ===" - cat /tmp/backend.log - exit 1 - fi done + echo "❌ Backend failed to start within 120 seconds" + echo "=== Backend Logs ===" + cat /tmp/backend.log + exit 1Similarly, update frontend readiness check (lines 393–409) for consistency.
Also applies to: 367-384
215-220: Minor: Database path uses relative path; environment variable uses absolute path.Line 219 initializes the database at
.codeframe/state.db(relative), while the environment variable (line 224) points to${{ github.workspace }}/.codeframe/state.db(absolute). Since the workflow runs from the workspace root, these are equivalent, but mixing relative and absolute paths is a minor clarity issue.For consistency, consider making both absolute or both relative. If using an absolute path, update line 219:
- python -c "from codeframe.persistence.database import Database; db = Database('.codeframe/state.db'); db.initialize(); db.close()" + python -c "from codeframe.persistence.database import Database; db = Database('${{ github.workspace }}/.codeframe/state.db'); db.initialize(); db.close()"Alternatively, reference the environment variable (if the Database class respects it), or ensure the working directory is always the workspace root before relative paths.
E2E_TESTS_FIX_PLAN.md (1)
100-138: Documentation example paths differ from actual workflow implementation.The plan shows example environment variables (lines 127–129) using
/tmp/paths:DATABASE_PATH: /tmp/codeframe_test.db WORKSPACE_ROOT: /tmp/codeframe_workspaceHowever, the actual workflow (.github/workflows/test.yml, lines 224–226) uses workspace-relative paths:
DATABASE_PATH: ${{ github.workspace }}/.codeframe/state.db WORKSPACE_ROOT: ${{ github.workspace }}The actual implementation is more appropriate for GitHub Actions, but the discrepancy may confuse readers attempting to follow the plan. Consider updating the documentation example to match the actual implementation for clarity.
Update lines 127–128 in the plan to reflect the actual implementation:
- DATABASE_PATH: /tmp/codeframe_test.db - WORKSPACE_ROOT: /tmp/codeframe_workspace + DATABASE_PATH: ${{ github.workspace }}/.codeframe/state.db + WORKSPACE_ROOT: ${{ github.workspace }}
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
.github/workflows/test.yml(4 hunks)E2E_TESTS_FIX_PLAN.md(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
**/*.md
📄 CodeRabbit inference engine (AGENTS.md)
Documentation files must be sized to fit in a single agent context window (spec.md ~200-400 lines, plan.md ~300-600 lines, tasks.md ~400-800 lines)
Files:
E2E_TESTS_FIX_PLAN.md
🧠 Learnings (4)
📚 Learning: 2025-11-25T19:08:20.036Z
Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to codeframe/**/*.py : Use FastAPI for backend API implementation
Applied to files:
.github/workflows/test.yml
📚 Learning: 2025-11-25T19:08:20.036Z
Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to codeframe/**/*.py : Use Python 3.11+ with async/await pattern, type hints, and comprehensive tests for backend code
Applied to files:
.github/workflows/test.yml
📚 Learning: 2025-11-25T19:08:20.036Z
Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to tests/e2e/**/*.py : Use TestSprite MCP for E2E test generation and Playwright for frontend E2E testing
Applied to files:
.github/workflows/test.yml
📚 Learning: 2025-11-25T19:08:20.036Z
Learnt from: CR
Repo: frankbria/codeframe PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-11-25T19:08:20.036Z
Learning: Applies to web-ui/**/*.{ts,tsx,test.ts,test.tsx} : Run frontend tests with npm test from web-ui directory
Applied to files:
.github/workflows/test.yml
🪛 LanguageTool
E2E_TESTS_FIX_PLAN.md
[uncategorized] ~38-~38: The official name of this software platform is spelled with a capital “H”.
Context: ...kage to Avoid Runtime Rebuild File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~57-~57: The official name of this software platform is spelled with a capital “H”.
Context: ...t Timeout and Add Retry Logic File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~84-~84: The official name of this software platform is spelled with a capital “H”.
Context: ...h Check Endpoint Verification File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~122-~122: The official name of this software platform is spelled with a capital “H”.
Context: ...ronment Variables for Testing File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~142-~142: The official name of this software platform is spelled with a capital “H”.
Context: ...apture on Success and Failure File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~157-~157: The official name of this software platform is spelled with a capital “H”.
Context: ... Add Health Check Diagnostics File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~181-~181: The official name of this software platform is spelled with a capital “H”.
Context: ... Same Fixes to Frontend Tests File: .github/workflows/test.yml Changes: Apply ...
(GITHUB)
[uncategorized] ~185-~185: The official name of this software platform is spelled with a capital “H”.
Context: ...d Server Starts After Backend File: .github/workflows/test.yml Changes: ```yam...
(GITHUB)
[uncategorized] ~222-~222: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...1**: Pre-install package ✅ High impact, low risk 2. Task 1.2: Increase timeout + ret...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~223-~223: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...Increase timeout + retry ✅ High impact, low risk 3. Task 1.3: Verify server startup ...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~228-~228: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...Set environment variables ✅ Low impact, low risk 6. Task 4.1: Apply fixes to fronten...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
[uncategorized] ~264-~264: The official name of this software platform is spelled with a capital “H”.
Context: ... If any changes cause issues: 1. Revert .github/workflows/test.yml to previous version...
(GITHUB)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: Frontend Unit Tests
- GitHub Check: Backend Unit Tests
🔇 Additional comments (9)
.github/workflows/test.yml (5)
210-213: ✅ Editable package installation prevents rebuild overhead.Pre-installing the package in editable mode (before server startup) eliminates the 20–30s rebuild time that was triggering timeouts. This is the primary fix and is well-placed in the workflow.
Also applies to: 334-337
215-220: ✅ Database initialization is explicit and well-sequenced.Database is initialized once before server startup, preventing migrations and schema issues during the server's startup sequence. The
mkdir -p .codeframeensures the directory exists.Also applies to: 339-344
222-231: ✅ Server startup via venv Python avoidsuv runoverhead.Using
source .venv/bin/activatefollowed bypython -m uvicornis cleaner and faster thanuv run uvicorn. The PID is captured for later verification and cleanup.
233-241: ✅ Early process verification detects immediate crashes.The 5-second sleep +
ps -pcheck catches server crashes before attempting the full 120-second health-check loop, improving diagnostics and reducing wasted wait time.
283-289: ✅ Log uploads properly configured with appropriate retention.Server and frontend logs are captured and uploaded as artifacts with 7-day retention for debugging. This is consistent across both E2E jobs and aligns with the fix plan.
Also applies to: 426-440
E2E_TESTS_FIX_PLAN.md (4)
1-33: ✅ Clear problem analysis and root-cause identification.The PR summary correctly identifies the core issue:
uv run uvicorntriggers package rebuilds (20–30s overhead), causing the 60s timeout to be exceeded. The root-cause analysis is thorough and well-structured.
35-177: ✅ Phased implementation plan is well-reasoned and prioritized.The four-phase approach (immediate, next sprint, optional, frontend) is sensible. Each task includes clear rationale and risk assessment. Phase 1 (pre-install, retry loop, process verification) addresses the critical timeout issue, while later phases add diagnostics and robustness.
219-235: ✅ Implementation order is pragmatic and risk-aware.Phase 1 (immediate, low-risk) focuses on the critical timeout fix. Phases 2–4 add enhancements with clear ordering. Risk mitigation for each phase is explicitly noted.
236-307: ✅ Success criteria, testing strategy, and rollback plan are comprehensive.The document includes clear success metrics (tests <3 min / <5 min, server starts <30s), local and CI testing approaches, rollback procedures, and realistic timeline estimates (11–16 hours). This level of detail supports smooth execution and debugging.
1 participant
Problem
E2E backend and frontend tests were consistently failing in GitHub Actions with timeout errors:
Root Cause:
uv run uvicorntriggers a package rebuild on every invocation, adding 20-30 seconds to startup time. Combined with the 60-second timeout, the server couldn't start in time.Solution
This PR implements comprehensive fixes to resolve the timeout issues:
Backend E2E Tests
uv pip install -e .(eliminates rebuild overhead)uv runFrontend E2E Tests
Environment Configuration
DATABASE_PATHto${{ github.workspace }}/.codeframe/state.dbWORKSPACE_ROOTto${{ github.workspace }}CODEFRAME_DEPLOYMENT_MODE=self_hostedChanges
.github/workflows/test.yml: Updated E2E test jobs with timeout fixesE2E_TESTS_FIX_PLAN.md: Full implementation plan and documentationExpected Impact
Testing
/healthreturns JSON)Success Criteria
Related Issues
Closes timeout issues in E2E tests reported in recent CI runs.
Checklist
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings.