You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Current E2E tests validate that features work, but don't test that users can access them through the UI. Tests bypass authentication and project creation via database seeding, leaving critical user journeys untested.
Problem
Global Setup Bypasses UI (tests/e2e/global-setup.ts):
Auth: Pre-seeds session token instead of testing login flow
Projects: Creates via direct database insertion instead of UI
Agents: Backend tests call agent.execute_task() directly, no UI interaction
Missing User Journeys:
❌ No login flow test (/login page interaction)
❌ No project creation test (root / → create project form)
❌ No "start agent" interaction test (discovery UI → execute button)
Risk: If login or project creation breaks, beta testers can't even access the features we've thoroughly tested.
Required Tests
Test 1: Authentication Flow
File:tests/e2e/test_auth_flow.spec.ts
test('should render login page',async({ page })=>{awaitpage.goto('/login');awaitexpect(page.locator('[data-testid="email-input"]')).toBeVisible();awaitexpect(page.locator('[data-testid="password-input"]')).toBeVisible();awaitexpect(page.locator('[data-testid="login-button"]')).toBeVisible();});test('should login successfully with valid credentials',async({ page })=>{awaitpage.goto('/login');awaitpage.fill('[data-testid="email-input"]','test@example.com');awaitpage.fill('[data-testid="password-input"]','TestPassword123!');awaitpage.click('[data-testid="login-button"]');// Should redirect to root or dashboardawaitexpect(page).toHaveURL(/\/projects|\/$/);awaitexpect(page.locator('[data-testid="user-menu"]')).toBeVisible();});test('should show error with invalid credentials',async({ page })=>{awaitpage.goto('/login');awaitpage.fill('[data-testid="email-input"]','test@example.com');awaitpage.fill('[data-testid="password-input"]','WrongPassword');awaitpage.click('[data-testid="login-button"]');awaitexpect(page.locator('[data-testid="auth-error"]')).toContainText('Invalid credentials');awaitexpect(page).toHaveURL('/login');// Should stay on login page});test('should logout successfully',async({ page })=>{// Login firstawaitloginUser(page);// Logoutawaitpage.click('[data-testid="user-menu"]');awaitpage.click('[data-testid="logout-button"]');awaitexpect(page).toHaveURL('/login');});
Test 2: Project Creation Flow
File:tests/e2e/test_project_creation.spec.ts
test('should display root page with create project option',async({ page })=>{awaitloginUser(page);awaitpage.goto('/');awaitexpect(page.locator('[data-testid="create-project-button"]')).toBeVisible();awaitexpect(page.locator('[data-testid="project-list"]')).toBeVisible();});test('should create new project via UI',async({ page })=>{awaitloginUser(page);awaitpage.goto('/');// Click create projectawaitpage.click('[data-testid="create-project-button"]');// Fill formawaitexpect(page.locator('[data-testid="project-name-input"]')).toBeVisible();awaitpage.fill('[data-testid="project-name-input"]','My E2E Test Project');awaitpage.fill('[data-testid="project-description-input"]','Created via E2E test');// Submitawaitpage.click('[data-testid="create-project-submit"]');// Should redirect to new project dashboardawaitexpect(page).toHaveURL(/\/projects\/\d+/);awaitexpect(page.locator('[data-testid="project-name"]')).toContainText('My E2E Test Project');awaitexpect(page.locator('[data-testid="dashboard-header"]')).toBeVisible();});test('should validate project name is required',async({ page })=>{awaitloginUser(page);awaitpage.goto('/');awaitpage.click('[data-testid="create-project-button"]');// Try to submit without nameawaitpage.click('[data-testid="create-project-submit"]');awaitexpect(page.locator('[data-testid="form-error"]')).toContainText('Project name is required');});
Test 3: Start Agent Interaction
File:tests/e2e/test_start_agent_flow.spec.ts
test('should start Socratic discovery from dashboard',async({ page })=>{constprojectId=awaitcreateTestProject(page);awaitpage.goto(`/projects/${projectId}`);// Click start discovery buttonconststartButton=page.locator('[data-testid="start-discovery-button"]');awaitexpect(startButton).toBeVisible();awaitstartButton.click();// Should show first discovery questionawaitexpect(page.locator('[data-testid="discovery-question"]')).toBeVisible();awaitexpect(page.locator('[data-testid="discovery-answer-input"]')).toBeVisible();});test('should answer discovery questions and generate PRD',async({ page })=>{constprojectId=awaitcreateTestProject(page);awaitpage.goto(`/projects/${projectId}`);// Start discoveryawaitpage.click('[data-testid="start-discovery-button"]');// Answer questions (simulate 3 questions)for(leti=0;i<3;i++){awaitpage.fill('[data-testid="discovery-answer-input"]',`Answer ${i+1}`);awaitpage.click('[data-testid="submit-answer-button"]');awaitpage.waitForTimeout(500);// Wait for next question}// Should show PRD generation or completionawaitexpect(page.locator('[data-testid="prd-generated"]')).toBeVisible({timeout: 10000});});test('should execute tasks after discovery completion',async({ page })=>{constprojectId=awaitcreateTestProjectWithDiscovery(page);awaitpage.goto(`/projects/${projectId}`);// Click execute/start agents buttonconstexecuteButton=page.locator('[data-testid="start-execution-button"]');awaitexpect(executeButton).toBeVisible();awaitexecuteButton.click();// Should show agents in progressawaitexpect(page.locator('[data-testid="agent-status-panel"]')).toBeVisible();awaitexpect(page.locator('[data-testid="agent-status"]')).toContainText('in_progress');});
Test 4: Complete User Journey (Smoke Test)
File:tests/e2e/test_complete_user_journey.spec.ts
test('should complete full workflow from login to agent execution',async({ page })=>{// 1. Loginawaitpage.goto('/login');awaitpage.fill('[data-testid="email-input"]','test@example.com');awaitpage.fill('[data-testid="password-input"]','TestPassword123!');awaitpage.click('[data-testid="login-button"]');awaitexpect(page).toHaveURL(/\//);// 2. Create projectawaitpage.click('[data-testid="create-project-button"]');awaitpage.fill('[data-testid="project-name-input"]','Journey Test Project');awaitpage.click('[data-testid="create-project-submit"]');awaitexpect(page).toHaveURL(/\/projects\/\d+/);// 3. Start discoveryawaitpage.click('[data-testid="start-discovery-button"]');// 4. Answer discovery questionsawaitpage.fill('[data-testid="discovery-answer-input"]','Build a REST API');awaitpage.click('[data-testid="submit-answer-button"]');awaitpage.waitForSelector('[data-testid="discovery-question"]');awaitpage.fill('[data-testid="discovery-answer-input"]','Python with FastAPI');awaitpage.click('[data-testid="submit-answer-button"]');// 5. Wait for PRD generation (or skip remaining questions if implemented)awaitexpect(page.locator('[data-testid="prd-generated"]')).toBeVisible({timeout: 15000});// 6. Start executionawaitpage.click('[data-testid="start-execution-button"]');// 7. Verify agents are runningawaitexpect(page.locator('[data-testid="agent-status"]')).toContainText('in_progress',{timeout: 10000});// 8. Verify dashboard features are accessibleawaitexpect(page.locator('[data-testid="metrics-panel"]')).toBeVisible();awaitexpect(page.locator('[data-testid="review-findings-panel"]')).toBeVisible();});
Test Utilities
Create helper functions in tests/e2e/test-utils.ts:
exportasyncfunctionloginUser(page: Page,email='test@example.com',password='TestPassword123!'){awaitpage.goto('/login');awaitpage.fill('[data-testid="email-input"]',email);awaitpage.fill('[data-testid="password-input"]',password);awaitpage.click('[data-testid="login-button"]');awaitpage.waitForURL(/\//,{timeout: 10000});}exportasyncfunctioncreateTestProject(page: Page,name='Test Project'): Promise<string>{awaitpage.goto('/');awaitpage.click('[data-testid="create-project-button"]');awaitpage.fill('[data-testid="project-name-input"]',name);awaitpage.click('[data-testid="create-project-submit"]');// Extract project ID from URLawaitpage.waitForURL(/\/projects\/\d+/);consturl=page.url();constmatch=url.match(/\/projects\/(\d+)/);returnmatch ? match[1] : '1';}exportasyncfunctionanswerDiscoveryQuestion(page: Page,answer: string){awaitpage.fill('[data-testid="discovery-answer-input"]',answer);awaitpage.click('[data-testid="submit-answer-button"]');awaitpage.waitForTimeout(500);}
Acceptance Criteria
test_auth_flow.spec.ts created with 4 tests (login success/failure, logout, render)
test_project_creation.spec.ts created with 3 tests (create, validation, list)
test_start_agent_flow.spec.ts created with 3 tests (discovery start, answer, execute)
test_complete_user_journey.spec.ts created with 1 comprehensive smoke test
Helper utilities added to test-utils.ts
All tests pass on Chromium, Firefox, and WebKit
Tests run in CI without flakiness
Test coverage for /login, / (root), and project dashboard interaction flows
Implementation Notes
Required UI Test IDs
Ensure the following data-testid attributes exist in the frontend:
Login Page (/login):
email-input
password-input
login-button
auth-error (error message container)
Root Page (/):
create-project-button
project-list
project-name-input
project-description-input
create-project-submit
form-error
Project Dashboard:
start-discovery-button
discovery-question
discovery-answer-input
submit-answer-button
prd-generated
start-execution-button
User Menu:
user-menu
logout-button
Test Data Management
Create dedicated test user in global setup: test@example.com / TestPassword123!
Clean up test projects after each test to avoid database bloat
Use unique project names with timestamps to avoid conflicts
Priority Justification
P1-high-beta because:
First User Interaction: Login is the FIRST thing beta testers will try
Blocking if Broken: If login/project creation fails, users can't access ANY features
High Risk: Currently untested - we don't know if these flows work end-to-end
Quick Wins: These tests are straightforward to implement (~4-6 hours total)
Overview
Current E2E tests validate that features work, but don't test that users can access them through the UI. Tests bypass authentication and project creation via database seeding, leaving critical user journeys untested.
Problem
Global Setup Bypasses UI (
tests/e2e/global-setup.ts):agent.execute_task()directly, no UI interactionMissing User Journeys:
/loginpage interaction)/→ create project form)Risk: If login or project creation breaks, beta testers can't even access the features we've thoroughly tested.
Required Tests
Test 1: Authentication Flow
File:
tests/e2e/test_auth_flow.spec.tsTest 2: Project Creation Flow
File:
tests/e2e/test_project_creation.spec.tsTest 3: Start Agent Interaction
File:
tests/e2e/test_start_agent_flow.spec.tsTest 4: Complete User Journey (Smoke Test)
File:
tests/e2e/test_complete_user_journey.spec.tsTest Utilities
Create helper functions in
tests/e2e/test-utils.ts:Acceptance Criteria
test_auth_flow.spec.tscreated with 4 tests (login success/failure, logout, render)test_project_creation.spec.tscreated with 3 tests (create, validation, list)test_start_agent_flow.spec.tscreated with 3 tests (discovery start, answer, execute)test_complete_user_journey.spec.tscreated with 1 comprehensive smoke testtest-utils.ts/login,/(root), and project dashboard interaction flowsImplementation Notes
Required UI Test IDs
Ensure the following
data-testidattributes exist in the frontend:Login Page (
/login):email-inputpassword-inputlogin-buttonauth-error(error message container)Root Page (
/):create-project-buttonproject-listproject-name-inputproject-description-inputcreate-project-submitform-errorProject Dashboard:
start-discovery-buttondiscovery-questiondiscovery-answer-inputsubmit-answer-buttonprd-generatedstart-execution-buttonUser Menu:
user-menulogout-buttonTest Data Management
test@example.com/TestPassword123!Priority Justification
P1-high-beta because:
Related Issues
Estimated Effort
4-6 hours total: