Skip to content

Add critical user journey E2E tests (login, project creation, start agent) #157

Description

@frankbria

Overview

Current E2E tests validate that features work, but don't test that users can access them through the UI. Tests bypass authentication and project creation via database seeding, leaving critical user journeys untested.

Problem

Global Setup Bypasses UI (tests/e2e/global-setup.ts):

  • Auth: Pre-seeds session token instead of testing login flow
  • Projects: Creates via direct database insertion instead of UI
  • Agents: Backend tests call agent.execute_task() directly, no UI interaction

Missing User Journeys:

  1. ❌ No login flow test (/login page interaction)
  2. ❌ No project creation test (root / → create project form)
  3. ❌ No "start agent" interaction test (discovery UI → execute button)

Risk: If login or project creation breaks, beta testers can't even access the features we've thoroughly tested.

Required Tests

Test 1: Authentication Flow

File: tests/e2e/test_auth_flow.spec.ts

test('should render login page', async ({ page }) => {
  await page.goto('/login');
  await expect(page.locator('[data-testid="email-input"]')).toBeVisible();
  await expect(page.locator('[data-testid="password-input"]')).toBeVisible();
  await expect(page.locator('[data-testid="login-button"]')).toBeVisible();
});

test('should login successfully with valid credentials', async ({ page }) => {
  await page.goto('/login');
  await page.fill('[data-testid="email-input"]', 'test@example.com');
  await page.fill('[data-testid="password-input"]', 'TestPassword123!');
  await page.click('[data-testid="login-button"]');
  
  // Should redirect to root or dashboard
  await expect(page).toHaveURL(/\/projects|\/$/);
  await expect(page.locator('[data-testid="user-menu"]')).toBeVisible();
});

test('should show error with invalid credentials', async ({ page }) => {
  await page.goto('/login');
  await page.fill('[data-testid="email-input"]', 'test@example.com');
  await page.fill('[data-testid="password-input"]', 'WrongPassword');
  await page.click('[data-testid="login-button"]');
  
  await expect(page.locator('[data-testid="auth-error"]')).toContainText('Invalid credentials');
  await expect(page).toHaveURL('/login'); // Should stay on login page
});

test('should logout successfully', async ({ page }) => {
  // Login first
  await loginUser(page);
  
  // Logout
  await page.click('[data-testid="user-menu"]');
  await page.click('[data-testid="logout-button"]');
  
  await expect(page).toHaveURL('/login');
});

Test 2: Project Creation Flow

File: tests/e2e/test_project_creation.spec.ts

test('should display root page with create project option', async ({ page }) => {
  await loginUser(page);
  await page.goto('/');
  
  await expect(page.locator('[data-testid="create-project-button"]')).toBeVisible();
  await expect(page.locator('[data-testid="project-list"]')).toBeVisible();
});

test('should create new project via UI', async ({ page }) => {
  await loginUser(page);
  await page.goto('/');
  
  // Click create project
  await page.click('[data-testid="create-project-button"]');
  
  // Fill form
  await expect(page.locator('[data-testid="project-name-input"]')).toBeVisible();
  await page.fill('[data-testid="project-name-input"]', 'My E2E Test Project');
  await page.fill('[data-testid="project-description-input"]', 'Created via E2E test');
  
  // Submit
  await page.click('[data-testid="create-project-submit"]');
  
  // Should redirect to new project dashboard
  await expect(page).toHaveURL(/\/projects\/\d+/);
  await expect(page.locator('[data-testid="project-name"]')).toContainText('My E2E Test Project');
  await expect(page.locator('[data-testid="dashboard-header"]')).toBeVisible();
});

test('should validate project name is required', async ({ page }) => {
  await loginUser(page);
  await page.goto('/');
  await page.click('[data-testid="create-project-button"]');
  
  // Try to submit without name
  await page.click('[data-testid="create-project-submit"]');
  
  await expect(page.locator('[data-testid="form-error"]')).toContainText('Project name is required');
});

Test 3: Start Agent Interaction

File: tests/e2e/test_start_agent_flow.spec.ts

test('should start Socratic discovery from dashboard', async ({ page }) => {
  const projectId = await createTestProject(page);
  await page.goto(`/projects/${projectId}`);
  
  // Click start discovery button
  const startButton = page.locator('[data-testid="start-discovery-button"]');
  await expect(startButton).toBeVisible();
  await startButton.click();
  
  // Should show first discovery question
  await expect(page.locator('[data-testid="discovery-question"]')).toBeVisible();
  await expect(page.locator('[data-testid="discovery-answer-input"]')).toBeVisible();
});

test('should answer discovery questions and generate PRD', async ({ page }) => {
  const projectId = await createTestProject(page);
  await page.goto(`/projects/${projectId}`);
  
  // Start discovery
  await page.click('[data-testid="start-discovery-button"]');
  
  // Answer questions (simulate 3 questions)
  for (let i = 0; i < 3; i++) {
    await page.fill('[data-testid="discovery-answer-input"]', `Answer ${i + 1}`);
    await page.click('[data-testid="submit-answer-button"]');
    await page.waitForTimeout(500); // Wait for next question
  }
  
  // Should show PRD generation or completion
  await expect(page.locator('[data-testid="prd-generated"]')).toBeVisible({ timeout: 10000 });
});

test('should execute tasks after discovery completion', async ({ page }) => {
  const projectId = await createTestProjectWithDiscovery(page);
  await page.goto(`/projects/${projectId}`);
  
  // Click execute/start agents button
  const executeButton = page.locator('[data-testid="start-execution-button"]');
  await expect(executeButton).toBeVisible();
  await executeButton.click();
  
  // Should show agents in progress
  await expect(page.locator('[data-testid="agent-status-panel"]')).toBeVisible();
  await expect(page.locator('[data-testid="agent-status"]')).toContainText('in_progress');
});

Test 4: Complete User Journey (Smoke Test)

File: tests/e2e/test_complete_user_journey.spec.ts

test('should complete full workflow from login to agent execution', async ({ page }) => {
  // 1. Login
  await page.goto('/login');
  await page.fill('[data-testid="email-input"]', 'test@example.com');
  await page.fill('[data-testid="password-input"]', 'TestPassword123!');
  await page.click('[data-testid="login-button"]');
  await expect(page).toHaveURL(/\//);
  
  // 2. Create project
  await page.click('[data-testid="create-project-button"]');
  await page.fill('[data-testid="project-name-input"]', 'Journey Test Project');
  await page.click('[data-testid="create-project-submit"]');
  await expect(page).toHaveURL(/\/projects\/\d+/);
  
  // 3. Start discovery
  await page.click('[data-testid="start-discovery-button"]');
  
  // 4. Answer discovery questions
  await page.fill('[data-testid="discovery-answer-input"]', 'Build a REST API');
  await page.click('[data-testid="submit-answer-button"]');
  await page.waitForSelector('[data-testid="discovery-question"]');
  await page.fill('[data-testid="discovery-answer-input"]', 'Python with FastAPI');
  await page.click('[data-testid="submit-answer-button"]');
  
  // 5. Wait for PRD generation (or skip remaining questions if implemented)
  await expect(page.locator('[data-testid="prd-generated"]')).toBeVisible({ timeout: 15000 });
  
  // 6. Start execution
  await page.click('[data-testid="start-execution-button"]');
  
  // 7. Verify agents are running
  await expect(page.locator('[data-testid="agent-status"]')).toContainText('in_progress', { timeout: 10000 });
  
  // 8. Verify dashboard features are accessible
  await expect(page.locator('[data-testid="metrics-panel"]')).toBeVisible();
  await expect(page.locator('[data-testid="review-findings-panel"]')).toBeVisible();
});

Test Utilities

Create helper functions in tests/e2e/test-utils.ts:

export async function loginUser(page: Page, email = 'test@example.com', password = 'TestPassword123!') {
  await page.goto('/login');
  await page.fill('[data-testid="email-input"]', email);
  await page.fill('[data-testid="password-input"]', password);
  await page.click('[data-testid="login-button"]');
  await page.waitForURL(/\//, { timeout: 10000 });
}

export async function createTestProject(page: Page, name = 'Test Project'): Promise<string> {
  await page.goto('/');
  await page.click('[data-testid="create-project-button"]');
  await page.fill('[data-testid="project-name-input"]', name);
  await page.click('[data-testid="create-project-submit"]');
  
  // Extract project ID from URL
  await page.waitForURL(/\/projects\/\d+/);
  const url = page.url();
  const match = url.match(/\/projects\/(\d+)/);
  return match ? match[1] : '1';
}

export async function answerDiscoveryQuestion(page: Page, answer: string) {
  await page.fill('[data-testid="discovery-answer-input"]', answer);
  await page.click('[data-testid="submit-answer-button"]');
  await page.waitForTimeout(500);
}

Acceptance Criteria

  • test_auth_flow.spec.ts created with 4 tests (login success/failure, logout, render)
  • test_project_creation.spec.ts created with 3 tests (create, validation, list)
  • test_start_agent_flow.spec.ts created with 3 tests (discovery start, answer, execute)
  • test_complete_user_journey.spec.ts created with 1 comprehensive smoke test
  • Helper utilities added to test-utils.ts
  • All tests pass on Chromium, Firefox, and WebKit
  • Tests run in CI without flakiness
  • Test coverage for /login, / (root), and project dashboard interaction flows

Implementation Notes

Required UI Test IDs

Ensure the following data-testid attributes exist in the frontend:

Login Page (/login):

  • email-input
  • password-input
  • login-button
  • auth-error (error message container)

Root Page (/):

  • create-project-button
  • project-list
  • project-name-input
  • project-description-input
  • create-project-submit
  • form-error

Project Dashboard:

  • start-discovery-button
  • discovery-question
  • discovery-answer-input
  • submit-answer-button
  • prd-generated
  • start-execution-button

User Menu:

  • user-menu
  • logout-button

Test Data Management

  • Create dedicated test user in global setup: test@example.com / TestPassword123!
  • Clean up test projects after each test to avoid database bloat
  • Use unique project names with timestamps to avoid conflicts

Priority Justification

P1-high-beta because:

  1. First User Interaction: Login is the FIRST thing beta testers will try
  2. Blocking if Broken: If login/project creation fails, users can't access ANY features
  3. High Risk: Currently untested - we don't know if these flows work end-to-end
  4. Quick Wins: These tests are straightforward to implement (~4-6 hours total)

Related Issues

Estimated Effort

4-6 hours total:

  • 1-2 hours: Add test IDs to frontend components
  • 2-3 hours: Write 11 test cases
  • 1 hour: Create helper utilities and test data fixtures

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1-high-betaHigh priority - should fix before beta for best experienceenhancementNew feature or requesttesting

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions