Add warning banner for gate status false positives

[frankbria]

Problem

The getGateStatus() function in QualityGatesPanel currently shows ALL gates as "passed" when overall status is "passed", even if only some gates have actually run. This can create false positives that mask incomplete quality checks.

Current Behavior:

• If only tests gate has run and passed, ALL 5 gates (tests, coverage, type-check, lint, review) show as "passed"
• Users may incorrectly assume all quality checks have been completed

Root Cause

Backend doesn't support gates_evaluated field to track which gates actually ran (see TODO comment in QualityGatesPanel.tsx:67-74).

Proposed Solutions

Short-term (UI Enhancement)

Add a warning banner when:

• Overall status is passed
• No failures exist (empty failures array)
• To alert users this is a summary view and not all gates may have been evaluated

Example Banner:

⚠️ Note: Showing summary status. Some individual gates may not have been evaluated yet.

Long-term (Backend Enhancement)

Implement gates_evaluated: string[] field in QualityGateStatus API response to track which gates actually ran.

Acceptance Criteria

Short-term:

• Add warning banner component to QualityGatesPanel
• Show banner only when overall status is "passed" but individual gate evaluation is uncertain
• Banner should be dismissible or auto-hide after user acknowledgment

Long-term:

• Backend returns gates_evaluated array in QualityGateStatus response
• Frontend uses gates_evaluated to accurately determine gate status
• Remove warning banner once backend enhancement is complete

Related

• PR feat: Implement Quality Gates Panel in Dashboard (#43) #50 (Quality Gates Panel implementation)
• Code review feedback: Known limitation in getGateStatus()

[traycerai]

Plan

Observations

The QualityGateStatus component (file:web-ui/src/components/quality-gates/QualityGateStatus.tsx) currently displays "All quality gates passed! Ready for deployment." when status === 'passed' and failures.length === 0 (lines 307-316). However, the backend doesn't track which gates actually ran via a gates_evaluated field, creating potential false positives. The codebase follows consistent Tailwind CSS patterns for warning banners (bg-yellow-50 border border-yellow-200) as seen in file:web-ui/src/components/checkpoints/CheckpointRestore.tsx. The component uses React hooks for state management and conditional rendering.

Approach

Implement a dismissible warning banner that appears when overall status is "passed" with no failures, alerting users that not all gates may have been evaluated. Use localStorage to persist dismissal state per task, following the codebase's existing patterns for banners and state management. The banner will be positioned between the status header and the success message, maintaining visual hierarchy. This short-term solution addresses the false positive issue until the backend implements gates_evaluated tracking.

Implementation Steps

1. Add Warning Banner State Management

In file:web-ui/src/components/quality-gates/QualityGateStatus.tsx:

• Import useState and useEffect (already imported)
• Add new state variable warningDismissed using useState<boolean>(false)
• Add useEffect hook to check localStorage on mount for dismissal state using key qualityGates_warning_dismissed_${taskId}
• Create helper function dismissWarning() that:
  • Sets warningDismissed to true
  • Stores dismissal in localStorage with the task-specific key
  • Uses localStorage.setItem() with JSON stringified value

2. Create Warning Banner Component

Add new banner section in the component's return statement (after line 247, before the "Running Progress Indicator" section):

• Conditionally render when:
  • status.status === 'passed'
  • (!status.failures || status.failures.length === 0)
  • !warningDismissed
• Use consistent styling pattern: bg-yellow-50 border border-yellow-200 rounded-lg p-4
• Structure with flex layout containing:
  • Warning icon (⚠️) with text-yellow-600 text-xl
  • Message text container with:
    • Bold heading: "Summary Status Only"
    • Description: "This shows overall status. Individual gates may not have been evaluated yet."
  • Dismiss button (×) with:
    • onClick={dismissWarning}
    • Styling: text-yellow-600 hover:text-yellow-800 font-bold text-xl
    • Accessible label: aria-label="Dismiss warning"

3. Update Success Message Positioning

Modify the existing success message section (lines 307-316):

• Keep the same conditional logic
• Ensure it renders after the warning banner
• No changes to content or styling needed

4. Add Test Coverage

In file:web-ui/tests/components/QualityGateStatus.test.tsx:

Add new test suite "Warning Banner" with tests for:

Test: "should display warning banner when status is passed with no failures"

• Mock fetchQualityGateStatus to return passed status with empty failures
• Render component
• Assert warning banner is visible with text matching "Summary Status Only"

Test: "should not display warning banner when status is failed"

• Mock status as 'failed'
• Render component
• Assert warning banner is not in document

Test: "should not display warning banner when status is running"

• Mock status as 'running'
• Render component
• Assert warning banner is not in document

Test: "should dismiss warning banner when close button clicked"

• Mock passed status
• Render component
• Find and click dismiss button
• Assert warning banner is no longer visible
• Verify localStorage.setItem was called with correct key

Test: "should not show warning banner if previously dismissed"

• Mock localStorage.getItem to return dismissed state
• Mock passed status
• Render component
• Assert warning banner is not visible

Test: "should use task-specific localStorage key"

• Render component with taskId=123
• Click dismiss button
• Verify localStorage key includes task ID: qualityGates_warning_dismissed_123

5. Update Type Definitions (Optional Enhancement)

In file:web-ui/src/types/qualityGates.ts:

Add comment documenting the limitation:

// TODO: Backend enhancement needed
// Add gates_evaluated?: QualityGateType[] to track which gates actually ran
// This will eliminate the need for the warning banner

Visual Structure

sequenceDiagram
    participant User
    participant Component
    participant LocalStorage
    
    User->>Component: Views QualityGateStatus
    Component->>LocalStorage: Check dismissal state
    LocalStorage-->>Component: Return dismissed status
    
    alt Not dismissed & status=passed & no failures
        Component->>User: Display warning banner
        User->>Component: Click dismiss button
        Component->>LocalStorage: Store dismissal (task-specific key)
        Component->>User: Hide warning banner
    else Already dismissed or other status
        Component->>User: No warning banner shown
    end

Loading

Component Layout

Section	Condition	Visual
Status Header	Always shown	White card with status badge
Warning Banner	passed + no failures + not dismissed	Yellow banner with dismiss button
Running Progress	status === 'running'	Yellow progress indicator
Failures List	failures.length > 0	Red failure cards
Success Message	passed + no failures	Green success message

Implementation Notes

• Dismissal Persistence: Use task-specific localStorage keys to allow per-task dismissal rather than global
• Accessibility: Include aria-label on dismiss button for screen readers
• Styling Consistency: Follow existing yellow warning pattern from file:web-ui/src/components/checkpoints/CheckpointRestore.tsx (lines 114-137)
• State Management: Use React hooks pattern consistent with existing component structure
• Testing: Mock localStorage in tests using jest.spyOn(Storage.prototype, 'getItem') and jest.spyOn(Storage.prototype, 'setItem')
• Future Removal: Add TODO comment indicating this is temporary until backend implements gates_evaluated field

Import In IDE

🤖 Prompt for AI Agents

## Observations

The `QualityGateStatus` component (file:web-ui/src/components/quality-gates/QualityGateStatus.tsx) currently displays "All quality gates passed! Ready for deployment." when `status === 'passed'` and `failures.length === 0` (lines 307-316). However, the backend doesn't track which gates actually ran via a `gates_evaluated` field, creating potential false positives. The codebase follows consistent Tailwind CSS patterns for warning banners (`bg-yellow-50 border border-yellow-200`) as seen in file:web-ui/src/components/checkpoints/CheckpointRestore.tsx. The component uses React hooks for state management and conditional rendering.

## Approach

Implement a dismissible warning banner that appears when overall status is "passed" with no failures, alerting users that not all gates may have been evaluated. Use localStorage to persist dismissal state per task, following the codebase's existing patterns for banners and state management. The banner will be positioned between the status header and the success message, maintaining visual hierarchy. This short-term solution addresses the false positive issue until the backend implements `gates_evaluated` tracking.

## Implementation Steps

### 1. Add Warning Banner State Management

In file:web-ui/src/components/quality-gates/QualityGateStatus.tsx:

- Import `useState` and `useEffect` (already imported)
- Add new state variable `warningDismissed` using `useState<boolean>(false)`
- Add `useEffect` hook to check localStorage on mount for dismissal state using key `qualityGates_warning_dismissed_${taskId}`
- Create helper function `dismissWarning()` that:
  - Sets `warningDismissed` to `true`
  - Stores dismissal in localStorage with the task-specific key
  - Uses `localStorage.setItem()` with JSON stringified value

### 2. Create Warning Banner Component

Add new banner section in the component's return statement (after line 247, before the "Running Progress Indicator" section):

- Conditionally render when:
  - `status.status === 'passed'`
  - `(!status.failures || status.failures.length === 0)`
  - `!warningDismissed`
- Use consistent styling pattern: `bg-yellow-50 border border-yellow-200 rounded-lg p-4`
- Structure with flex layout containing:
  - Warning icon (⚠️) with `text-yellow-600 text-xl`
  - Message text container with:
    - Bold heading: "Summary Status Only"
    - Description: "This shows overall status. Individual gates may not have been evaluated yet."
  - Dismiss button (×) with:
    - `onClick={dismissWarning}`
    - Styling: `text-yellow-600 hover:text-yellow-800 font-bold text-xl`
    - Accessible label: `aria-label="Dismiss warning"`

### 3. Update Success Message Positioning

Modify the existing success message section (lines 307-316):

- Keep the same conditional logic
- Ensure it renders after the warning banner
- No changes to content or styling needed

### 4. Add Test Coverage

In file:web-ui/__tests__/components/QualityGateStatus.test.tsx:

Add new test suite "Warning Banner" with tests for:

**Test: "should display warning banner when status is passed with no failures"**
- Mock `fetchQualityGateStatus` to return passed status with empty failures
- Render component
- Assert warning banner is visible with text matching "Summary Status Only"

**Test: "should not display warning banner when status is failed"**
- Mock status as 'failed'
- Render component
- Assert warning banner is not in document

**Test: "should not display warning banner when status is running"**
- Mock status as 'running'
- Render component
- Assert warning banner is not in document

**Test: "should dismiss warning banner when close button clicked"**
- Mock passed status
- Render component
- Find and click dismiss button
- Assert warning banner is no longer visible
- Verify localStorage.setItem was called with correct key

**Test: "should not show warning banner if previously dismissed"**
- Mock localStorage.getItem to return dismissed state
- Mock passed status
- Render component
- Assert warning banner is not visible

**Test: "should use task-specific localStorage key"**
- Render component with taskId=123
- Click dismiss button
- Verify localStorage key includes task ID: `qualityGates_warning_dismissed_123`

### 5. Update Type Definitions (Optional Enhancement)

In file:web-ui/src/types/qualityGates.ts:

Add comment documenting the limitation:
```
// TODO: Backend enhancement needed
// Add gates_evaluated?: QualityGateType[] to track which gates actually ran
// This will eliminate the need for the warning banner
```

## Visual Structure

```mermaid
sequenceDiagram
    participant User
    participant Component
    participant LocalStorage
    
    User->>Component: Views QualityGateStatus
    Component->>LocalStorage: Check dismissal state
    LocalStorage-->>Component: Return dismissed status
    
    alt Not dismissed & status=passed & no failures
        Component->>User: Display warning banner
        User->>Component: Click dismiss button
        Component->>LocalStorage: Store dismissal (task-specific key)
        Component->>User: Hide warning banner
    else Already dismissed or other status
        Component->>User: No warning banner shown
    end
```

## Component Layout

| Section | Condition | Visual |
|---------|-----------|--------|
| Status Header | Always shown | White card with status badge |
| **Warning Banner** | **passed + no failures + not dismissed** | **Yellow banner with dismiss button** |
| Running Progress | status === 'running' | Yellow progress indicator |
| Failures List | failures.length > 0 | Red failure cards |
| Success Message | passed + no failures | Green success message |

## Implementation Notes

- **Dismissal Persistence**: Use task-specific localStorage keys to allow per-task dismissal rather than global
- **Accessibility**: Include `aria-label` on dismiss button for screen readers
- **Styling Consistency**: Follow existing yellow warning pattern from file:web-ui/src/components/checkpoints/CheckpointRestore.tsx (lines 114-137)
- **State Management**: Use React hooks pattern consistent with existing component structure
- **Testing**: Mock localStorage in tests using `jest.spyOn(Storage.prototype, 'getItem')` and `jest.spyOn(Storage.prototype, 'setItem')`
- **Future Removal**: Add TODO comment indicating this is temporary until backend implements `gates_evaluated` field

---

Execution Information

Branch: main
Commit: 969b10b

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[frankbria]

Closed with PR #76