Skip to content

[P6.8.3] Reconcile CI -m v2 blind spot: ~535 tests (40% of files) never run in CI #669

Description

@frankbria

Problem

CI's backend job runs pytest tests/ --ignore=tests/e2e -m v2 (.github/workflows/test.yml:226). The root tests/conftest.py only auto-marks integration/slownot v2. So -m v2 deselects ~535 tests (3159 run / 3694 collected), and roughly 40% of test files carry no v2 marker at all. Those tests never gate a PR.

This is a silent coverage blind spot. It hides two different things:

  1. Dead tests — e.g. the orphaned v1 /api/projects tests removed in chore(tests): remove orphaned v1 tests for removed /api/projects endpoint #667 fail with 404 on a full local run but were invisible to CI because they lack the marker.
  2. Potentially-legitimate tests that simply were never marked v2 and so never run in CI (some tests/unit, tests/contract, tests/planning, tests/cli, tests/api, root test_new_feature.py, etc. — needs auditing).

Why it matters

The whole point of the v2 filter was to skip v1-legacy reference tests during the v1→v2 refactor. That refactor is done. Keeping a filter that silently drops 40% of files means real tests aren't protecting main, and dead tests rot undetected. This is the same class of risk as the disabled E2E jobs (#647).

Proposed work

  1. Run the deselected set: uv run pytest tests/ --ignore=tests/e2e -m "not v2" -q and categorize each failure/file as dead (remove) vs legit (keep).
  2. For legit tests: add the v2 marker (or, better, decide the filter is obsolete and drop -m v2 so CI runs all non-e2e tests). Auto-marking by directory in conftest.py is an option to avoid per-file churn.
  3. Delete remaining dead v1 tests (the /api/projects pair is already handled in chore(tests): remove orphaned v1 tests for removed /api/projects endpoint #667).
  4. Make CI green on whatever the new selection is, then remove/relax the -m v2 filter so the suite can't silently shrink again.

Acceptance criteria

  • The 535 deselected tests are categorized (dead vs legit) and the dead ones removed.
  • Legit tests run in CI (marked, or filter dropped).
  • CI no longer silently excludes ~40% of test files; the selection is explicit and green.

Source: discovered during #659 / #667 (2026-06-13). Related: #647 (disabled E2E gates).

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1-high-betaHigh priority - should fix before beta for best experienceci-cdphase-6.8Phase 6.8: Test coverage hardeningtesting

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions