Restore and expand Chronicle cross-dataset evaluation

[anth-volk]

Fixes #157

Summary

• restore the Cross-dataset comparison as a complete-US-Chronicle evaluation surface
• add a reusable harness and source-specific adapters for Microcosm + PolicyEngine-US, Public CPS + Tax-Calculator, Raw ACS PUMS, and the Yale reconstruction
• expand Yale coverage to 382 facts, including 58 exact Microcosm-aligned 2022 facts, 232 aligned 2023 facts, and 92 native 2024 facts
• retain original observations while scoring aligned facts against their approved 2024 transformed benchmarks
• report fact-level capped mean error, coverage, threshold distributions, unavailable reasons, and fact drill-downs
• replace the overloaded group table with a compact responsive comparison matrix driven by container width
• provide a manually invoked maintenance script for the Python harness and numerical adapter gates instead of running them on every commit
• document the Yale checkpoint and Chronicle update policies and keep generated mappings reproducible

Why

The prior Cross-dataset page had been removed, and the available comparison logic did not provide a complete, auditable classification of Chronicle facts across heterogeneous datasets and model/dataset pairs. Period-aligned facts also require source-specific handling so 2022 and 2023 observations are not incorrectly compared directly with 2024 estimates.

The restored page now uses explicit adapter capabilities, immutable evaluation artifacts, approved Microcosm aging alignments, and a responsive layout that switches before horizontal scrolling begins. The model-backed Python verification is an explicit periodic maintenance operation; lightweight frontend checks remain in CI.

User impact

Users can compare error and coverage across all four current sources, group results by Chronicle source, period, or geography, and drill down to individual facts. The matrix keeps all four sources visible on standard displays and becomes a 2×2 or 1×4 source grid when its own content area narrows.

Future maintainers can rerun the locked harness checks with uv run python scripts/verify_evaluation_harness.py, optionally supplying pinned Microcosm and ACS inputs for their numerical gates.

Validation

• manual maintenance script — 377 Python tests passed and 10 finite Public CPS + Tax-Calculator results returned
• bun test — 82 passed
• bun run lint — TypeScript passed
• visually verified the five-column/compact boundary at a 960px matrix width and the 2×2/1×4 responsive states
• generated artifact run evaluation-02ba5db8abd8414c22eb6625

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
calibration-diagnostics	Ready	Preview	Aug 13, 2026 7:16pm