feat(validation): P3-4 robustness matrix (universes x windows OOS cells)

AGENTS.md

@@ -107,6 +107,15 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
   - 报告 `phase3_oos_stability.md`:split 边界+跨界行披露/分期绩效(annual/vol/sharpe/maxDD/turnover)/逐序列 IC 分期(mean/IR/hit rate/sign consistency)/权重稳定性(每期权重含 train-test 标注、trained 行 sign flips、fallback 次数+原因)/小样本 caveat。
   - **真实结果（关键发现,~16min,77 成分/2 年,持有窗口切片）**:三个原始因子 train→test **IC 全部翻号**(momentum −0.023→+0.006 / roe −0.029→+0.007 / np_yoy −0.011→+0.005,sign consistency 全 NO),hit rate 46~53%≈抛硬币;权重 23 期 sign flips 7/3/4;绩效 eq train −11.92%/test −5.27%,ic train −8.31%/test −2.70%(跨界行 2023-06-30 排除;修切片前 train 被 test 期收益污染到 −6.81%/−1.69%,修正幅度本身就是边界 bug 的实证)。**结论:ic_weighted 两段都略好但 IC≈0 且翻号——P3-2 单年跑赢不可外推,这正是本验证层要拿到的证据;非收益声明。**
   - **回归不破**:phase3 equal_weight rerun −9.05%/0.0083、ic_weighted rerun −3.57% 均不变;demo 0.96/0.84 不变;secret scan 报告 0 处 token/config.json。
-- ✅ 质量门：`pytest` **285 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16）；`ruff` clean；`validate-config`（demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml`）+ `run-phase0`（demo）均 OK。
-- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、因子 IC 小样本不稳定（P3-3 实证）。
-- 路线图下一步：更长历史/更宽 universe 的稳定性复检,或分钟级（architecture.html §11）。
+- 🔧 **Phase 3-4 robustness matrix**（`p3-robustness-matrix` 分支,代劳待验收）：把 P3-3 OOS 检验批量跑到 **universe × window 矩阵**上,回答 P3-2/P3-3 结论是否 SSE50 小样本偶然。不加因子/alpha、不改 portfolio/execution/factor math。
+  - 新 run mode `run-phase3-robustness`（`qt/robustness.py`）+ `config/phase3_real_robustness_matrix.yaml`（SSE50+CSI300 × 2020-2022/2022-2024 两 fold;`skip_cells` 显式跳过 CSI300×2020-2022——runtime 预算,报告披露,绝不静默缩覆盖）。
+  - **每个 cell 逐字复用 P3-3 cell 核心**（`_run_oos_cell` 重构抽出,单 run 行为不变;持有窗口切片/实现日 IC 切片/walk-forward/ic_weighted guard 全继承）;cell config 仅替换 universe/窗口/split/output_name（重过全套 pydantic 验证,parquet 不互覆盖）;跨 cell 汇总严格按 cell 归属（测试锁定不串数）。
+  - **真实矩阵结果**（3 cells,总 2h:960s/934s/5301s,CSI300 399 名）:
+    - **P3-3 不漂移 smoke ✓**:SSE50|2022-2024 cell 逐数复现 P3-3 报告（−11.92%/−5.27%/−8.31%,boundary 2023-06-30）。
+    - 原始因子 train→test sign consistency:momentum 1/3 / roe 0/3 / np_yoy 0/3——**IC 翻号跨 cell 普遍,P3-3 结论非偶然**。
+    - `combo_ic_weighted` 是唯一跨 cell 稳定序列（test IC 3/3 正 + 3/3 sign consistent）但量级 ≈0.004~0.008 极小。
+    - **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**（sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重）。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。**
+  - secret scan 报告 0 处;demo 0.96/0.84 不变。
+- ✅ 质量门：`pytest` **299 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=14）；`ruff` clean；`validate-config`（旧 6 配置 + `phase3_real_robustness_matrix.yaml`）+ `run-phase0`（demo）均 OK。
+- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、三因子组 IC 跨期翻号且 ic_weighted 宽 universe 样本外崩（P3-3/P3-4 实证——当前因子组不具备可交易信号强度）。
+- 路线图下一步：换/扩因子集（在既有验证框架下迭代）,或分钟级（architecture.html §11）。

CLAUDE.md

@@ -107,6 +107,15 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
   - 报告 `phase3_oos_stability.md`:split 边界+跨界行披露/分期绩效(annual/vol/sharpe/maxDD/turnover)/逐序列 IC 分期(mean/IR/hit rate/sign consistency)/权重稳定性(每期权重含 train-test 标注、trained 行 sign flips、fallback 次数+原因)/小样本 caveat。
   - **真实结果（关键发现,~16min,77 成分/2 年,持有窗口切片）**:三个原始因子 train→test **IC 全部翻号**(momentum −0.023→+0.006 / roe −0.029→+0.007 / np_yoy −0.011→+0.005,sign consistency 全 NO),hit rate 46~53%≈抛硬币;权重 23 期 sign flips 7/3/4;绩效 eq train −11.92%/test −5.27%,ic train −8.31%/test −2.70%(跨界行 2023-06-30 排除;修切片前 train 被 test 期收益污染到 −6.81%/−1.69%,修正幅度本身就是边界 bug 的实证)。**结论:ic_weighted 两段都略好但 IC≈0 且翻号——P3-2 单年跑赢不可外推,这正是本验证层要拿到的证据;非收益声明。**
   - **回归不破**:phase3 equal_weight rerun −9.05%/0.0083、ic_weighted rerun −3.57% 均不变;demo 0.96/0.84 不变;secret scan 报告 0 处 token/config.json。
-- ✅ 质量门：`pytest` **285 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16）；`ruff` clean；`validate-config`（demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml`）+ `run-phase0`（demo）均 OK。
-- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、因子 IC 小样本不稳定（P3-3 实证）。
-- 路线图下一步：更长历史/更宽 universe 的稳定性复检,或分钟级（architecture.html §11）。
+- 🔧 **Phase 3-4 robustness matrix**（`p3-robustness-matrix` 分支,代劳待验收）：把 P3-3 OOS 检验批量跑到 **universe × window 矩阵**上,回答 P3-2/P3-3 结论是否 SSE50 小样本偶然。不加因子/alpha、不改 portfolio/execution/factor math。
+  - 新 run mode `run-phase3-robustness`（`qt/robustness.py`）+ `config/phase3_real_robustness_matrix.yaml`（SSE50+CSI300 × 2020-2022/2022-2024 两 fold;`skip_cells` 显式跳过 CSI300×2020-2022——runtime 预算,报告披露,绝不静默缩覆盖）。
+  - **每个 cell 逐字复用 P3-3 cell 核心**（`_run_oos_cell` 重构抽出,单 run 行为不变;持有窗口切片/实现日 IC 切片/walk-forward/ic_weighted guard 全继承）;cell config 仅替换 universe/窗口/split/output_name（重过全套 pydantic 验证,parquet 不互覆盖）;跨 cell 汇总严格按 cell 归属（测试锁定不串数）。
+  - **真实矩阵结果**（3 cells,总 2h:960s/934s/5301s,CSI300 399 名）:
+    - **P3-3 不漂移 smoke ✓**:SSE50|2022-2024 cell 逐数复现 P3-3 报告（−11.92%/−5.27%/−8.31%,boundary 2023-06-30）。
+    - 原始因子 train→test sign consistency:momentum 1/3 / roe 0/3 / np_yoy 0/3——**IC 翻号跨 cell 普遍,P3-3 结论非偶然**。
+    - `combo_ic_weighted` 是唯一跨 cell 稳定序列（test IC 3/3 正 + 3/3 sign consistent）但量级 ≈0.004~0.008 极小。
+    - **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**（sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重）。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。**
+  - secret scan 报告 0 处;demo 0.96/0.84 不变。
+- ✅ 质量门：`pytest` **299 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=14）；`ruff` clean；`validate-config`（旧 6 配置 + `phase3_real_robustness_matrix.yaml`）+ `run-phase0`（demo）均 OK。
+- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、三因子组 IC 跨期翻号且 ic_weighted 宽 universe 样本外崩（P3-3/P3-4 实证——当前因子组不具备可交易信号强度）。
+- 路线图下一步：换/扩因子集（在既有验证框架下迭代）,或分钟级（architecture.html §11）。

RUNBOOK.md

@@ -308,6 +308,46 @@ ic_weighted weight stability (per-rebalance weights with train/test labels,
 sign-flip counts on trained rows, fallback count + reasons); and the explicit
 caveat that this is a small-sample stability check, NOT a return claim.
 
+## Phase 3-4 — robustness matrix (longer history + wider universe)
+
+Re-runs the P3-3 OOS stability check on every (universe × window) cell to test
+whether the single-sample conclusions generalize. No new factor, no new alpha,
+no portfolio / execution / factor-math change. Documented by
+`config/phase3_real_robustness_matrix.yaml` (SSE50 + CSI300 × two 2-year
+folds 2020-2022 / 2022-2024; the CSI300×2020-2022 cell is explicitly
+`skip_cells`-listed for runtime budget and DISCLOSED in the report).
+
+```bash
+# validate (no network)
+... -m qt.cli validate-config        --config config/phase3_real_robustness_matrix.yaml
+# run the matrix (network + token; HEAVY — a CSI300 2y cell alone is ~60-90 min)
+... -m qt.cli run-phase3-robustness  --config config/phase3_real_robustness_matrix.yaml
+```
+
+Mechanics (locked by tests):
+
+- Every cell reuses the P3-3 cell core VERBATIM (`qt.oos_stability._run_oos_cell`):
+  holding-window performance slicing, realization-date IC slicing, walk-forward
+  weights, equal-weight fallback disclosure, and the shared guards (real source,
+  `oos` section, the ic_weighted fake-comparison guard). The single-run
+  `run-phase3-oos` behaviour is unchanged.
+- Per-cell configs are derived from the base with ONLY the cell identity
+  swapped (universe.index_code, data window, split, and a per-cell
+  `output_name` so panels never overwrite each other); the derived config
+  re-runs full pydantic validation.
+- `robustness.skip_cells` reduces coverage EXPLICITLY (validated against the
+  declared universes/windows; at least one cell must remain) and every skip is
+  disclosed in the report — coverage is never silently reduced.
+
+Report (`artifacts/reports/phase3_robustness_matrix.md`): the cell table
+(window / split / per-cell runtime) + skipped cells; a CROSS-CELL stability
+summary (per series: #cells with positive test IC, #cells train→test
+sign-consistent, test IC by cell; #cells where ic_weighted beats equal_weight
+on test annual); full per-cell diagnostics (subperiod performance, IC
+stability, weight sign flips, fallback counts, boundary rebalances); and the
+explicit caveat that this is a stability check — NOT a return claim, not a
+tuned result.
+
 ## Quality gate
 
 ```bash

TEST_REPORT.md

@@ -1,4 +1,4 @@
-# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability)
+# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix)
 
 ## Commands
 
@@ -13,19 +13,20 @@ Run from the repo root with the project python (env `quant_mf`):
 /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_multifactor.yaml
 /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_ic_weighted.yaml
 /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_oos_stability.yaml
+/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_robustness_matrix.yaml
 /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli run-phase0 --config config/example.yaml
 ```
 
 ## Results
 
 | Gate | Command | Result |
 |---|---|---|
-| Unit + integration | `pytest -q` | **285 passed, 0 failed** |
+| Unit + integration | `pytest -q` | **299 passed, 0 failed** |
 | Lint | `ruff check .` | **All checks passed** |
-| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml`) | exit `0`, prints `OK` |
+| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml`) | exit `0`, prints `OK` |
 | End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` |
 
-Counts below are the actual per-file `pytest` numbers (sum = 285).
+Counts below are the actual per-file `pytest` numbers (sum = 299).
 
 ## Per-file breakdown — Phase 0 core (97)
 
@@ -116,7 +117,13 @@ Counts below are the actual per-file `pytest` numbers (sum = 285).
 | Test file | Tests | Red-line / feature |
 |---|---|---|
 | `test_oos_stability.py` | 16 | **split-boundary no-leakage**: perturbing every post-split forward return leaves all train-period weights bit-identical; **holding-window slicing** (`split_nav_by_holding`: train = holding end ≤ split, test = start ≥ split, straddlers + unknown-end rows excluded and disclosed); `subperiod_perf` rebased nav + empty-slice NaN; `ic_period_stats` sliced by realization date (t+h) with mean/IR/hit-rate/n; `sign_consistent` nonzero-same-sign; `weight_sign_flips` on trained rows only (fallback rows excluded); fallback-reason aggregation; OOS config validates + split-inside-window ConfigError; runner rejects demo source / missing `oos` section / **non-ic_weighted alpha (fake-comparison guard)**; report renders boundaries / straddler disclosure / OOS metrics / weight stability / caveat / no secret |
-| **Total (P0 + P1 + P2-1..P2-4 + P3-1 + P3-2 + P3-3)** | **285** | |
+
+## Per-file breakdown — Phase 3-4 robustness matrix (14)
+
+| Test file | Tests | Red-line / feature |
+|---|---|---|
+| `test_robustness_matrix.py` | 14 | matrix config validation (universes/windows/unique labels, split-inside-window, unknown-skip + all-skipped ConfigErrors); cell enumeration excludes skips in config order; per-cell config derivation swaps ONLY the cell identity (incl. unique per-cell output_name; derived cell passes the shared OOS preconditions); cross-cell aggregation attributes strictly per cell (no mixing, no dilution); runner guards (demo source / missing robustness section / non-ic_weighted alpha); report renders cell labels + skipped-cell disclosure + boundary/fallback/sign-consistency + cross-cell summary + caveat + no secret |
+| **Total (P0 + P1 + P2-1..P2-4 + P3-1 + P3-2 + P3-3 + P3-4)** | **299** | |
 
 ## Real-data validation (manual, not in CI — TEST-002 keeps the suite network-free)
 
@@ -200,6 +207,11 @@ Counts below are the actual per-file `pytest` numbers (sum = 285).
   proves post-split forward returns cannot move any train-period weight. The
   runner refuses a non-ic_weighted `alpha.model` (a fake comparison guard).
   Portfolio / execution / factor math are untouched.
+- **P3-4 robustness matrix (locked by tests):** `run-phase3-robustness` runs the
+  UNCHANGED P3-3 cell core on every (universe × window) cell; per-cell configs
+  derive from the base with only the cell identity swapped; skipped cells are
+  config-validated and disclosed (never silent); cross-cell aggregation is
+  strictly per cell. Single-run `run-phase3-oos` behaviour is unchanged.
 - A duplicate test-function name across two files was found and renamed during P2-2
   (it had been silently shadowing one test in the full-suite run). A second, harmless
   duplicate (`test_enrich_does_not_mutate_input` in two files) was verified NOT to drop