Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 9 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,12 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- combo_ic_weighted 独立 test IC **8/8 正**(4 组×2 cells,0.0060~0.0238)——walk-forward IC 加权在 holdout 上仍有效。
- **组合净值仍未确立**:独立 base ic test——legacy_trio **+1.15%/+8.13%**(2/2 正,恰是此前实证"无信号"的组!)、full_pack +7.30%/−1.93%、value_lowvol +1.72%/−4.68%、liq +3.53%/−0.39%——组间排名跨 cell 翻转,~21 rebalances 小样本,**IC 符号确认 ≠ 组合盈利**;成本阶梯仍全单调。非收益声明。
- secret scan 报告+日志 0 处;demo 0.96/0.84 不变。
- ✅ 质量门:`pytest` **375 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22 / P3-6=27 / P3-7=25+1 throttle);`ruff` clean;`validate-config`(全部 10 配置)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、旧三因子无信号(P3-3/P3-4 实证;但其组合在 2024-2026 holdout 上 2/2 正——小样本翻转的又一例证);value/低波信号获得**独立样本符号级确认但量级衰减**(P3-7),组合级盈利能力未确立;subset 报告文件名跨 config 共用(后跑覆盖前跑)。
- 路线图下一步:更长 holdout 积累(2026-06 之后滚动复检)/ 新 universe(中证500)/ 成本模型细化(印花税卖侧不对称、冲击成本),或分钟级(architecture.html §11)。
- ✅ **Phase 3-8 CSI500 独立泛化检验**(分支 `p3-csi500-value-lowvol-generalization`,**EXPLORATORY**):P3-7 的符号级结论是否泛化到筛选 universe 之外——新增独立 cell **CSI500(000905.SH)|2024-2026**(universe+时间双独立)。**机器零改动**复用 P3-7 层(同组/同成本场景/同假设,不加因子不调参);唯一代码变更 `output.subset_report_name`(沿 `baseline_report_name` 先例,默认 None 保持旧文件名 bitwise,测试锁定)——P3-8 报告独立成文件,**不再覆盖已验收的 P3-7 artifact**。
- cell 设计:SSE50|2022-2024(screened 锚,须 ≡P3-6/P3-7)+ SSE50|2024-2026(independent 锚,须 ≡P3-7 verdict 数字)+ CSI500|2024-2026(主问题);CSI500|2022-2024 skip 披露(~645 名×2y 再加 ~3h 不值)。数据可行性先实测:index_weight 月度 500 名快照到 2026-05-29 含 2024-06-28 pre-start 锚。
- **真实 run(3 cells/~3.55h,CSI500 735 distinct 名主导;一次跑通)双锚对账 ✓**:screened raw IC 22/22 ≡ P3-5 报告;independent SSE50 verdict IC 逐数 ≡ P3-7(+0.0322/+0.0134、+0.0379/+0.0033、−0.0320/−0.0120)——复现性二度确认。
- **CSI500 verdict:SUPPORTED**(21 settled vs min 8):value_ep **+0.0083/+0.0145**(train/test)、value_bp **+0.0230/+0.0127**、volatility_20 **−0.0350/−0.0272**——三假设双子期全保持。**且衰减更小**:CSI500 test 子期量级(0.0127~0.0272)明显高于 SSE50/CSI300 holdout 的后段(0.003~0.016)——value/低波信号在中盘股上更强,**P3-7 结论泛化成立(GENERALIZES,未减弱)**。
- combo_ic_weighted CSI500 test IC 全 4 组正(0.0243/0.0294/0.0286/0.0285,为三个 holdout cells 最高);**组合净值 CSI500 base 全组正且 4× 高成本仍全正**(trio +17.80%→+10.57% / full_pack +7.83%→+2.95% / value_lowvol +4.19%→+1.14% / liq +4.02%→+0.84%)——首个全成本阶梯为正的 cell。⚠️ 仍然诚实:trio +17.80% 又是组间排名跨 cell 翻转的例证(中盘动量 regime);单窗口 ~21 调仓小样本;**非收益声明**。
- **报告标题配置化**(`output.subset_report_title`,沿 `subset_report_name` 先例):P3-8 报告 H1 自报 study 名「Phase 3-8 — CSI500 Independent Generalization Check」而非机器默认 P3-7 标签(review 同类 stale-wording 问题修复,测试断言首行非 P3-7;P3-6/P3-7 配置不设此项、保持 sample-aware 默认,回归锁定);文案/分区检查全过(无 stale P3-6 措辞、verdict 节只含独立 cells);secret scan 报告+日志 0 处;demo 0.96/0.84 不变。
- ✅ 质量门:`pytest` **383 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22 / P3-6=27 / P3-7=25+1 throttle / P3-8=8);`ruff` clean;`validate-config`(全部 11 配置)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、旧三因子无信号(P3-3/P3-4 实证;但其组合在 2024-2026 holdout 上 SSE50/CSI300/CSI500 全正、CSI500 高达 +17.8%——小样本/regime 翻转的持续例证);value/低波信号获得**独立样本符号级确认**(P3-7 SSE50/CSI300 量级衰减;P3-8 CSI500 泛化成立且更强),组合级盈利能力仍未确立(排名跨 cell 翻转);subset 报告文件名已可配置(P3-8 起不再互覆盖)。
- 路线图下一步:更长 holdout 积累(2026-06 之后滚动复检)/ 中证1000 或全市场扩展 / 成本模型细化(印花税卖侧不对称、冲击成本),或分钟级(architecture.html §11)。
12 changes: 9 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,12 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- combo_ic_weighted 独立 test IC **8/8 正**(4 组×2 cells,0.0060~0.0238)——walk-forward IC 加权在 holdout 上仍有效。
- **组合净值仍未确立**:独立 base ic test——legacy_trio **+1.15%/+8.13%**(2/2 正,恰是此前实证"无信号"的组!)、full_pack +7.30%/−1.93%、value_lowvol +1.72%/−4.68%、liq +3.53%/−0.39%——组间排名跨 cell 翻转,~21 rebalances 小样本,**IC 符号确认 ≠ 组合盈利**;成本阶梯仍全单调。非收益声明。
- secret scan 报告+日志 0 处;demo 0.96/0.84 不变。
- ✅ 质量门:`pytest` **375 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22 / P3-6=27 / P3-7=25+1 throttle);`ruff` clean;`validate-config`(全部 10 配置)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、旧三因子无信号(P3-3/P3-4 实证;但其组合在 2024-2026 holdout 上 2/2 正——小样本翻转的又一例证);value/低波信号获得**独立样本符号级确认但量级衰减**(P3-7),组合级盈利能力未确立;subset 报告文件名跨 config 共用(后跑覆盖前跑)。
- 路线图下一步:更长 holdout 积累(2026-06 之后滚动复检)/ 新 universe(中证500)/ 成本模型细化(印花税卖侧不对称、冲击成本),或分钟级(architecture.html §11)。
- ✅ **Phase 3-8 CSI500 独立泛化检验**(分支 `p3-csi500-value-lowvol-generalization`,**EXPLORATORY**):P3-7 的符号级结论是否泛化到筛选 universe 之外——新增独立 cell **CSI500(000905.SH)|2024-2026**(universe+时间双独立)。**机器零改动**复用 P3-7 层(同组/同成本场景/同假设,不加因子不调参);唯一代码变更 `output.subset_report_name`(沿 `baseline_report_name` 先例,默认 None 保持旧文件名 bitwise,测试锁定)——P3-8 报告独立成文件,**不再覆盖已验收的 P3-7 artifact**。
- cell 设计:SSE50|2022-2024(screened 锚,须 ≡P3-6/P3-7)+ SSE50|2024-2026(independent 锚,须 ≡P3-7 verdict 数字)+ CSI500|2024-2026(主问题);CSI500|2022-2024 skip 披露(~645 名×2y 再加 ~3h 不值)。数据可行性先实测:index_weight 月度 500 名快照到 2026-05-29 含 2024-06-28 pre-start 锚。
- **真实 run(3 cells/~3.55h,CSI500 735 distinct 名主导;一次跑通)双锚对账 ✓**:screened raw IC 22/22 ≡ P3-5 报告;independent SSE50 verdict IC 逐数 ≡ P3-7(+0.0322/+0.0134、+0.0379/+0.0033、−0.0320/−0.0120)——复现性二度确认。
- **CSI500 verdict:SUPPORTED**(21 settled vs min 8):value_ep **+0.0083/+0.0145**(train/test)、value_bp **+0.0230/+0.0127**、volatility_20 **−0.0350/−0.0272**——三假设双子期全保持。**且衰减更小**:CSI500 test 子期量级(0.0127~0.0272)明显高于 SSE50/CSI300 holdout 的后段(0.003~0.016)——value/低波信号在中盘股上更强,**P3-7 结论泛化成立(GENERALIZES,未减弱)**。
- combo_ic_weighted CSI500 test IC 全 4 组正(0.0243/0.0294/0.0286/0.0285,为三个 holdout cells 最高);**组合净值 CSI500 base 全组正且 4× 高成本仍全正**(trio +17.80%→+10.57% / full_pack +7.83%→+2.95% / value_lowvol +4.19%→+1.14% / liq +4.02%→+0.84%)——首个全成本阶梯为正的 cell。⚠️ 仍然诚实:trio +17.80% 又是组间排名跨 cell 翻转的例证(中盘动量 regime);单窗口 ~21 调仓小样本;**非收益声明**。
- **报告标题配置化**(`output.subset_report_title`,沿 `subset_report_name` 先例):P3-8 报告 H1 自报 study 名「Phase 3-8 — CSI500 Independent Generalization Check」而非机器默认 P3-7 标签(review 同类 stale-wording 问题修复,测试断言首行非 P3-7;P3-6/P3-7 配置不设此项、保持 sample-aware 默认,回归锁定);文案/分区检查全过(无 stale P3-6 措辞、verdict 节只含独立 cells);secret scan 报告+日志 0 处;demo 0.96/0.84 不变。
- ✅ 质量门:`pytest` **383 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22 / P3-6=27 / P3-7=25+1 throttle / P3-8=8);`ruff` clean;`validate-config`(全部 11 配置)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、旧三因子无信号(P3-3/P3-4 实证;但其组合在 2024-2026 holdout 上 SSE50/CSI300/CSI500 全正、CSI500 高达 +17.8%——小样本/regime 翻转的持续例证);value/低波信号获得**独立样本符号级确认**(P3-7 SSE50/CSI300 量级衰减;P3-8 CSI500 泛化成立且更强),组合级盈利能力仍未确立(排名跨 cell 翻转);subset 报告文件名已可配置(P3-8 起不再互覆盖)。
- 路线图下一步:更长 holdout 积累(2026-06 之后滚动复检)/ 中证1000 或全市场扩展 / 成本模型细化(印花税卖侧不对称、冲击成本),或分钟级(architecture.html §11)。
26 changes: 26 additions & 0 deletions RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -471,6 +471,32 @@ Semantics worth knowing before reading the report:
`artifacts/reports/phase3_subset_validation.md` — a P3-7 run overwrites a
P3-6 report (regenerable; numbers live in CLAUDE.md/TEST_REPORT).

## Phase 3-8 — CSI500 independent generalization check (EXPLORATORY)

Asks whether the P3-7 sign-level conclusion generalizes OUTSIDE the screened
universes: the same `run-phase3-subset` machinery (unchanged) with a NEW
independent cell `000905.SH|2024-2026` (CSI500 — independent in BOTH universe
and time), plus the two SSE50 anchors (screened 2022-2024 must reproduce
P3-6/P3-7; independent 2024-2026 must reproduce the P3-7 verdict numbers).
Documented by `config/phase3_real_csi500_generalization.yaml`.

```bash
# validate (no network)
... -m qt.cli validate-config --config config/phase3_real_csi500_generalization.yaml
# run (network + token; HEAVY ~3.5h — the 735-name CSI500 cell dominates)
... -m qt.cli run-phase3-subset --config config/phase3_real_csi500_generalization.yaml
```

- `output.subset_report_name` (the `baseline_report_name` precedent) gives this
study its own report file (`phase3_csi500_generalization.md`), so the run no
longer clobbers the accepted P3-7 artifact; configs without the key keep the
historical filename bitwise (locked by tests).
- `output.subset_report_title` sets the report's H1 so it names THIS study
("Phase 3-8 — CSI500 Independent Generalization Check") instead of the
machinery's default phase label; configs without it keep the renderer's
sample-aware default (P3-7 independent / P3-6 post-hoc), locked by tests.
CSI500|2022-2024 is skip_cells-listed (runtime budget) and disclosed.

## Quality gate

```bash
Expand Down
18 changes: 13 additions & 5 deletions TEST_REPORT.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix → factor candidates → subset + cost sensitivity → independent validation)
# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix → factor candidates → subset + cost sensitivity → independent validation → CSI500 generalization)

## Commands

Expand All @@ -17,19 +17,20 @@ Run from the repo root with the project python (env `quant_mf`):
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_factor_candidates.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_subset_costs.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_independent_validation.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_csi500_generalization.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli run-phase0 --config config/example.yaml
```

## Results

| Gate | Command | Result |
|---|---|---|
| Unit + integration | `pytest -q` | **375 passed, 0 failed** |
| Unit + integration | `pytest -q` | **383 passed, 0 failed** |
| Lint | `ruff check .` | **All checks passed** |
| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml` + `phase3_real_factor_candidates.yaml` + `phase3_real_subset_costs.yaml` + `phase3_real_independent_validation.yaml`) | exit `0`, prints `OK` |
| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml` + `phase3_real_factor_candidates.yaml` + `phase3_real_subset_costs.yaml` + `phase3_real_independent_validation.yaml` + `phase3_real_csi500_generalization.yaml`) | exit `0`, prints `OK` |
| End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` |

Counts below are the actual per-file `pytest` numbers (sum = 375).
Counts below are the actual per-file `pytest` numbers (sum = 383).

## Per-file breakdown — Phase 0 core (97)

Expand Down Expand Up @@ -147,7 +148,14 @@ Counts below are the actual per-file `pytest` numbers (sum = 375).
|---|---|---|
| `test_independent_validation.py` | 25 | independent-cells config validation (must reference declared robustness universes/windows; a skip-listed cell cannot be declared independent — a holdout that never runs is a contradiction; hypotheses must reference ENABLED factors with a positive/negative literal; min_rebalances > 0; **the P3-6 config still validates with inert defaults**); explicit sample-class labeling (undeclared → screened, the conservative default); **verdict logic** (HOLDS iff expected IC sign in BOTH subperiods; SUPPORTED / PARTIAL / NOT SUPPORTED; **INSUFFICIENT-DATA overrides the sign check** with n_settled vs threshold disclosed; NaN or missing IC never holds); **per-class summaries never mix** (screened cell attributions never appear under independent and vice versa); report renders sample column + per-class cross-cell sections + a verdict section containing ONLY independent cells + INSUFFICIENT-DATA disclosure; a P3-6-era result (defaults) renders the old report shape; no secret; **sample-aware title/framing/caveats** (a run with independent cells must not carry the P3-6 'same windows / not independent confirmation' framing — review HIGH x2; P3-6-era rendering and downgrades text unchanged, locked by regression tests) |
| `test_tushare_throttle.py` (+1) | 1 | the DEFAULT retry budget survives a multi-failure transient outage (6 attempts ≈ 23s of capped exponential backoff; two real ~2h runs died on ConnectionError under the old 3-attempt ≈ 3s budget) |
| **Total (P0 + P1 + P2-1..P2-4 + P3-1..P3-7)** | **375** | |
| **Total through P3-7** | **375** | |

## Per-file breakdown — Phase 3-8 CSI500 generalization (8)

| Test file | Tests | Red-line / feature |
|---|---|---|
| `test_csi500_generalization.py` | 8 | CSI500 config validates with the expected cell roles (screened anchor SSE50|2022-2024; independent SSE50|2024-2026 + 000905.SH|2024-2026; CSI500|2022-2024 skipped+disclosed; same groups/scenarios/hypotheses as P3-7 — no tuning); sample classes labeled correctly; **`output.subset_report_name`** lets each subset-validation study own its report file (default None keeps the historical `phase3_subset_validation.md` bitwise — the P3-6/P3-7 configs are locked unchanged), so a P3-8 run never clobbers the accepted P3-7 artifact; **`output.subset_report_title`** config-drives the report H1 so the CSI500 study names itself ("Phase 3-8 — CSI500 Independent Generalization Check", asserted NOT to start with Phase 3-7) while the P3-6/P3-7 configs leave it unset and keep the renderer's sample-aware default title (regression-locked) |
| **Total (P0 + P1 + P2-1..P2-4 + P3-1..P3-8)** | **383** | |

## Real-data validation (manual, not in CI — TEST-002 keeps the suite network-free)

Expand Down
Loading