Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,15 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- `combo_ic_weighted` 是唯一跨 cell 稳定序列(test IC 3/3 正 + 3/3 sign consistent)但量级 ≈0.004~0.008 极小。
- **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**(sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重)。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。**
- secret scan 报告 0 处;demo 0.96/0.84 不变。
- ✅ 质量门:`pytest` **299 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=14);`ruff` clean;`validate-config`(旧 6 配置 + `phase3_real_robustness_matrix.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、三因子组 IC 跨期翻号且 ic_weighted 宽 universe 样本外崩(P3-3/P3-4 实证——当前因子组不具备可交易信号强度)。
- 路线图下一步:换/扩因子集(在既有验证框架下迭代),或分钟级(architecture.html §11)。
- 🔧 **Phase 3-5 factor candidate pack**(`p3-factor-candidate-pack` 分支,代劳待验收,**EXPLORATORY**):保守日频 PIT-safe 候选因子组,经 P3-4 矩阵原样复检"弱信号是否只是因子集太窄"。不动 alpha/portfolio/execution/OOS/robustness、不调参。
- 新因子(`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定):`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`overnight_mom_20`(Σlog(open_t/close_{t-1}) 20 日,open 当日开盘已知,prev close 不跨 symbol)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。
- dispatch(registry)扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。
- `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。
- **真实矩阵(11 因子版 3 cells / 2h;旧三因子+前轮候选 per-cell IC 逐数不变=不漂移 smoke ✓;secret scan 0;一次 CSI300 网络故障后整 run 重跑成功)**:
- **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;`overnight_mom_20` test IC 2/3 正(+0.008/+0.016,2020-22 为负)中等偏弱;reversal/grossprofit_margin 翻号无信号。
- `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0253/0.0012/0.0395)——walk-forward IC 加权吃到了新信号;11 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。
- 组合绩效:ic test −2.21%/−5.02%/−2.76%(**3/3 cells 跑赢等权**;CSI300 train +23.11%→test −2.76%,train→test 衰减仍显著)。
- ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。
- ✅ 质量门:`pytest` **322 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22);`ruff` clean;`validate-config`(旧 7 配置 + `phase3_real_factor_candidates.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、旧三因子无信号(P3-3/P3-4 实证);候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检(P3-5)。
- 路线图下一步:value+lowvol 子集独立复检 / 成本敏感性,或分钟级(architecture.html §11)。
15 changes: 12 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -116,6 +116,15 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- `combo_ic_weighted` 是唯一跨 cell 稳定序列(test IC 3/3 正 + 3/3 sign consistent)但量级 ≈0.004~0.008 极小。
- **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**(sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重)。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。**
- secret scan 报告 0 处;demo 0.96/0.84 不变。
- ✅ 质量门:`pytest` **299 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=14);`ruff` clean;`validate-config`(旧 6 配置 + `phase3_real_robustness_matrix.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、三因子组 IC 跨期翻号且 ic_weighted 宽 universe 样本外崩(P3-3/P3-4 实证——当前因子组不具备可交易信号强度)。
- 路线图下一步:换/扩因子集(在既有验证框架下迭代),或分钟级(architecture.html §11)。
- 🔧 **Phase 3-5 factor candidate pack**(`p3-factor-candidate-pack` 分支,代劳待验收,**EXPLORATORY**):保守日频 PIT-safe 候选因子组,经 P3-4 矩阵原样复检"弱信号是否只是因子集太窄"。不动 alpha/portfolio/execution/OOS/robustness、不调参。
- 新因子(`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定):`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`overnight_mom_20`(Σlog(open_t/close_{t-1}) 20 日,open 当日开盘已知,prev close 不跨 symbol)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。
- dispatch(registry)扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。
- `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。
- **真实矩阵(11 因子版 3 cells / 2h;旧三因子+前轮候选 per-cell IC 逐数不变=不漂移 smoke ✓;secret scan 0;一次 CSI300 网络故障后整 run 重跑成功)**:
- **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;`overnight_mom_20` test IC 2/3 正(+0.008/+0.016,2020-22 为负)中等偏弱;reversal/grossprofit_margin 翻号无信号。
- `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0253/0.0012/0.0395)——walk-forward IC 加权吃到了新信号;11 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。
- 组合绩效:ic test −2.21%/−5.02%/−2.76%(**3/3 cells 跑赢等权**;CSI300 train +23.11%→test −2.76%,train→test 衰减仍显著)。
- ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。
- ✅ 质量门:`pytest` **322 passed**(P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22);`ruff` clean;`validate-config`(旧 7 配置 + `phase3_real_factor_candidates.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余(已显式披露):日线 only、demo 路径非真数据、旧三因子无信号(P3-3/P3-4 实证);候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检(P3-5)。
- 路线图下一步:value+lowvol 子集独立复检 / 成本敏感性,或分钟级(architecture.html §11)。
41 changes: 41 additions & 0 deletions RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -348,6 +348,47 @@ stability, weight sign flips, fallback counts, boundary rebalances); and the
explicit caveat that this is a stability check — NOT a return claim, not a
tuned result.

## Phase 3-5 — factor candidate pack (EXPLORATORY)

Adds a conservative, daily, PIT-safe candidate factor pack and re-runs the
P3-4 robustness matrix to test whether the legacy trio's weak signal was just
a too-narrow factor set. **Exploratory — not tuned, not a return claim.** No
alpha / portfolio / execution / OOS / robustness model change. Documented by
`config/phase3_real_factor_candidates.yaml` (legacy trio + 8 candidates,
same matrix shape as P3-4 incl. the disclosed CSI300×2020-2022 skip).

```bash
# validate (no network)
... -m qt.cli validate-config --config config/phase3_real_factor_candidates.yaml
# run (network + token; HEAVY ~2-2.5h — CSI300 adds ~400 names × daily_basic pe/pb)
... -m qt.cli run-phase3-robustness --config config/phase3_real_factor_candidates.yaml
```

Candidates (all locked by unit tests for math, leading-window NaN,
no-future-bars, per-symbol isolation, immutability):

| Factor | Definition | PIT argument |
|---|---|---|
| `reversal_5/20` | −(close[t]/close[t−w] − 1) | exact negative of momentum (≤t bars) |
| `volatility_20` | std of trailing 20 daily returns (ddof=1, full window) | ≤t bars |
| `liquidity_20` | log(mean(amount, 20)) (non-positive → NaN) | same-day bar amount |
| `overnight_mom_20` | Σ log(open[t]/close[t−1]) over 20d (non-positive → NaN) | open known at the t open; computed at the t close |
| `value_ep` / `value_bp` | 1/pe, 1/pb from `daily_basic` (one fetch; ≤0 → NaN) | ratios published same-day |
| `grossprofit_margin` | financial quality field | ann_date as-of (existing machinery) |

Registry / dispatch: `_build_factors` resolves `reversal*` / `volatility*` /
`liquidity*` / `overnight_mom*` / value fields / financial fields; window-named factors must agree
with `params.window` (a mismatch is a readable config error, never a silent
mislabel). Legacy configs are untouched and reproduce their numbers.

Old-vs-new comparison reads off ONE run: raw-factor ICs are per-column (each
factor's IC depends only on its own column + forward returns), so the legacy
trio's per-cell ICs in the candidates run double as a no-drift cross-check
against the P3-4 report, and the candidate ICs are directly comparable in the
same report. Note `drop_missing` requires ALL enabled factors per name/date
(e.g. a loss-maker's NaN `value_ep` drops it that day) — the combo legs are
therefore NOT the P3-4 combos; disclosed.

## Quality gate

```bash
Expand Down
25 changes: 20 additions & 5 deletions TEST_REPORT.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix)
# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix → factor candidates)

## Commands

Expand All @@ -14,19 +14,20 @@ Run from the repo root with the project python (env `quant_mf`):
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_ic_weighted.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_oos_stability.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_robustness_matrix.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_factor_candidates.yaml
/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli run-phase0 --config config/example.yaml
```

## Results

| Gate | Command | Result |
|---|---|---|
| Unit + integration | `pytest -q` | **299 passed, 0 failed** |
| Unit + integration | `pytest -q` | **322 passed, 0 failed** |
| Lint | `ruff check .` | **All checks passed** |
| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml`) | exit `0`, prints `OK` |
| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml` + `phase3_real_factor_candidates.yaml`) | exit `0`, prints `OK` |
| End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` |

Counts below are the actual per-file `pytest` numbers (sum = 299).
Counts below are the actual per-file `pytest` numbers (sum = 322).

## Per-file breakdown — Phase 0 core (97)

Expand Down Expand Up @@ -123,7 +124,14 @@ Counts below are the actual per-file `pytest` numbers (sum = 299).
| Test file | Tests | Red-line / feature |
|---|---|---|
| `test_robustness_matrix.py` | 14 | matrix config validation (universes/windows/unique labels, split-inside-window, unknown-skip + all-skipped ConfigErrors); cell enumeration excludes skips in config order; per-cell config derivation swaps ONLY the cell identity (incl. unique per-cell output_name; derived cell passes the shared OOS preconditions); cross-cell aggregation attributes strictly per cell (no mixing, no dilution); runner guards (demo source / missing robustness section / non-ic_weighted alpha); report renders cell labels + skipped-cell disclosure + boundary/fallback/sign-consistency + cross-cell summary + caveat + no secret |
| **Total (P0 + P1 + P2-1..P2-4 + P3-1 + P3-2 + P3-3 + P3-4)** | **299** | |

## Per-file breakdown — Phase 3-5 factor candidates (22 + 1 reworked)

| Test file | Tests | Red-line / feature |
|---|---|---|
| `test_candidate_factors.py` | 22 | per-factor math vs manual references (reversal == −momentum, volatility rolling std, liquidity log-mean-amount, overnight Σlog(open/prev-close) incl. prev-close-never-crosses-symbols); leading-window NaN; future-bar perturbation invariance; per-symbol isolation; non-positive amount → NaN never −inf; missing-column readable errors; ValueFactor surfaces enriched column + rejects unknown fields; grossprofit_margin in SUPPORTED_FIELDS; dispatch builds every candidate + rejects name/params mismatch; value enrichment ONE fetch for both fields + 1/pe 1/pb math + non-positive guards + demo readable error + input immutability; demo e2e with price candidates (per-factor analytics, report, no secret); candidates config validates |
| `test_multifactor_pipeline.py` (reworked 1) | — | duplicate-name test now uses a true duplicate spec (the new name/params-mismatch guard catches the old mislabel construction earlier) |
| **Total (P0 + P1 + P2-1..P2-4 + P3-1..P3-5)** | **322** | |

## Real-data validation (manual, not in CI — TEST-002 keeps the suite network-free)

Expand Down Expand Up @@ -212,6 +220,13 @@ Counts below are the actual per-file `pytest` numbers (sum = 299).
derive from the base with only the cell identity swapped; skipped cells are
config-validated and disclosed (never silent); cross-cell aggregation is
strictly per cell. Single-run `run-phase3-oos` behaviour is unchanged.
- **P3-5 factor candidates (locked by tests):** conservative daily PIT-safe
candidates (reversal_5/20, volatility_20, liquidity_20, overnight_mom_20, value_ep/bp via one
daily_basic fetch with non-positive guards, grossprofit_margin via the
existing ann_date machinery). The dispatch rejects a window-name/params
mismatch; legacy configs reproduce their numbers (demo ic 0.96 / annual
0.84 unchanged). EXPLORATORY — validated through the unchanged P3-4
robustness matrix; not tuned, not a return claim.
- A duplicate test-function name across two files was found and renamed during P2-2
(it had been silently shadowing one test in the full-suite run). A second, harmless
duplicate (`test_enrich_does_not_mutate_input` in two files) was verified NOT to drop
Expand Down
Loading