From b94317aff863a2a666a9379c5f39f3c4e8898046 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Thu, 11 Jun 2026 00:10:52 +0800 Subject: [PATCH 1/4] feat(factors): P3-5 candidate factor pack, validated through the P3-4 matrix EXPLORATORY: tests whether the legacy trio's weak signal (P3-3/P3-4) was just a too-narrow factor set. No alpha / portfolio / execution / OOS / robustness model change; nothing tuned; not a return claim. Candidates (factors/compute/candidates.py; math, leading-window NaN, future-bar invariance, per-symbol isolation and immutability all unit-locked): - reversal_5/20: exact negative of momentum (reuses the same machinery); - volatility_20: trailing daily-return std (ddof=1, full window only); - liquidity_20: log trailing mean turnover amount (non-positive -> NaN); - value_ep / value_bp: 1/pe, 1/pb from ONE daily_basic fetch (same-day published, PIT-safe; non-positive ratios -> NaN); demo -> readable error; - grossprofit_margin: joins SUPPORTED_FIELDS, rides the existing ann_date as-of machinery unchanged. Wiring: _build_factors dispatch extended (reversal*/volatility*/liquidity*/ value fields); a window-name/params mismatch is now a readable config error (never a silently mislabelled column); _maybe_enrich_value wired into phase0 / phase2 / OOS-cell paths. Legacy configs reproduce their numbers (demo ic 0.96 / annual 0.84 unchanged). config/phase3_real_factor_candidates.yaml: legacy trio + 7 candidates through the UNCHANGED P3-4 robustness matrix (same cells; CSI300x2020-2022 stays an explicit, disclosed skip). Old-vs-new reads off one run: raw-factor ICs are per-column, so the trio's per-cell ICs double as a no-drift cross-check. Real matrix (3 cells, 2.3h; trio ICs identical to the P3-4 report = no-drift smoke PASSED; secret scan clean): - The too-narrow hypothesis is SUPPORTED: value_ep/value_bp test IC 3/3 positive at 0.037-0.056 and volatility_20 3/3 negative at -0.044..-0.079 -- an order of magnitude above the trio (|IC| <= 0.015) with cross-cell-stable direction. liquidity weakly negative 3/3; reversal / grossprofit_margin sign-flip with no signal. - combo_ic_weighted test IC 3/3 positive + 3/3 sign-consistent (the walk-forward weights pick up the new signal); the 10-factor EQUAL-WEIGHT combo is diluted by the sign-flipping factors (0/3, CSI300 test -34.6%) -- equal weight is sensitive to factor-set quality. - Portfolio: ic test -3.62% / -4.88% / -2.64% (beats equal weight in 2/3 cells; CSI300 train +23.69% -> test -2.64%: train->test decay remains). - CAVEATS disclosed: value/low-vol were a known strong A-share regime in 2020-2024; the three cells overlap in time (not independent samples); no cost-sensitivity yet. Next: an independent re-check of a value+lowvol subset. Tests: 317 passed (was 300): +17 candidates; the P3-1 duplicate-name test now uses a true duplicate spec (the new mismatch guard catches the old construction earlier). --- AGENTS.md | 15 +- CLAUDE.md | 15 +- RUNBOOK.md | 40 +++ TEST_REPORT.md | 23 +- config/phase3_real_factor_candidates.yaml | 164 +++++++++++ data/feed/tushare_covariates.py | 36 +++ factors/compute/candidates.py | 150 ++++++++++ factors/compute/financial.py | 6 +- qt/oos_stability.py | 2 + qt/phase2_baseline.py | 2 + qt/pipeline.py | 92 ++++++- tests/test_candidate_factors.py | 317 ++++++++++++++++++++++ tests/test_multifactor_pipeline.py | 2 +- 13 files changed, 844 insertions(+), 20 deletions(-) create mode 100644 config/phase3_real_factor_candidates.yaml create mode 100644 factors/compute/candidates.py create mode 100644 tests/test_candidate_factors.py diff --git a/AGENTS.md b/AGENTS.md index 9bc81fe..96ca09d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -116,6 +116,15 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - `combo_ic_weighted` 是唯一跨 cell 稳定序列（test IC 3/3 正 + 3/3 sign consistent）但量级 ≈0.004~0.008 极小。 - **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**（sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重）。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。** - secret scan 报告 0 处;demo 0.96/0.84 不变。 -- ✅ 质量门：`pytest` **299 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=14）；`ruff` clean；`validate-config`（旧 6 配置 + `phase3_real_robustness_matrix.yaml`）+ `run-phase0`（demo）均 OK。 -- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、三因子组 IC 跨期翻号且 ic_weighted 宽 universe 样本外崩（P3-3/P3-4 实证——当前因子组不具备可交易信号强度）。 -- 路线图下一步：换/扩因子集（在既有验证框架下迭代）,或分钟级（architecture.html §11）。 +- 🔧 **Phase 3-5 factor candidate pack**（`p3-factor-candidate-pack` 分支,代劳待验收,**EXPLORATORY**）：保守日频 PIT-safe 候选因子组,经 P3-4 矩阵原样复检"弱信号是否只是因子集太窄"。不动 alpha/portfolio/execution/OOS/robustness、不调参。 + - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 + - dispatch（registry）扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。 + - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选七 = 10 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 + - **真实矩阵（3 cells / 2.3h;旧三因子 per-cell IC 与 P3-4 逐数一致=不漂移 smoke ✓;secret scan 0）**: + - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;reversal/grossprofit_margin 翻号无信号。 + - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−34.6%**)——等权对因子集质量敏感。 + - 组合绩效:ic test −3.62%/−4.88%/−2.64%(2/3 cells 跑赢等权;CSI300 train +23.69%→test −2.64%,train→test 衰减仍显著)。 + - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 +- ✅ 质量门：`pytest` **317 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=17）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 +- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、旧三因子无信号（P3-3/P3-4 实证）;候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检（P3-5）。 +- 路线图下一步：value+lowvol 子集独立复检 / 成本敏感性,或分钟级（architecture.html §11）。 diff --git a/CLAUDE.md b/CLAUDE.md index 9bc81fe..96ca09d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -116,6 +116,15 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - `combo_ic_weighted` 是唯一跨 cell 稳定序列（test IC 3/3 正 + 3/3 sign consistent）但量级 ≈0.004~0.008 极小。 - **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**（sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重）。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。** - secret scan 报告 0 处;demo 0.96/0.84 不变。 -- ✅ 质量门：`pytest` **299 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=14）；`ruff` clean；`validate-config`（旧 6 配置 + `phase3_real_robustness_matrix.yaml`）+ `run-phase0`（demo）均 OK。 -- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、三因子组 IC 跨期翻号且 ic_weighted 宽 universe 样本外崩（P3-3/P3-4 实证——当前因子组不具备可交易信号强度）。 -- 路线图下一步：换/扩因子集（在既有验证框架下迭代）,或分钟级（architecture.html §11）。 +- 🔧 **Phase 3-5 factor candidate pack**（`p3-factor-candidate-pack` 分支,代劳待验收,**EXPLORATORY**）：保守日频 PIT-safe 候选因子组,经 P3-4 矩阵原样复检"弱信号是否只是因子集太窄"。不动 alpha/portfolio/execution/OOS/robustness、不调参。 + - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 + - dispatch（registry）扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。 + - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选七 = 10 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 + - **真实矩阵（3 cells / 2.3h;旧三因子 per-cell IC 与 P3-4 逐数一致=不漂移 smoke ✓;secret scan 0）**: + - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;reversal/grossprofit_margin 翻号无信号。 + - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−34.6%**)——等权对因子集质量敏感。 + - 组合绩效:ic test −3.62%/−4.88%/−2.64%(2/3 cells 跑赢等权;CSI300 train +23.69%→test −2.64%,train→test 衰减仍显著)。 + - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 +- ✅ 质量门：`pytest` **317 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=17）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 +- ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、旧三因子无信号（P3-3/P3-4 实证）;候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检（P3-5）。 +- 路线图下一步：value+lowvol 子集独立复检 / 成本敏感性,或分钟级（architecture.html §11）。 diff --git a/RUNBOOK.md b/RUNBOOK.md index 383dd35..3509e41 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -348,6 +348,46 @@ stability, weight sign flips, fallback counts, boundary rebalances); and the explicit caveat that this is a stability check — NOT a return claim, not a tuned result. +## Phase 3-5 — factor candidate pack (EXPLORATORY) + +Adds a conservative, daily, PIT-safe candidate factor pack and re-runs the +P3-4 robustness matrix to test whether the legacy trio's weak signal was just +a too-narrow factor set. **Exploratory — not tuned, not a return claim.** No +alpha / portfolio / execution / OOS / robustness model change. Documented by +`config/phase3_real_factor_candidates.yaml` (legacy trio + 7 candidates, +same matrix shape as P3-4 incl. the disclosed CSI300×2020-2022 skip). + +```bash +# validate (no network) +... -m qt.cli validate-config --config config/phase3_real_factor_candidates.yaml +# run (network + token; HEAVY ~2-2.5h — CSI300 adds ~400 names × daily_basic pe/pb) +... -m qt.cli run-phase3-robustness --config config/phase3_real_factor_candidates.yaml +``` + +Candidates (all locked by unit tests for math, leading-window NaN, +no-future-bars, per-symbol isolation, immutability): + +| Factor | Definition | PIT argument | +|---|---|---| +| `reversal_5/20` | −(close[t]/close[t−w] − 1) | exact negative of momentum (≤t bars) | +| `volatility_20` | std of trailing 20 daily returns (ddof=1, full window) | ≤t bars | +| `liquidity_20` | log(mean(amount, 20)) (non-positive → NaN) | same-day bar amount | +| `value_ep` / `value_bp` | 1/pe, 1/pb from `daily_basic` (one fetch; ≤0 → NaN) | ratios published same-day | +| `grossprofit_margin` | financial quality field | ann_date as-of (existing machinery) | + +Registry / dispatch: `_build_factors` resolves `reversal*` / `volatility*` / +`liquidity*` / value fields / financial fields; window-named factors must agree +with `params.window` (a mismatch is a readable config error, never a silent +mislabel). Legacy configs are untouched and reproduce their numbers. + +Old-vs-new comparison reads off ONE run: raw-factor ICs are per-column (each +factor's IC depends only on its own column + forward returns), so the legacy +trio's per-cell ICs in the candidates run double as a no-drift cross-check +against the P3-4 report, and the candidate ICs are directly comparable in the +same report. Note `drop_missing` requires ALL enabled factors per name/date +(e.g. a loss-maker's NaN `value_ep` drops it that day) — the combo legs are +therefore NOT the P3-4 combos; disclosed. + ## Quality gate ```bash diff --git a/TEST_REPORT.md b/TEST_REPORT.md index ec8e468..8641ae9 100644 --- a/TEST_REPORT.md +++ b/TEST_REPORT.md @@ -1,4 +1,4 @@ -# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix) +# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 + Phase 3 (bias-boundary → execution realism → PIT industry → standard analytics → multi-factor → walk-forward IC alpha → OOS stability → robustness matrix → factor candidates) ## Commands @@ -14,6 +14,7 @@ Run from the repo root with the project python (env `quant_mf`): /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_ic_weighted.yaml /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_oos_stability.yaml /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_robustness_matrix.yaml +/home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli validate-config --config config/phase3_real_factor_candidates.yaml /home/shaofl/Development/env_tools/envs/quant_mf/bin/python -m qt.cli run-phase0 --config config/example.yaml ``` @@ -23,10 +24,10 @@ Run from the repo root with the project python (env `quant_mf`): |---|---|---| | Unit + integration | `pytest -q` | **299 passed, 0 failed** | | Lint | `ruff check .` | **All checks passed** | -| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml`) | exit `0`, prints `OK` | +| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml` + `phase3_real_factor_candidates.yaml`) | exit `0`, prints `OK` | | End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` | -Counts below are the actual per-file `pytest` numbers (sum = 299). +Counts below are the actual per-file `pytest` numbers (sum = 317). ## Per-file breakdown — Phase 0 core (97) @@ -123,7 +124,14 @@ Counts below are the actual per-file `pytest` numbers (sum = 299). | Test file | Tests | Red-line / feature | |---|---|---| | `test_robustness_matrix.py` | 14 | matrix config validation (universes/windows/unique labels, split-inside-window, unknown-skip + all-skipped ConfigErrors); cell enumeration excludes skips in config order; per-cell config derivation swaps ONLY the cell identity (incl. unique per-cell output_name; derived cell passes the shared OOS preconditions); cross-cell aggregation attributes strictly per cell (no mixing, no dilution); runner guards (demo source / missing robustness section / non-ic_weighted alpha); report renders cell labels + skipped-cell disclosure + boundary/fallback/sign-consistency + cross-cell summary + caveat + no secret | -| **Total (P0 + P1 + P2-1..P2-4 + P3-1 + P3-2 + P3-3 + P3-4)** | **299** | | + +## Per-file breakdown — Phase 3-5 factor candidates (17 + 1 reworked) + +| Test file | Tests | Red-line / feature | +|---|---|---| +| `test_candidate_factors.py` | 17 | per-factor math vs manual references (reversal == −momentum, volatility rolling std, liquidity log-mean-amount); leading-window NaN; future-bar perturbation invariance; per-symbol isolation; non-positive amount → NaN never −inf; missing-column readable errors; ValueFactor surfaces enriched column + rejects unknown fields; grossprofit_margin in SUPPORTED_FIELDS; dispatch builds every candidate + rejects name/params mismatch; value enrichment ONE fetch for both fields + 1/pe 1/pb math + non-positive guards + demo readable error + input immutability; demo e2e with price candidates (per-factor analytics, report, no secret); candidates config validates | +| `test_multifactor_pipeline.py` (reworked 1) | — | duplicate-name test now uses a true duplicate spec (the new name/params-mismatch guard catches the old mislabel construction earlier) | +| **Total (P0 + P1 + P2-1..P2-4 + P3-1..P3-5)** | **317** | | ## Real-data validation (manual, not in CI — TEST-002 keeps the suite network-free) @@ -212,6 +220,13 @@ Counts below are the actual per-file `pytest` numbers (sum = 299). derive from the base with only the cell identity swapped; skipped cells are config-validated and disclosed (never silent); cross-cell aggregation is strictly per cell. Single-run `run-phase3-oos` behaviour is unchanged. +- **P3-5 factor candidates (locked by tests):** conservative daily PIT-safe + candidates (reversal_5/20, volatility_20, liquidity_20, value_ep/bp via one + daily_basic fetch with non-positive guards, grossprofit_margin via the + existing ann_date machinery). The dispatch rejects a window-name/params + mismatch; legacy configs reproduce their numbers (demo ic 0.96 / annual + 0.84 unchanged). EXPLORATORY — validated through the unchanged P3-4 + robustness matrix; not tuned, not a return claim. - A duplicate test-function name across two files was found and renamed during P2-2 (it had been silently shadowing one test in the full-suite run). A second, harmless duplicate (`test_enrich_does_not_mutate_input` in two files) was verified NOT to drop diff --git a/config/phase3_real_factor_candidates.yaml b/config/phase3_real_factor_candidates.yaml new file mode 100644 index 0000000..743c5f2 --- /dev/null +++ b/config/phase3_real_factor_candidates.yaml @@ -0,0 +1,164 @@ +# Phase 3-5 — REAL factor-candidate robustness matrix (EXPLORATORY). +# +# Tests whether the legacy trio's weak signal (P3-3/P3-4 finding) was just a +# too-narrow factor set: the LEGACY trio (momentum_20 + roe + netprofit_yoy) +# plus a conservative candidate pack (reversal_5/20, volatility_20, +# liquidity_20, value_ep/bp, grossprofit_margin) run through the SAME P3-4 +# robustness matrix (same universes x windows, same walk-forward ic_weighted +# vs equal_weight comparison, same holding-window slicing). +# +# EXPLORATORY: factors are conservative textbook definitions, NOT tuned; the +# run is NOT a return claim. Raw-factor ICs are per-column, so the legacy +# trio's per-cell ICs double as a no-drift cross-check against the P3-4 report. +# PIT safety: price/volume factors use only <=t bars; value_ep/bp invert +# same-day daily_basic pe/pb (non-positive -> NaN); grossprofit_margin rides +# the ann_date as-of machinery. drop_missing requires ALL enabled factors per +# name/date (e.g. a loss-maker's NaN value_ep drops it that day) -- disclosed. +# +# Run: python -m qt.cli run-phase3-robustness --config config/phase3_real_factor_candidates.yaml +# (report: artifacts/reports/phase3_robustness_matrix.md under this config's +# output dirs; needs the tushare token; HEAVY ~2-2.5h: the CSI300 cell pulls +# ~400 names x daily_basic pe/pb on top of the P3-4 data. CSI300x2020-2022 +# stays skip_cells-listed -- disclosed in the report.) + +project: + name: quantitative_trading_phase3_factor_candidates + timezone: Asia/Shanghai + +data: + source: tushare + freq: D + start: "2022-07-01" + end: "2024-06-30" + external_secret_file: "/home/shaofl/Projects/financial_projects/.config.json" + tushare_token_key: "tushare.token" + output_name: phase3fc_daily + +universe: + type: index + index_code: "000016.SH" + symbols: [] + min_listing_days: 60 + filters: + missing_close: true + suspended: true + st: true + limit_up_down: true + +factors: + # legacy trio (kept verbatim so the old-vs-new comparison reads off one run: + # raw-factor ICs are per-column and independent of the other columns) + - name: momentum_20 + enabled: true + params: + window: 20 + price_col: close + - name: roe + enabled: true + params: {} + - name: netprofit_yoy + enabled: true + params: {} + # P3-5 candidate pack (conservative, daily, PIT-safe; EXPLORATORY — not tuned) + - name: reversal_5 + enabled: true + params: + window: 5 + - name: reversal_20 + enabled: true + params: + window: 20 + - name: volatility_20 + enabled: true + params: + window: 20 + - name: liquidity_20 + enabled: true + params: + window: 20 + - name: value_ep # 1/pe from daily_basic (same-day, PIT-safe; pe<=0 -> NaN) + enabled: true + params: {} + - name: value_bp # 1/pb from daily_basic + enabled: true + params: {} + - name: grossprofit_margin # quality; ann_date as-of like roe/netprofit_yoy + enabled: true + params: {} + +processing: + drop_missing: true + standardize: + enabled: true + method: zscore + winsorize: + enabled: false + method: mad + n: 3.0 + neutralize: + enabled: true + industry_col: industry + size_col: market_cap + industry_level: L1 # PIT SW level (L1/L2/L3); L1 = 31 broad sectors (standard, DOF-safe) + +alpha: + model: ic_weighted # P3-2 walk-forward rolling-IC weights + params: + window: 60 # trailing trading days of REALIZED ICs (conservative rolling) + min_periods: 20 # < this many valid realized ICs -> equal-weight fallback (disclosed) + mode: rolling # rolling (default) | expanding + +portfolio: + constructor: topn_equal_weight + top_n: 20 + long_only: true + max_weight: null + turnover_cap: null + +backtest: + initial_nav: 1.0 + rebalance: monthly + event_order: close_to_next_period + cash_return: 0.0 + +cost: + fee_rate: 0.001 + slippage_rate: 0.0 + turnover_formula: l1 + +analytics: + forward_return_periods: + - 1 + - 5 + - 20 + quantiles: 5 + benchmark: null + +output: + root_dir: artifacts + data_dir: artifacts/data + factor_dir: artifacts/factors + report_dir: artifacts/reports + log_dir: artifacts/logs + overwrite: true + +oos: + split_date: "2023-07-01" # base-config validation only; per-cell splits below + +robustness: + universes: + - "000016.SH" # SSE50 (the P3-3 sample) + - "000300.SH" # CSI300 (wider universe) + windows: + - label: "2020-2022" + start: "2020-07-01" + end: "2022-06-30" + split: "2021-07-01" + - label: "2022-2024" + start: "2022-07-01" + end: "2024-06-30" + split: "2023-07-01" + # cells = universes x windows MINUS the skips below (runtime budget; disclosed). + skip_cells: + - universe: "000300.SH" + window: "2020-2022" diff --git a/data/feed/tushare_covariates.py b/data/feed/tushare_covariates.py index ae95dd0..c1ed7f5 100644 --- a/data/feed/tushare_covariates.py +++ b/data/feed/tushare_covariates.py @@ -11,6 +11,9 @@ * ``market_cap(symbols, s, e)`` -> DataFrame[date, symbol, market_cap] (tushare ``daily_basic.total_mv``, in 10k CNY; only the log is used, so units do not matter). Genuinely per-date. + * ``value_ratios(symbols, s, e)`` -> DataFrame[date, symbol, pe, pb] + (tushare ``daily_basic``; published same-day, PIT-safe by construction; + the value-factor inversion happens in the pipeline, P3-5). * ``listing_dates(symbols)`` -> {symbol: list_date} (``stock_basic.list_date``, for the ``min_listing_days`` buy filter, UNI-008). * ``industry(symbols)`` -> {symbol: industry} (``stock_basic.industry``, @@ -128,6 +131,39 @@ def pit_sw_intervals( out[str(sym)] = rows return out + def value_ratios(self, symbols: list[str], start: str, end: str) -> pd.DataFrame: + """Return DataFrame[date, symbol, pe, pb] from daily_basic (P3-5). + + The ratios are published same-day (PIT-safe by construction); the + inversion to value_ep / value_bp (with non-positive guards) happens in + the pipeline's value enrichment, not here. + """ + pro = self._client() + s = pd.Timestamp(start).strftime("%Y%m%d") + e = pd.Timestamp(end).strftime("%Y%m%d") + frames: list[pd.DataFrame] = [] + for sym in symbols: + df = self._call( + pro.daily_basic, + ts_code=sym, + start_date=s, + end_date=e, + fields="ts_code,trade_date,pe,pb", + ) + if df is not None and len(df) > 0: + frames.append(df) + if not frames: + return pd.DataFrame( + {"date": pd.Series([], dtype="datetime64[ns]"), + "symbol": pd.Series([], dtype=object), + "pe": pd.Series([], dtype=float), + "pb": pd.Series([], dtype=float)} + ) + out = pd.concat(frames, ignore_index=True).rename(columns={"ts_code": "symbol"}) + out["date"] = pd.to_datetime(out["trade_date"].astype(str), format="%Y%m%d") + out["symbol"] = out["symbol"].astype(str) + return out[["date", "symbol", "pe", "pb"]] + def market_cap(self, symbols: list[str], start: str, end: str) -> pd.DataFrame: """Return DataFrame[date, symbol, market_cap] from daily_basic.total_mv.""" pro = self._client() diff --git a/factors/compute/candidates.py b/factors/compute/candidates.py new file mode 100644 index 0000000..7216038 --- /dev/null +++ b/factors/compute/candidates.py @@ -0,0 +1,150 @@ +"""P3-5 candidate factor pack: conservative, daily, PIT-safe additions. + +EXPLORATORY factors used to test whether the legacy trio's weak signal was just +a too-narrow factor set (validated through the P3-4 robustness matrix; not +tuned, not a return claim). Every factor here uses ONLY data known at the trade +date and computes strictly per symbol (grouped on the ``symbol`` index level): + + * ``reversal_w`` = -(close[t] / close[t-w] - 1) — short-horizon reversal, + the exact negative of the momentum definition (same no-lookahead argument). + * ``volatility_w`` = std of the trailing ``w`` daily returns (ddof=1, + min_periods=w → the leading window is NaN, never a partial estimate). + * ``liquidity_w`` = log of the trailing ``w``-day mean turnover ``amount`` + (non-positive means → NaN, never -inf; the panel's ``amount`` column comes + from the bar feed, known same-day). + * ``value_ep`` / ``value_bp`` surface a daily_basic-enriched column + (1/pe, 1/pb placed on the panel by the pipeline's value enrichment; the + ratios are published same-day, PIT-safe by construction). They require the + tushare path — a demo run has no pe/pb and fails readably upstream. + +The quality field ``grossprofit_margin`` is NOT here: it joins +``factors.compute.financial.SUPPORTED_FIELDS`` and rides the existing ann_date +as-of machinery unchanged. +""" + +from __future__ import annotations + +import numpy as np +import pandas as pd + +from factors.base import Factor +from factors.compute.momentum import MomentumFactor + +# daily_basic-derived value fields the pipeline can enrich + surface (P3-5). +VALUE_FIELDS: tuple[str, ...] = ("value_ep", "value_bp") + + +class ReversalFactor(Factor): + """Short-horizon reversal: the exact negative of ``momentum_w``. + + Reuses :class:`MomentumFactor`'s computation (per-symbol, strictly lagged, + no lookahead) and flips the sign, so the two definitions can never drift + apart. + """ + + name: str = "reversal_20" + + def __init__(self, window: int = 20, price_col: str = "close") -> None: + self._momentum = MomentumFactor(window=window, price_col=price_col) + self.name = f"reversal_{window}" + + def compute(self, panel: pd.DataFrame) -> pd.Series: + return (-self._momentum.compute(panel)).rename(self.name) + + +class VolatilityFactor(Factor): + """Trailing daily-return volatility over a fixed window (per symbol). + + ``min_periods=window`` keeps the leading window NaN (a partial-window std + would silently change meaning across the panel head). + """ + + name: str = "volatility_20" + + def __init__(self, window: int = 20, price_col: str = "close") -> None: + if not isinstance(window, int) or window < 2: + raise ValueError( + f"volatility window must be an integer >= 2, got {window!r}." + ) + self._window = window + self._price_col = price_col + self.name = f"volatility_{window}" + + def compute(self, panel: pd.DataFrame) -> pd.Series: + if self._price_col not in panel.columns: + raise ValueError( + f"volatility factor needs a '{self._price_col}' column; panel has " + f"{list(panel.columns)}." + ) + price = panel[self._price_col] + grouped = price.groupby(level="symbol", group_keys=False) + # pct_change within each symbol (never across); rolling std needs a FULL + # window of returns -> leading rows are NaN, all inputs are <= t. + vol = grouped.apply( + lambda s: s.pct_change().rolling( + self._window, min_periods=self._window + ).std(ddof=1) + ) + return vol.reindex(panel.index).rename(self.name) + + +class LiquidityFactor(Factor): + """Log of the trailing mean turnover ``amount`` over a fixed window. + + A simple size-of-trading liquidity proxy (its correlation with market cap + is handled by the existing size neutralization). Non-positive rolling means + map to NaN — never a silent ``-inf``. + """ + + name: str = "liquidity_20" + + def __init__(self, window: int = 20, amount_col: str = "amount") -> None: + if not isinstance(window, int) or window < 1: + raise ValueError( + f"liquidity window must be a positive integer, got {window!r}." + ) + self._window = window + self._amount_col = amount_col + self.name = f"liquidity_{window}" + + def compute(self, panel: pd.DataFrame) -> pd.Series: + if self._amount_col not in panel.columns: + raise ValueError( + f"liquidity factor needs an '{self._amount_col}' column (turnover " + f"amount from the bar feed); panel has {list(panel.columns)}." + ) + amount = panel[self._amount_col] + grouped = amount.groupby(level="symbol", group_keys=False) + mean_amt = grouped.apply( + lambda s: s.rolling(self._window, min_periods=self._window).mean() + ) + # log of a non-positive mean is undefined -> NaN (degenerate liquidity). + safe = mean_amt.where(mean_amt > 0) + return np.log(safe).reindex(panel.index).rename(self.name) + + +class ValueFactor(Factor): + """Surface a daily_basic-enriched value column (``value_ep`` / ``value_bp``). + + The column is placed on the panel by the pipeline's value enrichment + (1/pe, 1/pb; same-day-published ratios, PIT-safe by construction). Like the + financial factors, this does no temporal logic of its own. + """ + + def __init__(self, field: str) -> None: + if field not in VALUE_FIELDS: + raise ValueError( + f"ValueFactor field {field!r} not supported; choose one of " + f"{VALUE_FIELDS}." + ) + self.name = field + self._field = field + + def compute(self, panel: pd.DataFrame) -> pd.Series: + if self._field not in panel.columns: + raise ValueError( + f"ValueFactor('{self._field}') needs an enriched '{self._field}' " + f"column on the panel (daily_basic pe/pb; tushare path only — " + f"demo data has neither)." + ) + return panel[self._field].rename(self.name) diff --git a/factors/compute/financial.py b/factors/compute/financial.py index f24f846..93b2577 100644 --- a/factors/compute/financial.py +++ b/factors/compute/financial.py @@ -13,8 +13,10 @@ from factors.base import Factor -# financial fields that may be requested as a factor (P1). -SUPPORTED_FIELDS: tuple[str, ...] = ("roe", "netprofit_yoy") +# financial fields that may be requested as a factor (P1; grossprofit_margin +# joined in P3-5 as the conservative quality candidate — same ann_date as-of +# machinery, no new temporal logic). +SUPPORTED_FIELDS: tuple[str, ...] = ("roe", "netprofit_yoy", "grossprofit_margin") class FinancialFactor(Factor): diff --git a/qt/oos_stability.py b/qt/oos_stability.py index c30a294..a17f487 100644 --- a/qt/oos_stability.py +++ b/qt/oos_stability.py @@ -51,6 +51,7 @@ _maybe_enrich_covariates, _maybe_enrich_financials, _maybe_enrich_listing, + _maybe_enrich_value, _periods_per_year, _process_factors, ) @@ -359,6 +360,7 @@ def _run_oos_cell( panel = _load_panel(cfg, symbols, logger) factors = _build_factors(cfg) panel = _maybe_enrich_financials(cfg, panel, symbols, factors, logger) + panel = _maybe_enrich_value(cfg, panel, symbols, factors, logger) panel = _maybe_enrich_covariates(cfg, panel, symbols, logger) panel = _maybe_enrich_listing(cfg, panel, symbols, logger) factor_panel = _compute_factor_panel(cfg, panel, factors, logger) diff --git a/qt/phase2_baseline.py b/qt/phase2_baseline.py index 34a0240..5194b59 100644 --- a/qt/phase2_baseline.py +++ b/qt/phase2_baseline.py @@ -52,6 +52,7 @@ _maybe_enrich_covariates, _maybe_enrich_financials, _maybe_enrich_listing, + _maybe_enrich_value, _periods_per_year, _process_factors, ) @@ -363,6 +364,7 @@ def run_phase2_baseline(config_path: str) -> Phase2Result: factors = _build_factors(cfg) primary = factors[0] panel = _maybe_enrich_financials(cfg, panel, symbols, factors, logger) + panel = _maybe_enrich_value(cfg, panel, symbols, factors, logger) panel = _maybe_enrich_covariates(cfg, panel, symbols, logger) panel = _maybe_enrich_listing(cfg, panel, symbols, logger) factor_panel = _compute_factor_panel(cfg, panel, factors, logger) diff --git a/qt/pipeline.py b/qt/pipeline.py index b9f9d7c..64c88c4 100644 --- a/qt/pipeline.py +++ b/qt/pipeline.py @@ -53,6 +53,13 @@ from data.feed.tushare_fina import TushareFinancialFeed from data.feed.tushare_flags import TushareFlagsFeed from data.store.panel_store import PanelStore +from factors.compute.candidates import ( + VALUE_FIELDS, + LiquidityFactor, + ReversalFactor, + ValueFactor, + VolatilityFactor, +) from factors.compute.financial import SUPPORTED_FIELDS as SUPPORTED_FINANCIAL_FIELDS from factors.compute.financial import FinancialFactor from factors.compute.momentum import MomentumFactor @@ -432,6 +439,7 @@ def run_phase0(config_path: str) -> Phase0Result: factors = _build_factors(cfg) primary = factors[0] panel = _maybe_enrich_financials(cfg, panel, symbols, factors, logger) + panel = _maybe_enrich_value(cfg, panel, symbols, factors, logger) panel = _maybe_enrich_covariates(cfg, panel, symbols, logger) panel = _maybe_enrich_listing(cfg, panel, symbols, logger) factor_panel = _compute_factor_panel(cfg, panel, factors, logger) @@ -670,20 +678,42 @@ def _build_factors(cfg: RootConfig) -> list: factors: list = [] for spec in enabled: params = dict(spec.params) + window = int(params.get("window", 20)) if spec.name in SUPPORTED_FINANCIAL_FIELDS: - factors.append(FinancialFactor(field=spec.name)) + factor = FinancialFactor(field=spec.name) + elif spec.name in VALUE_FIELDS: + factor = ValueFactor(spec.name) elif spec.name.startswith("momentum"): - factors.append( - MomentumFactor( - window=int(params.get("window", 20)), - price_col=str(params.get("price_col", "close")), - ) + factor = MomentumFactor( + window=window, price_col=str(params.get("price_col", "close")) + ) + elif spec.name.startswith("reversal"): + factor = ReversalFactor( + window=window, price_col=str(params.get("price_col", "close")) + ) + elif spec.name.startswith("volatility"): + factor = VolatilityFactor( + window=window, price_col=str(params.get("price_col", "close")) + ) + elif spec.name.startswith("liquidity"): + factor = LiquidityFactor( + window=window, amount_col=str(params.get("amount_col", "amount")) ) else: raise ValueError( - f"Unknown factor {spec.name!r}; expected 'momentum*' or one of " + f"Unknown factor {spec.name!r}; expected 'momentum*', 'reversal*', " + f"'volatility*', 'liquidity*', one of {VALUE_FIELDS} or one of " f"{SUPPORTED_FINANCIAL_FIELDS}." ) + # window-named factors derive their name from params: a spec named + # reversal_5 with params.window=10 would silently mislabel the column. + if factor.name != spec.name: + raise ValueError( + f"Factor name/params mismatch: config names {spec.name!r} but the " + f"params resolve to {factor.name!r} (window-named factors must " + "agree with params.window)." + ) + factors.append(factor) names = [f.name for f in factors] dupes = sorted({n for n in names if names.count(n) > 1}) if dupes: @@ -738,6 +768,54 @@ def _maybe_enrich_financials( return enriched +def _maybe_enrich_value( + cfg: RootConfig, + panel: pd.DataFrame, + symbols: list[str], + factors: list, + logger: logging.Logger, +) -> pd.DataFrame: + """Attach value_ep / value_bp columns when value factors are enabled (P3-5). + + ONE daily_basic fetch covers both fields. The published pe/pb are same-day + ratios (PIT-safe by construction); the inversion guards non-positive ratios + to NaN (a negative or zero pe/pb would otherwise flip the value ranking). + Demo has no pe/pb — a readable error, never fabricated ratios. + """ + fields = [f.name for f in factors if isinstance(f, ValueFactor)] + if not fields: + return panel + if cfg.data.source != "tushare": + raise ValueError( + f"Factor(s) {fields} need daily_basic pe/pb (data.source='tushare'); " + "they cannot run on demo data." + ) + feed = TushareCovariatesFeed( + cfg.data.external_secret_file, token_key=cfg.data.tushare_token_key + ) + ratios = feed.value_ratios(symbols, cfg.data.start, cfg.data.end) + enriched = panel.copy() + if ratios.empty: + for field in fields: + enriched[field] = float("nan") + logger.info("value: daily_basic returned no rows; %s all-NaN", fields) + return enriched + r = ratios.copy() + r["symbol"] = r["symbol"].astype(str) + r = r.set_index(["date", "symbol"]).sort_index() + inverted = { + "value_ep": 1.0 / r["pe"].where(r["pe"] > 0), + "value_bp": 1.0 / r["pb"].where(r["pb"] > 0), + } + for field in fields: + enriched[field] = inverted[field].reindex(enriched.index) + logger.info( + "value: daily_basic pe/pb enriched -> %s (%d ratio rows, single fetch)", + fields, len(r), + ) + return enriched + + def _maybe_enrich_listing( cfg: RootConfig, panel: pd.DataFrame, symbols: list[str], logger: logging.Logger ) -> pd.DataFrame: diff --git a/tests/test_candidate_factors.py b/tests/test_candidate_factors.py new file mode 100644 index 0000000..62b0bc9 --- /dev/null +++ b/tests/test_candidate_factors.py @@ -0,0 +1,317 @@ +"""P3-5: candidate factor pack (network-free). + +Locks the candidate-factor contract: + * every factor uses ONLY data known at the trade date (perturbing future bars + cannot change today's value); leading windows are NaN; computation is + strictly per-symbol (no cross-symbol leakage); inputs are never mutated; + * value_ep / value_bp surface an enriched daily_basic column (1/pe, 1/pb; + non-positive ratios -> NaN) and fail readably on the demo source; + * grossprofit_margin joins the ann_date-aligned financial fields; + * the factor dispatch (registry) builds every new name, keeps the legacy + names byte-identical, and rejects a name/params mismatch; + * a demo end-to-end run with the price-based candidates works (per-factor + analytics populated) — legacy demo numbers unchanged elsewhere. +""" + +from __future__ import annotations + +import logging +import math +from pathlib import Path + +import numpy as np +import pandas as pd +import pytest +import yaml + +from factors.compute.candidates import ( + LiquidityFactor, + ReversalFactor, + ValueFactor, + VolatilityFactor, +) +from factors.compute.financial import SUPPORTED_FIELDS, FinancialFactor +from factors.compute.momentum import MomentumFactor +from qt.config import load_config +from qt.pipeline import _build_factors, run_phase0 + +_CANDIDATES_CONFIG = str( + Path(__file__).resolve().parents[1] + / "config" / "phase3_real_factor_candidates.yaml" +) + + +def _panel(n_days=30, syms=("A", "B")): + dates = pd.bdate_range("2024-01-01", periods=n_days) + idx = pd.MultiIndex.from_product([dates, list(syms)], names=["date", "symbol"]) + close = [] + amount = [] + for i in range(n_days): + for j, _ in enumerate(syms): + close.append(100.0 + i + 50.0 * j) # deterministic per-symbol path + amount.append(1e6 * (1 + j) + 1e3 * i) + return pd.DataFrame({"close": close, "amount": amount}, index=idx) + + +# --------------------------------------------------------------------------- # +# reversal +# --------------------------------------------------------------------------- # +def test_reversal_is_negative_momentum_and_lagged_only(): + panel = _panel() + rev = ReversalFactor(window=5).compute(panel) + mom = MomentumFactor(window=5).compute(panel) + pd.testing.assert_series_equal( + rev, (-mom).rename("reversal_5"), check_names=True + ) + assert rev.name == "reversal_5" + # leading window is NaN + first_sym_a = rev.xs("A", level="symbol") + assert first_sym_a.iloc[:5].isna().all() and math.isfinite(first_sym_a.iloc[5]) + + +def test_reversal_ignores_future_bars(): + panel = _panel() + t = pd.Timestamp(panel.index.get_level_values("date").unique()[10]) + before = ReversalFactor(window=5).compute(panel).loc[(t, "A")] + poisoned = panel.copy() + future = poisoned.index.get_level_values("date") > t + poisoned.loc[future, "close"] = 9_999.0 + after = ReversalFactor(window=5).compute(poisoned).loc[(t, "A")] + assert before == after + + +# --------------------------------------------------------------------------- # +# volatility +# --------------------------------------------------------------------------- # +def test_volatility_matches_manual_rolling_std(): + panel = _panel() + vol = VolatilityFactor(window=20).compute(panel) + assert vol.name == "volatility_20" + close_a = panel.xs("A", level="symbol")["close"] + manual = close_a.pct_change().rolling(20, min_periods=20).std(ddof=1) + got = vol.xs("A", level="symbol") + pd.testing.assert_series_equal( + got, manual.rename("volatility_20"), check_names=True + ) + assert got.iloc[:20].isna().all() # needs a full window of RETURNS + + +def test_volatility_is_per_symbol_no_cross_leakage(): + panel = _panel() + # poison symbol B entirely; A's volatility must not move. + before = VolatilityFactor(window=20).compute(panel).xs("A", level="symbol") + poisoned = panel.copy() + b_rows = poisoned.index.get_level_values("symbol") == "B" + poisoned.loc[b_rows, "close"] = 1.0 + after = VolatilityFactor(window=20).compute(poisoned).xs("A", level="symbol") + pd.testing.assert_series_equal(before, after) + + +# --------------------------------------------------------------------------- # +# liquidity +# --------------------------------------------------------------------------- # +def test_liquidity_is_log_mean_amount(): + panel = _panel() + liq = LiquidityFactor(window=20).compute(panel) + assert liq.name == "liquidity_20" + amt_a = panel.xs("A", level="symbol")["amount"] + manual = np.log(amt_a.rolling(20, min_periods=20).mean()) + pd.testing.assert_series_equal( + liq.xs("A", level="symbol"), manual.rename("liquidity_20"), + check_names=True, + ) + + +def test_liquidity_nonpositive_amount_is_nan_not_crash(): + panel = _panel() + panel = panel.copy() + panel.loc[panel.index[:8], "amount"] = 0.0 # degenerate turnover + liq = LiquidityFactor(window=4).compute(panel) + assert not np.isinf(liq.dropna()).any() # log(0) never leaks as -inf + + +def test_liquidity_requires_amount_column(): + panel = _panel().drop(columns=["amount"]) + with pytest.raises(ValueError, match="amount"): + LiquidityFactor(window=20).compute(panel) + + +# --------------------------------------------------------------------------- # +# value (daily_basic enrichment surface) +# --------------------------------------------------------------------------- # +def test_value_factor_surfaces_enriched_column(): + panel = _panel().copy() + panel["value_ep"] = 0.05 + out = ValueFactor("value_ep").compute(panel) + assert out.name == "value_ep" + assert (out == 0.05).all() + + +def test_value_factor_missing_column_is_readable_error(): + with pytest.raises(ValueError, match="value_ep"): + ValueFactor("value_ep").compute(_panel()) + + +def test_value_factor_rejects_unknown_field(): + with pytest.raises(ValueError, match="value_ep|value_bp"): + ValueFactor("value_cashflow") + + +# --------------------------------------------------------------------------- # +# quality financial field +# --------------------------------------------------------------------------- # +def test_grossprofit_margin_is_a_supported_financial_field(): + assert "grossprofit_margin" in SUPPORTED_FIELDS + factor = FinancialFactor("grossprofit_margin") + assert factor.name == "grossprofit_margin" + + +# --------------------------------------------------------------------------- # +# dispatch (factor registry) +# --------------------------------------------------------------------------- # +def _cfg_with(tmp_path, example_config_path, factors, source="demo"): + raw = yaml.safe_load(Path(example_config_path).read_text(encoding="utf-8")) + raw["data"]["source"] = source + raw["factors"] = factors + out = tmp_path / "artifacts" + raw["output"] = { + "root_dir": str(out), "data_dir": str(out / "data"), + "factor_dir": str(out / "factors"), "report_dir": str(out / "reports"), + "log_dir": str(out / "logs"), "overwrite": True, + } + p = tmp_path / "cfg.yaml" + p.write_text(yaml.safe_dump(raw), encoding="utf-8") + return load_config(str(p)) + + +def test_dispatch_builds_every_candidate(tmp_path, example_config_path): + specs = [ + {"name": "momentum_20", "enabled": True, "params": {"window": 20}}, + {"name": "reversal_5", "enabled": True, "params": {"window": 5}}, + {"name": "reversal_20", "enabled": True, "params": {"window": 20}}, + {"name": "volatility_20", "enabled": True, "params": {"window": 20}}, + {"name": "liquidity_20", "enabled": True, "params": {"window": 20}}, + {"name": "value_ep", "enabled": True, "params": {}}, + {"name": "value_bp", "enabled": True, "params": {}}, + {"name": "roe", "enabled": True, "params": {}}, + {"name": "grossprofit_margin", "enabled": True, "params": {}}, + ] + cfg = _cfg_with(tmp_path, example_config_path, specs, source="tushare") + factors = _build_factors(cfg) + assert [f.name for f in factors] == [s["name"] for s in specs] + assert isinstance(factors[1], ReversalFactor) + assert isinstance(factors[3], VolatilityFactor) + assert isinstance(factors[4], LiquidityFactor) + assert isinstance(factors[5], ValueFactor) + assert isinstance(factors[8], FinancialFactor) + + +def test_dispatch_rejects_name_params_mismatch(tmp_path, example_config_path): + # spec says reversal_5 but params build reversal_10 -> a silent mislabel. + specs = [{"name": "reversal_5", "enabled": True, "params": {"window": 10}}] + cfg = _cfg_with(tmp_path, example_config_path, specs) + with pytest.raises(ValueError, match="mismatch|resolve"): + _build_factors(cfg) + + +# --------------------------------------------------------------------------- # +# value enrichment wiring +# --------------------------------------------------------------------------- # +def test_value_enrichment_demo_source_raises(tmp_path, example_config_path): + from qt.pipeline import _maybe_enrich_value + + specs = [{"name": "value_ep", "enabled": True, "params": {}}] + cfg = _cfg_with(tmp_path, example_config_path, specs, source="demo") + with pytest.raises(ValueError, match="demo"): + _maybe_enrich_value( + cfg, _panel(), ["000001.SZ"], [ValueFactor("value_ep")], + logging.getLogger("test"), + ) + + +def test_value_enrichment_inverts_ratios_and_guards_nonpositive( + tmp_path, example_config_path, monkeypatch +): + from qt.pipeline import _maybe_enrich_value + + specs = [{"name": "value_ep", "enabled": True, "params": {}}, + {"name": "value_bp", "enabled": True, "params": {}}] + cfg = _cfg_with(tmp_path, example_config_path, specs, source="tushare") + calls = [] + + class _FakeFeed: + def __init__(self, *a, **k): + pass + + def value_ratios(self, symbols, start, end): + calls.append(list(symbols)) + return pd.DataFrame( + { + "date": pd.to_datetime(["2024-01-01", "2024-01-02", + "2024-01-03"]), + "symbol": ["A", "A", "A"], + "pe": [20.0, -5.0, None], # negative / missing -> NaN + "pb": [2.0, 0.0, 4.0], # zero -> NaN + } + ) + + monkeypatch.setattr("qt.pipeline.TushareCovariatesFeed", _FakeFeed) + idx = pd.MultiIndex.from_product( + [pd.to_datetime(["2024-01-01", "2024-01-02", "2024-01-03"]), ["A"]], + names=["date", "symbol"], + ) + panel = pd.DataFrame({"close": [1.0, 1.1, 1.2]}, index=idx) + out = _maybe_enrich_value( + cfg, panel, ["A"], + [ValueFactor("value_ep"), ValueFactor("value_bp")], + logging.getLogger("test"), + ) + assert len(calls) == 1 # ONE fetch for both value fields + assert out.loc[(pd.Timestamp("2024-01-01"), "A"), "value_ep"] == pytest.approx(1 / 20) + assert math.isnan(out.loc[(pd.Timestamp("2024-01-02"), "A"), "value_ep"]) # pe<0 + assert math.isnan(out.loc[(pd.Timestamp("2024-01-03"), "A"), "value_ep"]) # pe NaN + assert out.loc[(pd.Timestamp("2024-01-01"), "A"), "value_bp"] == pytest.approx(0.5) + assert math.isnan(out.loc[(pd.Timestamp("2024-01-02"), "A"), "value_bp"]) # pb==0 + assert "value_ep" not in panel.columns # input not mutated + + +# --------------------------------------------------------------------------- # +# demo e2e with the price-based candidates + config validation +# --------------------------------------------------------------------------- # +def test_demo_e2e_with_price_candidates(tmp_path, example_config_path): + specs = [ + {"name": "momentum_20", "enabled": True, "params": {"window": 20}}, + {"name": "reversal_5", "enabled": True, "params": {"window": 5}}, + {"name": "volatility_20", "enabled": True, "params": {"window": 20}}, + {"name": "liquidity_20", "enabled": True, "params": {"window": 20}}, + ] + raw = yaml.safe_load(Path(example_config_path).read_text(encoding="utf-8")) + raw["factors"] = specs + out = tmp_path / "artifacts" + raw["output"] = { + "root_dir": str(out), "data_dir": str(out / "data"), + "factor_dir": str(out / "factors"), "report_dir": str(out / "reports"), + "log_dir": str(out / "logs"), "overwrite": True, + } + p = tmp_path / "cfg.yaml" + p.write_text(yaml.safe_dump(raw), encoding="utf-8") + result = run_phase0(str(p)) + assert result.factor_names == ( + "momentum_20", "reversal_5", "volatility_20", "liquidity_20" + ) + for name in result.factor_names: + assert name in result.per_factor + text = result.report_path.read_text(encoding="utf-8") + assert "reversal_5" in text and "liquidity_20" in text + assert "token" not in text.lower() + + +def test_candidates_config_validates(): + cfg = load_config(_CANDIDATES_CONFIG) + names = [f.name for f in cfg.factors if f.enabled] + # legacy trio kept for the old-vs-new comparison + the candidate pack + assert {"momentum_20", "roe", "netprofit_yoy"} <= set(names) + assert {"reversal_5", "reversal_20", "volatility_20", "liquidity_20", + "value_ep", "value_bp", "grossprofit_margin"} <= set(names) + assert cfg.robustness is not None # runs through the P3-4 matrix + assert cfg.alpha.model == "ic_weighted" diff --git a/tests/test_multifactor_pipeline.py b/tests/test_multifactor_pipeline.py index 7469c58..b24eaa5 100644 --- a/tests/test_multifactor_pipeline.py +++ b/tests/test_multifactor_pipeline.py @@ -89,7 +89,7 @@ def test_build_factors_mixed_price_and_financial(tmp_path, example_config_path): def test_build_factors_duplicate_names_raise(tmp_path, example_config_path): specs = [ {"name": "momentum_20", "enabled": True, "params": {"window": 20}}, - {"name": "momentum_x", "enabled": True, "params": {"window": 20}}, # same name + {"name": "momentum_20", "enabled": True, "params": {"window": 20}}, # dup ] cfg = load_config(str(_write_cfg(tmp_path, example_config_path, specs))) with pytest.raises(ValueError, match="[Dd]uplicate"): From 479099741a6358c55a5046127b647e3a99a146b0 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Thu, 11 Jun 2026 10:33:11 +0800 Subject: [PATCH 2/4] feat(factors): add overnight_mom_20 to the candidate pack overnight_ret[t] = log(open[t] / close[t-1]); overnight_mom_20[t] = the trailing 20-day sum. PIT: open[t] is known at the t open and the factor is computed at the t close; prices are front-adjusted by one anchor so the ratio is ex-dividend-safe; the t-1 close never crosses symbols (grouped shift); non-positive prices -> NaN (never -inf); leading window NaN (w returns need w+1 bars). Wired into the dispatch (overnight_mom*) and into config/phase3_real_factor_candidates.yaml (now legacy trio + 8 candidates). +5 unit tests (manual-reference math, future-bar invariance, per-symbol isolation, non-positive guard, missing-column error); dispatch + config tests extended. 322 passed; demo unchanged. Real matrix rerun with the extended pack is in flight; numbers land in a follow-up commit + the PR body. --- AGENTS.md | 6 +- CLAUDE.md | 6 +- RUNBOOK.md | 3 +- TEST_REPORT.md | 10 ++-- config/phase3_real_factor_candidates.yaml | 4 ++ factors/compute/candidates.py | 48 +++++++++++++++ qt/pipeline.py | 11 +++- tests/test_candidate_factors.py | 73 +++++++++++++++++++++-- 8 files changed, 142 insertions(+), 19 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 96ca09d..7000764 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -117,14 +117,14 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**（sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重）。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。** - secret scan 报告 0 处;demo 0.96/0.84 不变。 - 🔧 **Phase 3-5 factor candidate pack**（`p3-factor-candidate-pack` 分支,代劳待验收,**EXPLORATORY**）：保守日频 PIT-safe 候选因子组,经 P3-4 矩阵原样复检"弱信号是否只是因子集太窄"。不动 alpha/portfolio/execution/OOS/robustness、不调参。 - - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 + - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`overnight_mom_20`(Σlog(open_t/close_{t-1}) 20 日,open 当日开盘已知,prev close 不跨 symbol)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 - dispatch（registry）扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。 - - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选七 = 10 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 + - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 - **真实矩阵（3 cells / 2.3h;旧三因子 per-cell IC 与 P3-4 逐数一致=不漂移 smoke ✓;secret scan 0）**: - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;reversal/grossprofit_margin 翻号无信号。 - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−34.6%**)——等权对因子集质量敏感。 - 组合绩效:ic test −3.62%/−4.88%/−2.64%(2/3 cells 跑赢等权;CSI300 train +23.69%→test −2.64%,train→test 衰减仍显著)。 - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 -- ✅ 质量门：`pytest` **317 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=17）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 +- ✅ 质量门：`pytest` **322 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 - ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、旧三因子无信号（P3-3/P3-4 实证）;候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检（P3-5）。 - 路线图下一步：value+lowvol 子集独立复检 / 成本敏感性,或分钟级（architecture.html §11）。 diff --git a/CLAUDE.md b/CLAUDE.md index 96ca09d..7000764 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -117,14 +117,14 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - **关键负面发现**:ic_weighted 组合绩效不稳健——train→test 翻车形态出现在 **3 cells 中的 2 个**:SSE50|2020-2022 train **+7.09%→test −7.85%**、CSI300|2022-2024 train **+11.07%→test −17.74%**（sharpe +0.59→−1.01;CSI300 换手 1.2~1.6 成本拖累更重）。**凡 ic_weighted 样本内显著为正,样本外即翻负——典型过拟合签名;P3-2 的"跑赢"不能外推;微弱正 IC ≠ 净值赢。非收益声明。** - secret scan 报告 0 处;demo 0.96/0.84 不变。 - 🔧 **Phase 3-5 factor candidate pack**（`p3-factor-candidate-pack` 分支,代劳待验收,**EXPLORATORY**）：保守日频 PIT-safe 候选因子组,经 P3-4 矩阵原样复检"弱信号是否只是因子集太窄"。不动 alpha/portfolio/execution/OOS/robustness、不调参。 - - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 + - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`overnight_mom_20`(Σlog(open_t/close_{t-1}) 20 日,open 当日开盘已知,prev close 不跨 symbol)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 - dispatch（registry）扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。 - - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选七 = 10 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 + - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 - **真实矩阵（3 cells / 2.3h;旧三因子 per-cell IC 与 P3-4 逐数一致=不漂移 smoke ✓;secret scan 0）**: - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;reversal/grossprofit_margin 翻号无信号。 - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−34.6%**)——等权对因子集质量敏感。 - 组合绩效:ic test −3.62%/−4.88%/−2.64%(2/3 cells 跑赢等权;CSI300 train +23.69%→test −2.64%,train→test 衰减仍显著)。 - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 -- ✅ 质量门：`pytest` **317 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=17）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 +- ✅ 质量门：`pytest` **322 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 - ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、旧三因子无信号（P3-3/P3-4 实证）;候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检（P3-5）。 - 路线图下一步：value+lowvol 子集独立复检 / 成本敏感性,或分钟级（architecture.html §11）。 diff --git a/RUNBOOK.md b/RUNBOOK.md index 3509e41..28a7b47 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -372,11 +372,12 @@ no-future-bars, per-symbol isolation, immutability): | `reversal_5/20` | −(close[t]/close[t−w] − 1) | exact negative of momentum (≤t bars) | | `volatility_20` | std of trailing 20 daily returns (ddof=1, full window) | ≤t bars | | `liquidity_20` | log(mean(amount, 20)) (non-positive → NaN) | same-day bar amount | +| `overnight_mom_20` | Σ log(open[t]/close[t−1]) over 20d (non-positive → NaN) | open known at the t open; computed at the t close | | `value_ep` / `value_bp` | 1/pe, 1/pb from `daily_basic` (one fetch; ≤0 → NaN) | ratios published same-day | | `grossprofit_margin` | financial quality field | ann_date as-of (existing machinery) | Registry / dispatch: `_build_factors` resolves `reversal*` / `volatility*` / -`liquidity*` / value fields / financial fields; window-named factors must agree +`liquidity*` / `overnight_mom*` / value fields / financial fields; window-named factors must agree with `params.window` (a mismatch is a readable config error, never a silent mislabel). Legacy configs are untouched and reproduce their numbers. diff --git a/TEST_REPORT.md b/TEST_REPORT.md index 8641ae9..36801d9 100644 --- a/TEST_REPORT.md +++ b/TEST_REPORT.md @@ -27,7 +27,7 @@ Run from the repo root with the project python (env `quant_mf`): | Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml` + `phase3_real_factor_candidates.yaml`) | exit `0`, prints `OK` | | End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` | -Counts below are the actual per-file `pytest` numbers (sum = 317). +Counts below are the actual per-file `pytest` numbers (sum = 322). ## Per-file breakdown — Phase 0 core (97) @@ -125,13 +125,13 @@ Counts below are the actual per-file `pytest` numbers (sum = 317). |---|---|---| | `test_robustness_matrix.py` | 14 | matrix config validation (universes/windows/unique labels, split-inside-window, unknown-skip + all-skipped ConfigErrors); cell enumeration excludes skips in config order; per-cell config derivation swaps ONLY the cell identity (incl. unique per-cell output_name; derived cell passes the shared OOS preconditions); cross-cell aggregation attributes strictly per cell (no mixing, no dilution); runner guards (demo source / missing robustness section / non-ic_weighted alpha); report renders cell labels + skipped-cell disclosure + boundary/fallback/sign-consistency + cross-cell summary + caveat + no secret | -## Per-file breakdown — Phase 3-5 factor candidates (17 + 1 reworked) +## Per-file breakdown — Phase 3-5 factor candidates (22 + 1 reworked) | Test file | Tests | Red-line / feature | |---|---|---| -| `test_candidate_factors.py` | 17 | per-factor math vs manual references (reversal == −momentum, volatility rolling std, liquidity log-mean-amount); leading-window NaN; future-bar perturbation invariance; per-symbol isolation; non-positive amount → NaN never −inf; missing-column readable errors; ValueFactor surfaces enriched column + rejects unknown fields; grossprofit_margin in SUPPORTED_FIELDS; dispatch builds every candidate + rejects name/params mismatch; value enrichment ONE fetch for both fields + 1/pe 1/pb math + non-positive guards + demo readable error + input immutability; demo e2e with price candidates (per-factor analytics, report, no secret); candidates config validates | +| `test_candidate_factors.py` | 22 | per-factor math vs manual references (reversal == −momentum, volatility rolling std, liquidity log-mean-amount, overnight Σlog(open/prev-close) incl. prev-close-never-crosses-symbols); leading-window NaN; future-bar perturbation invariance; per-symbol isolation; non-positive amount → NaN never −inf; missing-column readable errors; ValueFactor surfaces enriched column + rejects unknown fields; grossprofit_margin in SUPPORTED_FIELDS; dispatch builds every candidate + rejects name/params mismatch; value enrichment ONE fetch for both fields + 1/pe 1/pb math + non-positive guards + demo readable error + input immutability; demo e2e with price candidates (per-factor analytics, report, no secret); candidates config validates | | `test_multifactor_pipeline.py` (reworked 1) | — | duplicate-name test now uses a true duplicate spec (the new name/params-mismatch guard catches the old mislabel construction earlier) | -| **Total (P0 + P1 + P2-1..P2-4 + P3-1..P3-5)** | **317** | | +| **Total (P0 + P1 + P2-1..P2-4 + P3-1..P3-5)** | **322** | | ## Real-data validation (manual, not in CI — TEST-002 keeps the suite network-free) @@ -221,7 +221,7 @@ Counts below are the actual per-file `pytest` numbers (sum = 317). config-validated and disclosed (never silent); cross-cell aggregation is strictly per cell. Single-run `run-phase3-oos` behaviour is unchanged. - **P3-5 factor candidates (locked by tests):** conservative daily PIT-safe - candidates (reversal_5/20, volatility_20, liquidity_20, value_ep/bp via one + candidates (reversal_5/20, volatility_20, liquidity_20, overnight_mom_20, value_ep/bp via one daily_basic fetch with non-positive guards, grossprofit_margin via the existing ann_date machinery). The dispatch rejects a window-name/params mismatch; legacy configs reproduce their numbers (demo ic 0.96 / annual diff --git a/config/phase3_real_factor_candidates.yaml b/config/phase3_real_factor_candidates.yaml index 743c5f2..71c5c3b 100644 --- a/config/phase3_real_factor_candidates.yaml +++ b/config/phase3_real_factor_candidates.yaml @@ -76,6 +76,10 @@ factors: enabled: true params: window: 20 + - name: overnight_mom_20 # sum of log(open_t/close_{t-1}) over 20d (open known at t) + enabled: true + params: + window: 20 - name: value_ep # 1/pe from daily_basic (same-day, PIT-safe; pe<=0 -> NaN) enabled: true params: {} diff --git a/factors/compute/candidates.py b/factors/compute/candidates.py index 7216038..94b40c9 100644 --- a/factors/compute/candidates.py +++ b/factors/compute/candidates.py @@ -123,6 +123,54 @@ def compute(self, panel: pd.DataFrame) -> pd.Series: return np.log(safe).reindex(panel.index).rename(self.name) +class OvernightMomentumFactor(Factor): + """Cumulative overnight (close→open) log return over a fixed window. + + overnight_ret[t] = log(open[t] / close[t-1]) + overnight_mom_w[t] = sum(overnight_ret[t-w+1 .. t]) + + PIT argument: open[t] is known at the t open and close[t-1] the prior + close, so the value at t uses only <=t information (factors are computed at + the t close). Both prices are front-adjusted by the same anchor, so the + ratio is ex-dividend-safe. Computation is strictly per symbol (the t-1 + close never crosses symbols); non-positive prices map to NaN (never a + silent ``-inf``); the leading window is NaN (min_periods = w full overnight + returns, which need w+1 bars). + """ + + name: str = "overnight_mom_20" + + def __init__( + self, window: int = 20, open_col: str = "open", close_col: str = "close" + ) -> None: + if not isinstance(window, int) or window < 1: + raise ValueError( + f"overnight momentum window must be a positive integer, got {window!r}." + ) + self._window = window + self._open_col = open_col + self._close_col = close_col + self.name = f"overnight_mom_{window}" + + def compute(self, panel: pd.DataFrame) -> pd.Series: + missing = [c for c in (self._open_col, self._close_col) if c not in panel.columns] + if missing: + raise ValueError( + f"overnight momentum factor needs {missing} column(s); panel has " + f"{list(panel.columns)}." + ) + open_px = panel[self._open_col] + close_px = panel[self._close_col] + # strictly-lagged close within each symbol; never crosses symbols. + prev_close = close_px.groupby(level="symbol").shift(1) + ratio = (open_px / prev_close).where((open_px > 0) & (prev_close > 0)) + overnight = np.log(ratio) + mom = overnight.groupby(level="symbol", group_keys=False).apply( + lambda s: s.rolling(self._window, min_periods=self._window).sum() + ) + return mom.reindex(panel.index).rename(self.name) + + class ValueFactor(Factor): """Surface a daily_basic-enriched value column (``value_ep`` / ``value_bp``). diff --git a/qt/pipeline.py b/qt/pipeline.py index 64c88c4..8ac0c6c 100644 --- a/qt/pipeline.py +++ b/qt/pipeline.py @@ -56,6 +56,7 @@ from factors.compute.candidates import ( VALUE_FIELDS, LiquidityFactor, + OvernightMomentumFactor, ReversalFactor, ValueFactor, VolatilityFactor, @@ -683,6 +684,12 @@ def _build_factors(cfg: RootConfig) -> list: factor = FinancialFactor(field=spec.name) elif spec.name in VALUE_FIELDS: factor = ValueFactor(spec.name) + elif spec.name.startswith("overnight_mom"): + factor = OvernightMomentumFactor( + window=window, + open_col=str(params.get("open_col", "open")), + close_col=str(params.get("close_col", "close")), + ) elif spec.name.startswith("momentum"): factor = MomentumFactor( window=window, price_col=str(params.get("price_col", "close")) @@ -702,8 +709,8 @@ def _build_factors(cfg: RootConfig) -> list: else: raise ValueError( f"Unknown factor {spec.name!r}; expected 'momentum*', 'reversal*', " - f"'volatility*', 'liquidity*', one of {VALUE_FIELDS} or one of " - f"{SUPPORTED_FINANCIAL_FIELDS}." + f"'volatility*', 'liquidity*', 'overnight_mom*', one of " + f"{VALUE_FIELDS} or one of {SUPPORTED_FINANCIAL_FIELDS}." ) # window-named factors derive their name from params: a spec named # reversal_5 with params.window=10 would silently mislabel the column. diff --git a/tests/test_candidate_factors.py b/tests/test_candidate_factors.py index 62b0bc9..fecba7e 100644 --- a/tests/test_candidate_factors.py +++ b/tests/test_candidate_factors.py @@ -26,6 +26,7 @@ from factors.compute.candidates import ( LiquidityFactor, + OvernightMomentumFactor, ReversalFactor, ValueFactor, VolatilityFactor, @@ -45,12 +46,17 @@ def _panel(n_days=30, syms=("A", "B")): dates = pd.bdate_range("2024-01-01", periods=n_days) idx = pd.MultiIndex.from_product([dates, list(syms)], names=["date", "symbol"]) close = [] + open_ = [] amount = [] for i in range(n_days): for j, _ in enumerate(syms): - close.append(100.0 + i + 50.0 * j) # deterministic per-symbol path + c = 100.0 + i + 50.0 * j # deterministic per-symbol path + close.append(c) + open_.append(c - 0.4 - 0.1 * j) # a stable overnight gap amount.append(1e6 * (1 + j) + 1e3 * i) - return pd.DataFrame({"close": close, "amount": amount}, index=idx) + return pd.DataFrame( + {"open": open_, "close": close, "amount": amount}, index=idx + ) # --------------------------------------------------------------------------- # @@ -136,6 +142,60 @@ def test_liquidity_requires_amount_column(): LiquidityFactor(window=20).compute(panel) +# --------------------------------------------------------------------------- # +# overnight momentum +# --------------------------------------------------------------------------- # +def test_overnight_mom_matches_manual_sum_of_log_gaps(): + panel = _panel() + out = OvernightMomentumFactor(window=20).compute(panel) + assert out.name == "overnight_mom_20" + a = panel.xs("A", level="symbol") + manual = np.log(a["open"] / a["close"].shift(1)).rolling( + 20, min_periods=20 + ).sum() + pd.testing.assert_series_equal( + out.xs("A", level="symbol"), manual.rename("overnight_mom_20"), + check_names=True, + ) + # needs w overnight returns (= w+1 bars): rows 0..19 are NaN, row 20 finite + got_a = out.xs("A", level="symbol") + assert got_a.iloc[:20].isna().all() and math.isfinite(got_a.iloc[20]) + + +def test_overnight_mom_ignores_future_bars(): + panel = _panel() + t = pd.Timestamp(panel.index.get_level_values("date").unique()[25]) + before = OvernightMomentumFactor(window=20).compute(panel).loc[(t, "A")] + poisoned = panel.copy() + future = poisoned.index.get_level_values("date") > t + poisoned.loc[future, ["open", "close"]] = 9_999.0 + after = OvernightMomentumFactor(window=20).compute(poisoned).loc[(t, "A")] + assert before == after + + +def test_overnight_mom_prev_close_never_crosses_symbols(): + panel = _panel() + before = OvernightMomentumFactor(window=5).compute(panel).xs("A", level="symbol") + poisoned = panel.copy() + b_rows = poisoned.index.get_level_values("symbol") == "B" + poisoned.loc[b_rows, ["open", "close"]] = 1.0 + after = OvernightMomentumFactor(window=5).compute(poisoned).xs("A", level="symbol") + pd.testing.assert_series_equal(before, after) + + +def test_overnight_mom_nonpositive_prices_are_nan_not_inf(): + panel = _panel().copy() + panel.loc[panel.index[:6], "open"] = 0.0 + out = OvernightMomentumFactor(window=3).compute(panel) + assert not np.isinf(out.dropna()).any() + + +def test_overnight_mom_requires_open_column(): + panel = _panel().drop(columns=["open"]) + with pytest.raises(ValueError, match="open"): + OvernightMomentumFactor(window=20).compute(panel) + + # --------------------------------------------------------------------------- # # value (daily_basic enrichment surface) # --------------------------------------------------------------------------- # @@ -191,6 +251,7 @@ def test_dispatch_builds_every_candidate(tmp_path, example_config_path): {"name": "reversal_20", "enabled": True, "params": {"window": 20}}, {"name": "volatility_20", "enabled": True, "params": {"window": 20}}, {"name": "liquidity_20", "enabled": True, "params": {"window": 20}}, + {"name": "overnight_mom_20", "enabled": True, "params": {"window": 20}}, {"name": "value_ep", "enabled": True, "params": {}}, {"name": "value_bp", "enabled": True, "params": {}}, {"name": "roe", "enabled": True, "params": {}}, @@ -202,8 +263,9 @@ def test_dispatch_builds_every_candidate(tmp_path, example_config_path): assert isinstance(factors[1], ReversalFactor) assert isinstance(factors[3], VolatilityFactor) assert isinstance(factors[4], LiquidityFactor) - assert isinstance(factors[5], ValueFactor) - assert isinstance(factors[8], FinancialFactor) + assert isinstance(factors[5], OvernightMomentumFactor) + assert isinstance(factors[6], ValueFactor) + assert isinstance(factors[9], FinancialFactor) def test_dispatch_rejects_name_params_mismatch(tmp_path, example_config_path): @@ -312,6 +374,7 @@ def test_candidates_config_validates(): # legacy trio kept for the old-vs-new comparison + the candidate pack assert {"momentum_20", "roe", "netprofit_yoy"} <= set(names) assert {"reversal_5", "reversal_20", "volatility_20", "liquidity_20", - "value_ep", "value_bp", "grossprofit_margin"} <= set(names) + "overnight_mom_20", "value_ep", "value_bp", + "grossprofit_margin"} <= set(names) assert cfg.robustness is not None # runs through the P3-4 matrix assert cfg.alpha.model == "ic_weighted" From a4659714f3e76b4091974563ee571d62a6a0ea92 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Thu, 11 Jun 2026 13:28:29 +0800 Subject: [PATCH 3/4] docs: 11-factor matrix real numbers (incl. overnight_mom_20) Real rerun with the extended pack (3 cells, 2h; one CSI300 network failure -> full clean rerun): all 10 prior series' per-cell ICs unchanged (per-column independence re-confirmed); overnight_mom_20 test IC 2/3 positive (+0.008/+0.016; negative in 2020-2022) -- a moderate-weak signal. ic_weighted now beats equal weight in 3/3 cells (test -2.21%/-5.02%/-2.76% vs eq -4.30%/-6.83%/-25.16%); CSI300 train +23.11% -> test -2.76% (decay remains). Caveats unchanged: exploratory, overlapping windows, no cost sensitivity. --- AGENTS.md | 8 ++++---- CLAUDE.md | 8 ++++---- 2 files changed, 8 insertions(+), 8 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 7000764..01c2fc7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -120,10 +120,10 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`overnight_mom_20`(Σlog(open_t/close_{t-1}) 20 日,open 当日开盘已知,prev close 不跨 symbol)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 - dispatch（registry）扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。 - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 - - **真实矩阵（3 cells / 2.3h;旧三因子 per-cell IC 与 P3-4 逐数一致=不漂移 smoke ✓;secret scan 0）**: - - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;reversal/grossprofit_margin 翻号无信号。 - - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−34.6%**)——等权对因子集质量敏感。 - - 组合绩效:ic test −3.62%/−4.88%/−2.64%(2/3 cells 跑赢等权;CSI300 train +23.69%→test −2.64%,train→test 衰减仍显著)。 + - **真实矩阵（11 因子版 3 cells / 2h;旧三因子+前轮候选 per-cell IC 逐数不变=不漂移 smoke ✓;secret scan 0;一次 CSI300 网络故障后整 run 重跑成功）**: + - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;`overnight_mom_20` test IC 2/3 正(+0.008/+0.016,2020-22 为负)中等偏弱;reversal/grossprofit_margin 翻号无信号。 + - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。 + - 组合绩效:ic test −2.21%/−5.02%/−2.76%(**3/3 cells 跑赢等权**;CSI300 train +23.11%→test −2.76%,train→test 衰减仍显著)。 - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 - ✅ 质量门：`pytest` **322 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 - ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、旧三因子无信号（P3-3/P3-4 实证）;候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检（P3-5）。 diff --git a/CLAUDE.md b/CLAUDE.md index 7000764..01c2fc7 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -120,10 +120,10 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - 新因子（`factors/compute/candidates.py`,数学/前导 NaN/未来扰动不变/跨 symbol 隔离/不可变全测试锁定）:`reversal_5/20`(=−momentum 复用同机器)/`volatility_20`(滚动收益 std,满窗)/`liquidity_20`(log 滚动均额,非正→NaN)/`overnight_mom_20`(Σlog(open_t/close_{t-1}) 20 日,open 当日开盘已知,prev close 不跨 symbol)/`value_ep`/`value_bp`(daily_basic pe/pb **一次 fetch** 取倒数,≤0→NaN,当日发布 PIT-safe)/`grossprofit_margin`(进 SUPPORTED_FIELDS,走既有 ann_date as-of)。 - dispatch（registry）扩展;**窗口名/params 不一致 → 可读报错**(绝不静默错标列);旧配置逐 bit 复现(demo 0.96/0.84 不变)。 - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 - - **真实矩阵（3 cells / 2.3h;旧三因子 per-cell IC 与 P3-4 逐数一致=不漂移 smoke ✓;secret scan 0）**: - - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;reversal/grossprofit_margin 翻号无信号。 - - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−34.6%**)——等权对因子集质量敏感。 - - 组合绩效:ic test −3.62%/−4.88%/−2.64%(2/3 cells 跑赢等权;CSI300 train +23.69%→test −2.64%,train→test 衰减仍显著)。 + - **真实矩阵（11 因子版 3 cells / 2h;旧三因子+前轮候选 per-cell IC 逐数不变=不漂移 smoke ✓;secret scan 0;一次 CSI300 网络故障后整 run 重跑成功）**: + - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;`overnight_mom_20` test IC 2/3 正(+0.008/+0.016,2020-22 为负)中等偏弱;reversal/grossprofit_margin 翻号无信号。 + - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。 + - 组合绩效:ic test −2.21%/−5.02%/−2.76%(**3/3 cells 跑赢等权**;CSI300 train +23.11%→test −2.76%,train→test 衰减仍显著)。 - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 - ✅ 质量门：`pytest` **322 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 - ⚠️ 剩余（已显式披露）：日线 only、demo 路径非真数据、旧三因子无信号（P3-3/P3-4 实证）;候选 value/低波信号为 EXPLORATORY 发现,待独立窗口/成本敏感性复检（P3-5）。 From a4eee000b566acb9fe485d1487d8c3d37635dd7b Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Thu, 11 Jun 2026 14:38:51 +0800 Subject: [PATCH 4/4] docs: fix stale counts/numbers vs the 11-factor real run Review findings on PR #16, doc-only: - TEST_REPORT top results line said 299 passed while the breakdown summed to 322 (two earlier count bumps never matched the stale line) -> 322. - RUNBOOK said 'legacy trio + 7 candidates' -> 8 (overnight_mom_20). - CLAUDE/AGENTS said '10-factor equal-weight combo' and carried the 10-factor run's combo_ic_weighted ICs (0.0255/0.0014/0.0402) -> 11-factor wording and the real report's 0.0253/0.0012/0.0395. Verified by grep against artifacts/reports/phase3_robustness_matrix.md; no matrix rerun needed (display-only). --- AGENTS.md | 2 +- CLAUDE.md | 2 +- RUNBOOK.md | 2 +- TEST_REPORT.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 01c2fc7..30a2695 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -122,7 +122,7 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 - **真实矩阵（11 因子版 3 cells / 2h;旧三因子+前轮候选 per-cell IC 逐数不变=不漂移 smoke ✓;secret scan 0;一次 CSI300 网络故障后整 run 重跑成功）**: - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;`overnight_mom_20` test IC 2/3 正(+0.008/+0.016,2020-22 为负)中等偏弱;reversal/grossprofit_margin 翻号无信号。 - - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。 + - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0253/0.0012/0.0395)——walk-forward IC 加权吃到了新信号;11 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。 - 组合绩效:ic test −2.21%/−5.02%/−2.76%(**3/3 cells 跑赢等权**;CSI300 train +23.11%→test −2.76%,train→test 衰减仍显著)。 - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 - ✅ 质量门：`pytest` **322 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 diff --git a/CLAUDE.md b/CLAUDE.md index 01c2fc7..30a2695 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -122,7 +122,7 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - `config/phase3_real_factor_candidates.yaml`:旧三 + 候选八 = 11 因子,同 P3-4 矩阵形状(CSI300×2020-2022 显式 skip 披露);**旧 vs 新一跑读出**(raw IC 按列独立)。 - **真实矩阵（11 因子版 3 cells / 2h;旧三因子+前轮候选 per-cell IC 逐数不变=不漂移 smoke ✓;secret scan 0;一次 CSI300 网络故障后整 run 重跑成功）**: - **"因子集太窄"假设获得支持**:`value_ep`/`value_bp` test IC **3/3 正、0.037~0.056**;`volatility_20` **3/3 负、−0.044~−0.079**(低波方向稳定)——比旧三因子(|IC|≤0.015)高一个量级且方向跨 cell 一致。liquidity 3/3 负但量级小;`overnight_mom_20` test IC 2/3 正(+0.008/+0.016,2020-22 为负)中等偏弱;reversal/grossprofit_margin 翻号无信号。 - - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0255/0.0014/0.0402)——walk-forward IC 加权吃到了新信号;10 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。 + - `combo_ic_weighted` test IC **3/3 正 + 3/3 sign consistent**(0.0253/0.0012/0.0395)——walk-forward IC 加权吃到了新信号;11 因子**等权** combo 反被翻号因子稀释(IC 0/3 正,CSI300 test **−25.2%**)——等权对因子集质量敏感。 - 组合绩效:ic test −2.21%/−5.02%/−2.76%(**3/3 cells 跑赢等权**;CSI300 train +23.11%→test −2.76%,train→test 衰减仍显著)。 - ⚠️ **EXPLORATORY,非收益声明**:value/低波在 2020-2024 A 股是已知强势 regime;三 cells 窗口重叠非独立样本;未调参、未做成本敏感性。下一步候选:value+lowvol 子集独立复检。 - ✅ 质量门：`pytest` **322 passed**（P0=97 / P1=78 / P2-1=22 / P2-2=22 / P2-3=14 / P2-4=8 / P3-1=10 / P3-2=18 / P3-3=16 / P3-4=15 / P3-5=22）；`ruff` clean；`validate-config`（旧 7 配置 + `phase3_real_factor_candidates.yaml`）+ `run-phase0`（demo）均 OK。 diff --git a/RUNBOOK.md b/RUNBOOK.md index 28a7b47..3ef0eb5 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -354,7 +354,7 @@ Adds a conservative, daily, PIT-safe candidate factor pack and re-runs the P3-4 robustness matrix to test whether the legacy trio's weak signal was just a too-narrow factor set. **Exploratory — not tuned, not a return claim.** No alpha / portfolio / execution / OOS / robustness model change. Documented by -`config/phase3_real_factor_candidates.yaml` (legacy trio + 7 candidates, +`config/phase3_real_factor_candidates.yaml` (legacy trio + 8 candidates, same matrix shape as P3-4 incl. the disclosed CSI300×2020-2022 skip). ```bash diff --git a/TEST_REPORT.md b/TEST_REPORT.md index 36801d9..d796939 100644 --- a/TEST_REPORT.md +++ b/TEST_REPORT.md @@ -22,7 +22,7 @@ Run from the repo root with the project python (env `quant_mf`): | Gate | Command | Result | |---|---|---| -| Unit + integration | `pytest -q` | **299 passed, 0 failed** | +| Unit + integration | `pytest -q` | **322 passed, 0 failed** | | Lint | `ruff check .` | **All checks passed** | | Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml` + `phase3_real_multifactor.yaml` + `phase3_real_ic_weighted.yaml` + `phase3_real_oos_stability.yaml` + `phase3_real_robustness_matrix.yaml` + `phase3_real_factor_candidates.yaml`) | exit `0`, prints `OK` | | End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` |