Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri

## 开发约定
- **交流中文**;代码/注释/commit message 用**英文**。
- **Git**:feature 分支 + PR。main 已有骨架;`data` 分支已含 P0+P1(PR #1 `data→main`,OPEN)。commit 用 conventional 格式,**无 attribution**(不加 Co-Authored-By)。
- **Git**:feature 分支 + PR。**PR #1(P0+P1)已 merge 到 `main`**;当前在 `p2-real-baseline` 分支推进 P2-1。commit 用 conventional 格式,**无 attribution**(不加 Co-Authored-By)。
- **不过度设计**:按路线图 MVP 先打通一条端到端链路,再加层(architecture.html §11,Phase 0→3)。
- **secrets** 一律走外部 `.config.json`;repo `.gitignore` 已排除数据产物(`*.parquet`等)、缓存、`tmp/`(仅留架构文档)。
- 文件小而专(<800 行),immutable 优先。
Expand All @@ -72,7 +72,8 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- 财务 `ann_date` 披露日 as-of(绝不按 end_date;500 天 lookback carry forward)
- 行业 + 市值中性化(按 date 截面 OLS 残差;欠定/无自由度截面 → NaN)
- 路径感知降级披露(demo/static vs tushare/index/ann_date,绝不把 demo 当真实验证)
- ✅ 质量门:`pytest` **168 passed**;`ruff` clean;`validate-config`(demo + `config/example_tushare.yaml`)+ `run-phase0`(demo)均 OK。
- ✅ 真数据实证(tushare,非 CI):复权除权日 raw−5.74%→qfq+0.99% / CSI300 全年 24 快照 328 名换手 / ann_date Q1 延后至 04-20 / 中性化 corr −0.617→0。详见 `BIAS_AUDIT.md`、`artifacts/reports/phase1_summary.md`。
- ✅ **Phase 2-1 真实数据可复现基准**(`p2-real-baseline` 分支,**PR #2 OPEN**):新 run mode `run-phase2-baseline` + `config/phase2_real_baseline.yaml`(上证50 `000016.SH`,2023-07~2024-06)。**复用 P0/P1 全套机器,不扩因子、不调参**。一次真实跑 ~11min(68 成分 / 25 loaded 快照 / in-window distinct 60 / 11 settled 调仓,候选 12 末日跳过),输出 `artifacts/reports/phase2_real_baseline.md`(gitignored):数据窗口 / PIT 成分摘要(loaded vs in-window) / ann_date 覆盖率(100%) / 可交易过滤命中(首命中互斥) / 每期持仓 / 换手成本 / IC(≈0.008) / 绩效(年化−17.6%,**亏损动量基准,非业绩声明**) / 全部 P2 降级。诊断只读、demo 源拒绝、token 不入报告。
- ✅ 质量门:`pytest` **182 passed**(P0=93 / P1=75 / P2-1=14);`ruff` clean;`validate-config`(demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余 P2(已显式披露):行业标签用**当前值**非 PIT、涨跌停未方向感知、`min_listing_days` no-op、日线 only、简版 IC/绩效未走 alphalens/quantstats、demo 路径非真数据。
- 路线图下一步:财务因子组合 / 历史 PIT 行业 / 更细交易约束(architecture.html §11)。
33 changes: 33 additions & 0 deletions RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,39 @@ Correctness details (locked by tests):
- **Neutralization returns NaN** on a saturated cross-section (names ≤ 1 + #industries,
i.e. no residual degrees of freedom) rather than fabricated ~0 residuals.

## Phase 2-1 — small-scale real-data reproducibility baseline

A REAL (tushare) end-to-end run of the EXISTING P0/P1 spine over a small universe
(SSE50, `000016.SH`) and a ~1-year window, designed to finish in ~10-30 min. It
adds NO new factor and does NO parameter search — it validates the real-data
plumbing and emits a richer diagnostic report. Documented by
`config/phase2_real_baseline.yaml`.

```bash
# validate (no network)
... -m qt.cli validate-config --config config/phase2_real_baseline.yaml
# run the real baseline (network + token; ~10-30 min for ~50-70 names)
... -m qt.cli run-phase2-baseline --config config/phase2_real_baseline.yaml
```

Output: `artifacts/reports/phase2_real_baseline.md` (git-ignored, regenerable). The
report contains: data window, PIT membership summary (snapshots / distinct names /
churn), ann_date as-of financial coverage (a DATA-QUALITY diagnostic on `roe`, not
the alpha factor), tradability filter-hit funnel, rebalance dates, per-period
holdings, per-period turnover/cost, IC / quantile returns, performance summary, and
all P2 downgrades.

Guards (correctness / honesty):

- **Demo source is refused.** `run-phase2-baseline` raises a readable error on
`data.source != 'tushare'` — a "baseline" on offline demo data carries no PIT /
ann_date / tradability meaning and must not masquerade as a real validation.
- **Holdings are reconstructed read-only** (universe → scores → `constructor.build`,
the same chain the driver runs); the reporting never sees forward returns at the
factor stage and never re-derives returns.
- **No secret leak.** The report echoes only non-sensitive config (window, universe,
factor); the token / secret file path is never written into it.

## Quality gate

```bash
Expand Down
96 changes: 96 additions & 0 deletions config/phase2_real_baseline.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# Phase 2-1 — small-scale REAL-data (tushare) reproducibility baseline.
#
# Purpose: run the EXISTING P0/P1 spine end-to-end on the real tushare path over a
# SMALL universe + SHORT window so it finishes in ~10-30 min and produces a rich
# diagnostic report (artifacts/reports/phase2_real_baseline.md). It is NOT a new
# strategy: same momentum_20 factor, same equal-weight alpha, same TopN portfolio.
# No new factor, no parameter search, no live trading.
#
# Universe: 上证50 / SSE50 (000016.SH) — 50 names, the small end of A-share indices,
# enough to exercise PIT membership + churn without a 300-name pull.
#
# Run: python -m qt.cli run-phase2-baseline --config config/phase2_real_baseline.yaml
# (needs the tushare token in the external .config.json; hits the network; heavy.)

project:
name: quantitative_trading_phase2_real_baseline
timezone: Asia/Shanghai

data:
source: tushare
freq: D
start: "2023-07-01"
end: "2024-06-30"
external_secret_file: "/home/shaofl/Projects/financial_projects/.config.json"
tushare_token_key: "tushare.token"
output_name: phase2_daily

universe:
type: index
index_code: "000016.SH"
symbols: []
min_listing_days: 60
filters:
missing_close: true
suspended: true
st: true
limit_up_down: true

factors:
- name: momentum_20
enabled: true
params:
window: 20
price_col: close

processing:
drop_missing: true
standardize:
enabled: true
method: zscore
winsorize:
enabled: false
method: mad
n: 3.0
neutralize:
enabled: true
industry_col: industry
size_col: market_cap

alpha:
model: equal_weight
params: {}

portfolio:
constructor: topn_equal_weight
top_n: 20
long_only: true
max_weight: null
turnover_cap: null

backtest:
initial_nav: 1.0
rebalance: monthly
event_order: close_to_next_period
cash_return: 0.0

cost:
fee_rate: 0.001
slippage_rate: 0.0
turnover_formula: l1

analytics:
forward_return_periods:
- 1
- 5
- 20
quantiles: 5
benchmark: null

output:
root_dir: artifacts
data_dir: artifacts/data
factor_dir: artifacts/factors
report_dir: artifacts/reports
log_dir: artifacts/logs
overwrite: true
39 changes: 34 additions & 5 deletions qt/cli.py
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
"""Command-line entry point for the Phase 0 framework.

Subcommands:
validate-config --config PATH Load + validate a YAML config.
run-phase0 --config PATH Run the full end-to-end pipeline + report.
fetch-data --config PATH Stage helper (runs the pipeline; see note).
compute-factors --config PATH Stage helper (runs the pipeline; see note).
run-backtest --config PATH Stage helper (runs the pipeline; see note).
validate-config --config PATH Load + validate a YAML config.
run-phase0 --config PATH Run the full end-to-end pipeline + report.
run-phase2-baseline --config PATH Run the small-scale REAL (tushare) baseline.
fetch-data --config PATH Stage helper (runs the pipeline; see note).
compute-factors --config PATH Stage helper (runs the pipeline; see note).
run-backtest --config PATH Stage helper (runs the pipeline; see note).

The CLI is intentionally thin: orchestration lives in :mod:`qt.pipeline`. Errors
are reported as readable one-line messages (CLI-003), never raw tracebacks. Run
Expand Down Expand Up @@ -69,6 +70,27 @@ def _cmd_run_phase0(args: argparse.Namespace) -> int:
return _run_pipeline_cmd(args.config, "run-phase0")


def _cmd_run_phase2_baseline(args: argparse.Namespace) -> int:
"""Run the small-scale REAL-data (tushare) reproducibility baseline + report."""
# Imported lazily so validate-config / demo runs never import the heavy
# real-data baseline module.
from qt.phase2_baseline import run_phase2_baseline

try:
result = run_phase2_baseline(args.config)
except (ConfigError, ValueError, FileNotFoundError) as exc:
print(f"ERROR: {exc}", file=sys.stderr)
return 1
print(
f"OK run-phase2-baseline: symbols={result.panel_symbols}, "
f"rebalances={len(result.rebalance_dates)}, ic_mean={result.ic_mean:.4f}, "
f"annual_return={result.performance.get('annual_return', float('nan')):.4f} "
f"({result.elapsed_seconds:.1f}s)\n"
f"report: {result.report_path}"
)
return 0


def _cmd_fetch_data(args: argparse.Namespace) -> int:
"""Stage helper: run the spine and report the data-fetch stage."""
return _run_pipeline_cmd(args.config, "fetch-data")
Expand Down Expand Up @@ -100,6 +122,13 @@ def build_parser() -> argparse.ArgumentParser:
p_run.add_argument("--config", required=True, help="Path to the YAML config.")
p_run.set_defaults(func=_cmd_run_phase0)

p_p2 = sub.add_parser(
"run-phase2-baseline",
help="Run the small-scale REAL-data (tushare) reproducibility baseline.",
)
p_p2.add_argument("--config", required=True, help="Path to the YAML config.")
p_p2.set_defaults(func=_cmd_run_phase2_baseline)

for name, func, help_text in (
("fetch-data", _cmd_fetch_data, "Run the spine, report data fetch."),
("compute-factors", _cmd_compute_factors, "Run the spine, report factor compute."),
Expand Down
Loading