diff --git a/CLAUDE.md b/CLAUDE.md index 7c00719..b22400a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -57,7 +57,7 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri ## 开发约定 - **交流中文**；代码/注释/commit message 用**英文**。 -- **Git**：feature 分支 + PR。main 已有骨架；`data` 分支已含 P0+P1（PR #1 `data→main`，OPEN）。commit 用 conventional 格式，**无 attribution**（不加 Co-Authored-By）。 +- **Git**：feature 分支 + PR。**PR #1（P0+P1）已 merge 到 `main`**；当前在 `p2-real-baseline` 分支推进 P2-1。commit 用 conventional 格式，**无 attribution**（不加 Co-Authored-By）。 - **不过度设计**：按路线图 MVP 先打通一条端到端链路，再加层（architecture.html §11，Phase 0→3）。 - **secrets** 一律走外部 `.config.json`；repo `.gitignore` 已排除数据产物(`*.parquet`等)、缓存、`tmp/`（仅留架构文档）。 - 文件小而专（<800 行），immutable 优先。 @@ -72,7 +72,8 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri - 财务 `ann_date` 披露日 as-of（绝不按 end_date；500 天 lookback carry forward） - 行业 + 市值中性化（按 date 截面 OLS 残差；欠定/无自由度截面 → NaN） - 路径感知降级披露（demo/static vs tushare/index/ann_date，绝不把 demo 当真实验证） -- ✅ 质量门：`pytest` **168 passed**；`ruff` clean；`validate-config`（demo + `config/example_tushare.yaml`）+ `run-phase0`（demo）均 OK。 - ✅ 真数据实证（tushare，非 CI）：复权除权日 raw−5.74%→qfq+0.99% / CSI300 全年 24 快照 328 名换手 / ann_date Q1 延后至 04-20 / 中性化 corr −0.617→0。详见 `BIAS_AUDIT.md`、`artifacts/reports/phase1_summary.md`。 +- ✅ **Phase 2-1 真实数据可复现基准**（`p2-real-baseline` 分支，**PR #2 OPEN**）：新 run mode `run-phase2-baseline` + `config/phase2_real_baseline.yaml`（上证50 `000016.SH`，2023-07~2024-06）。**复用 P0/P1 全套机器，不扩因子、不调参**。一次真实跑 ~11min（68 成分 / 25 loaded 快照 / in-window distinct 60 / 11 settled 调仓，候选 12 末日跳过），输出 `artifacts/reports/phase2_real_baseline.md`（gitignored）：数据窗口 / PIT 成分摘要(loaded vs in-window) / ann_date 覆盖率(100%) / 可交易过滤命中(首命中互斥) / 每期持仓 / 换手成本 / IC(≈0.008) / 绩效(年化−17.6%，**亏损动量基准，非业绩声明**) / 全部 P2 降级。诊断只读、demo 源拒绝、token 不入报告。 +- ✅ 质量门：`pytest` **182 passed**（P0=93 / P1=75 / P2-1=14）；`ruff` clean；`validate-config`（demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`）+ `run-phase0`（demo）均 OK。 - ⚠️ 剩余 P2（已显式披露）：行业标签用**当前值**非 PIT、涨跌停未方向感知、`min_listing_days` no-op、日线 only、简版 IC/绩效未走 alphalens/quantstats、demo 路径非真数据。 - 路线图下一步：财务因子组合 / 历史 PIT 行业 / 更细交易约束（architecture.html §11）。 diff --git a/RUNBOOK.md b/RUNBOOK.md index 84cf867..c860260 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -119,6 +119,39 @@ Correctness details (locked by tests): - **Neutralization returns NaN** on a saturated cross-section (names ≤ 1 + #industries, i.e. no residual degrees of freedom) rather than fabricated ~0 residuals. +## Phase 2-1 — small-scale real-data reproducibility baseline + +A REAL (tushare) end-to-end run of the EXISTING P0/P1 spine over a small universe +(SSE50, `000016.SH`) and a ~1-year window, designed to finish in ~10-30 min. It +adds NO new factor and does NO parameter search — it validates the real-data +plumbing and emits a richer diagnostic report. Documented by +`config/phase2_real_baseline.yaml`. + +```bash +# validate (no network) +... -m qt.cli validate-config --config config/phase2_real_baseline.yaml +# run the real baseline (network + token; ~10-30 min for ~50-70 names) +... -m qt.cli run-phase2-baseline --config config/phase2_real_baseline.yaml +``` + +Output: `artifacts/reports/phase2_real_baseline.md` (git-ignored, regenerable). The +report contains: data window, PIT membership summary (snapshots / distinct names / +churn), ann_date as-of financial coverage (a DATA-QUALITY diagnostic on `roe`, not +the alpha factor), tradability filter-hit funnel, rebalance dates, per-period +holdings, per-period turnover/cost, IC / quantile returns, performance summary, and +all P2 downgrades. + +Guards (correctness / honesty): + +- **Demo source is refused.** `run-phase2-baseline` raises a readable error on + `data.source != 'tushare'` — a "baseline" on offline demo data carries no PIT / + ann_date / tradability meaning and must not masquerade as a real validation. +- **Holdings are reconstructed read-only** (universe → scores → `constructor.build`, + the same chain the driver runs); the reporting never sees forward returns at the + factor stage and never re-derives returns. +- **No secret leak.** The report echoes only non-sensitive config (window, universe, + factor); the token / secret file path is never written into it. + ## Quality gate ```bash diff --git a/config/phase2_real_baseline.yaml b/config/phase2_real_baseline.yaml new file mode 100644 index 0000000..f7fb4e0 --- /dev/null +++ b/config/phase2_real_baseline.yaml @@ -0,0 +1,96 @@ +# Phase 2-1 — small-scale REAL-data (tushare) reproducibility baseline. +# +# Purpose: run the EXISTING P0/P1 spine end-to-end on the real tushare path over a +# SMALL universe + SHORT window so it finishes in ~10-30 min and produces a rich +# diagnostic report (artifacts/reports/phase2_real_baseline.md). It is NOT a new +# strategy: same momentum_20 factor, same equal-weight alpha, same TopN portfolio. +# No new factor, no parameter search, no live trading. +# +# Universe: 上证50 / SSE50 (000016.SH) — 50 names, the small end of A-share indices, +# enough to exercise PIT membership + churn without a 300-name pull. +# +# Run: python -m qt.cli run-phase2-baseline --config config/phase2_real_baseline.yaml +# (needs the tushare token in the external .config.json; hits the network; heavy.) + +project: + name: quantitative_trading_phase2_real_baseline + timezone: Asia/Shanghai + +data: + source: tushare + freq: D + start: "2023-07-01" + end: "2024-06-30" + external_secret_file: "/home/shaofl/Projects/financial_projects/.config.json" + tushare_token_key: "tushare.token" + output_name: phase2_daily + +universe: + type: index + index_code: "000016.SH" + symbols: [] + min_listing_days: 60 + filters: + missing_close: true + suspended: true + st: true + limit_up_down: true + +factors: + - name: momentum_20 + enabled: true + params: + window: 20 + price_col: close + +processing: + drop_missing: true + standardize: + enabled: true + method: zscore + winsorize: + enabled: false + method: mad + n: 3.0 + neutralize: + enabled: true + industry_col: industry + size_col: market_cap + +alpha: + model: equal_weight + params: {} + +portfolio: + constructor: topn_equal_weight + top_n: 20 + long_only: true + max_weight: null + turnover_cap: null + +backtest: + initial_nav: 1.0 + rebalance: monthly + event_order: close_to_next_period + cash_return: 0.0 + +cost: + fee_rate: 0.001 + slippage_rate: 0.0 + turnover_formula: l1 + +analytics: + forward_return_periods: + - 1 + - 5 + - 20 + quantiles: 5 + benchmark: null + +output: + root_dir: artifacts + data_dir: artifacts/data + factor_dir: artifacts/factors + report_dir: artifacts/reports + log_dir: artifacts/logs + overwrite: true diff --git a/qt/cli.py b/qt/cli.py index a8ddbc2..efc8016 100644 --- a/qt/cli.py +++ b/qt/cli.py @@ -1,11 +1,12 @@ """Command-line entry point for the Phase 0 framework. Subcommands: - validate-config --config PATH Load + validate a YAML config. - run-phase0 --config PATH Run the full end-to-end pipeline + report. - fetch-data --config PATH Stage helper (runs the pipeline; see note). - compute-factors --config PATH Stage helper (runs the pipeline; see note). - run-backtest --config PATH Stage helper (runs the pipeline; see note). + validate-config --config PATH Load + validate a YAML config. + run-phase0 --config PATH Run the full end-to-end pipeline + report. + run-phase2-baseline --config PATH Run the small-scale REAL (tushare) baseline. + fetch-data --config PATH Stage helper (runs the pipeline; see note). + compute-factors --config PATH Stage helper (runs the pipeline; see note). + run-backtest --config PATH Stage helper (runs the pipeline; see note). The CLI is intentionally thin: orchestration lives in :mod:`qt.pipeline`. Errors are reported as readable one-line messages (CLI-003), never raw tracebacks. Run @@ -69,6 +70,27 @@ def _cmd_run_phase0(args: argparse.Namespace) -> int: return _run_pipeline_cmd(args.config, "run-phase0") +def _cmd_run_phase2_baseline(args: argparse.Namespace) -> int: + """Run the small-scale REAL-data (tushare) reproducibility baseline + report.""" + # Imported lazily so validate-config / demo runs never import the heavy + # real-data baseline module. + from qt.phase2_baseline import run_phase2_baseline + + try: + result = run_phase2_baseline(args.config) + except (ConfigError, ValueError, FileNotFoundError) as exc: + print(f"ERROR: {exc}", file=sys.stderr) + return 1 + print( + f"OK run-phase2-baseline: symbols={result.panel_symbols}, " + f"rebalances={len(result.rebalance_dates)}, ic_mean={result.ic_mean:.4f}, " + f"annual_return={result.performance.get('annual_return', float('nan')):.4f} " + f"({result.elapsed_seconds:.1f}s)\n" + f"report: {result.report_path}" + ) + return 0 + + def _cmd_fetch_data(args: argparse.Namespace) -> int: """Stage helper: run the spine and report the data-fetch stage.""" return _run_pipeline_cmd(args.config, "fetch-data") @@ -100,6 +122,13 @@ def build_parser() -> argparse.ArgumentParser: p_run.add_argument("--config", required=True, help="Path to the YAML config.") p_run.set_defaults(func=_cmd_run_phase0) + p_p2 = sub.add_parser( + "run-phase2-baseline", + help="Run the small-scale REAL-data (tushare) reproducibility baseline.", + ) + p_p2.add_argument("--config", required=True, help="Path to the YAML config.") + p_p2.set_defaults(func=_cmd_run_phase2_baseline) + for name, func, help_text in ( ("fetch-data", _cmd_fetch_data, "Run the spine, report data fetch."), ("compute-factors", _cmd_compute_factors, "Run the spine, report factor compute."), diff --git a/qt/phase2_baseline.py b/qt/phase2_baseline.py new file mode 100644 index 0000000..ad5adf3 --- /dev/null +++ b/qt/phase2_baseline.py @@ -0,0 +1,454 @@ +"""Phase 2-1: a small-scale REAL-data (tushare) reproducibility baseline. + +This is NOT a new strategy and adds NO new factor or parameter search. It runs +the *existing* P0/P1 spine on the real tushare path end-to-end over a small +universe + short window (designed to finish in ~10-30 min) and emits a richer +diagnostic report (``artifacts/reports/phase2_real_baseline.md``) so the real +path can be eyeballed and reproduced: + + data window · PIT membership summary · ann_date as-of coverage · + tradability filter hits · rebalance dates · per-period holdings · + turnover / cost · IC / quantile returns · performance · P2 downgrades. + +It reuses :mod:`qt.pipeline`'s step helpers verbatim (same universe / panel / +factor / neutralization / backtest), so "baseline == the real pipeline". The +extra reporting (holdings, filter hits, coverage) is **read-only**: it never +changes the strategy and never sees forward returns at the factor stage. + +Guard: the baseline REQUIRES ``data.source='tushare'``. Running it on the demo +source is a category error (no PIT / ann_date / tradability meaning) and raises +a readable error rather than silently producing a meaningless "real" report. +""" + +from __future__ import annotations + +import time +from dataclasses import dataclass +from pathlib import Path + +import pandas as pd + +from alpha.equal_weight import EqualWeightAlpha +from analytics.performance import performance_summary +from factors.compute.financial import SUPPORTED_FIELDS as SUPPORTED_FINANCIAL_FIELDS +from factors.compute.financial import FinancialFactor +from portfolio.construct import TopNEqualWeight +from qt.config import RootConfig, load_config +from qt.pipeline import ( + _FrameScores, + _build_factor, + _build_scores, + _build_universe, + _collect_downgrades, + _compute_factor_panel, + _factor_analytics, + _load_panel, + _make_logger, + _maybe_enrich_covariates, + _maybe_enrich_financials, + _periods_per_year, + _process_factors, +) +from qt.reports import write_phase2_baseline_summary +from runtime.backtest.driver import BacktestDriver +from runtime.backtest.sim_execution import SimExecution +from universe.index_universe import PITIndexUniverse + +_LOGGER_NAME = "qt.run_phase2_baseline" + +# Tradability reasons reported (mirrors universe.filters.apply_tradable_filters). +_FILTER_REASONS = ("missing_close", "suspended", "is_st", "at_up_limit", "at_down_limit") + + +@dataclass(frozen=True) +class Phase2Result: + """Immutable summary of one phase2 real-baseline run (what the report consumes).""" + + config: RootConfig + elapsed_seconds: float + # data window + first_trade_date: pd.Timestamp | None + last_trade_date: pd.Timestamp | None + trade_days: int + panel_rows: int + panel_symbols: int + # universe / PIT membership + universe_summary: dict + # ann_date financial coverage (diagnostic) + financial_field: str + financial_coverage_overall: float + financial_coverage_by_rebalance: pd.DataFrame + # tradability filter hits + tradability_hits: pd.DataFrame + # rebalance + holdings (rebalance_dates are the SETTLED dates == nav_table.index) + rebalance_dates: tuple[pd.Timestamp, ...] + candidate_rebalance_dates: tuple[pd.Timestamp, ...] + skipped_terminal_dates: tuple[pd.Timestamp, ...] + holdings: pd.DataFrame + # turnover / cost / performance + factor_name: str + nav_table: pd.DataFrame + avg_turnover: float + cost_drag: float + ic_mean: float + ic_ir: float + quantile_returns: pd.DataFrame + performance: dict + # disclosure + downgrades: tuple[str, ...] + # paths + report_path: Path + log_path: Path + + +# --------------------------------------------------------------------------- # +# Pure collectors (network-free; unit-tested with synthetic inputs). +# --------------------------------------------------------------------------- # +def summarize_universe(universe, universe_type: str, start, end) -> dict: + """Summarize the (PIT or static) universe for the report. + + For a :class:`PITIndexUniverse` the feed deliberately loads snapshots from + BEFORE ``start`` (a ~370-day pre-start lookback) so the as-of membership at the + window start resolves correctly. This summary therefore separates two things: + + * LOADED snapshots — everything fetched, incl. the pre-start lookback (so the + reader sees exactly what was pulled); + * IN-WINDOW membership — the snapshots dated within ``[start, end]`` plus the + as-of anchor active at ``start`` (the lookback snapshot the early dates + resolve to). ``distinct_names_in_window`` / size / churn are reported over + this in-window view, so "distinct names over window" means what the + backtest actually saw, not names that only existed before the window. + + Static universes carry no point-in-time membership, recorded explicitly. + """ + if isinstance(universe, PITIndexUniverse): + snaps = universe.membership_snapshots() + loaded = sorted(snaps) + start_ts = pd.Timestamp(start).normalize() + end_ts = pd.Timestamp(end).normalize() + in_window = [d for d in loaded if start_ts <= d <= end_ts] + prior = [d for d in loaded if d <= start_ts] + anchor = max(prior) if prior else None + # names seen during the window = anchor membership (held into the window) + # ∪ every in-window snapshot's membership. + active = ([anchor] if anchor is not None else []) + in_window + sizes = [len(snaps[d]) for d in active] + distinct = sorted({s for d in active for s in snaps[d]}) + churn_in: list[int] = [] + churn_out: list[int] = [] + # churn is reported over in-window transitions only (membership changes + # that actually occur inside the backtest window). + for prev, cur in zip(in_window, in_window[1:]): + prev_set, cur_set = set(snaps[prev]), set(snaps[cur]) + churn_in.append(len(cur_set - prev_set)) + churn_out.append(len(prev_set - cur_set)) + return { + "pit": True, + "type": "index", + "n_loaded_snapshots": len(loaded), + "loaded_first": loaded[0] if loaded else None, + "loaded_last": loaded[-1] if loaded else None, + "n_window_snapshots": len(in_window), + "anchor_snapshot": anchor, + "distinct_names_in_window": len(distinct), + "min_size": min(sizes) if sizes else 0, + "max_size": max(sizes) if sizes else 0, + "avg_churn_in": (sum(churn_in) / len(churn_in)) if churn_in else 0.0, + "avg_churn_out": (sum(churn_out) / len(churn_out)) if churn_out else 0.0, + } + return { + "pit": False, + "type": universe_type, + "distinct_names_in_window": len(getattr(universe, "_symbols", []) or []), + } + + +def tradability_hit_stats( + universe, + panel: pd.DataFrame, + rebalance_dates: list[pd.Timestamp], + filters: dict, +) -> pd.DataFrame: + """Count, summed over rebalance dates, how many members each filter knocks out. + + For every rebalance date we take the universe members, look at that day's + cross-section, and tally the reason a name is dropped. The reasons are + EXCLUSIVE and use the SAME first-match-wins order as + :func:`universe.filters.apply_tradable_filters` (missing close always; then + suspended; then ST; then at-limit — each gated by its toggle and the flag + column being present). A name flagged by several filters is counted once, in + the first matching bucket, so ``sum(hits) == candidates - tradable`` exactly + and the funnel is auditable. Returns a frame indexed by reason with a ``hits`` + column plus ``candidates`` / ``tradable`` totals in ``.attrs``. + """ + counts = {r: 0 for r in _FILTER_REASONS} + n_candidates = 0 + n_tradable = 0 + for date in rebalance_dates: + target = pd.Timestamp(date).normalize() + members = list(universe.members(target)) + try: + cross = panel.xs(target, level="date") + except KeyError: + cross = panel.iloc[0:0] + for sym in members: + n_candidates += 1 + if sym not in cross.index: + counts["missing_close"] += 1 + continue + row = cross.loc[sym] + if pd.isna(row["close"]): + counts["missing_close"] += 1 + continue + # First-match-wins, mirroring apply_tradable_filters (exclusive buckets). + if filters.get("suspended") and bool(row.get("suspended", False)): + counts["suspended"] += 1 + continue + if filters.get("st") and bool(row.get("is_st", False)): + counts["is_st"] += 1 + continue + if filters.get("limit_up_down") and bool(row.get("at_up_limit", False)): + counts["at_up_limit"] += 1 + continue + if filters.get("limit_up_down") and bool(row.get("at_down_limit", False)): + counts["at_down_limit"] += 1 + continue + n_tradable += 1 + frame = pd.DataFrame( + {"hits": [counts[r] for r in _FILTER_REASONS]}, + index=list(_FILTER_REASONS), + ) + frame.index.name = "reason" + frame.attrs["candidates"] = n_candidates + frame.attrs["tradable"] = n_tradable + return frame + + +def reconstruct_holdings( + scores: _FrameScores, + universe, + panel: pd.DataFrame, + constructor: TopNEqualWeight, + rebalance_dates: list[pd.Timestamp], +) -> pd.DataFrame: + """Rebuild per-period target holdings READ-ONLY (mirrors the driver's step). + + For each rebalance date: tradable members -> scores -> constructor.build, the + exact same chain :meth:`BacktestDriver._step` runs. We only read the resulting + target weights (the holdings); we never settle or see forward returns, so this + is a faithful, side-effect-free view of what the backtest held each period. + Returns a long-form frame with columns ``[date, symbol, weight, rank]``. + """ + rows: list[dict] = [] + for date in rebalance_dates: + target_date = pd.Timestamp(date).normalize() + tradable = list(universe.tradable(target_date, panel)) + if not tradable: + continue + cross_scores = scores.get(target_date, tradable) + weights = constructor.build(cross_scores) + for rank, (sym, w) in enumerate(weights.items(), start=1): + rows.append( + {"date": target_date, "symbol": str(sym), "weight": float(w), "rank": rank} + ) + return pd.DataFrame(rows, columns=["date", "symbol", "weight", "rank"]) + + +def financial_coverage_at_dates( + aligned_col: pd.Series, + universe, + panel: pd.DataFrame, + rebalance_dates: list[pd.Timestamp], +) -> pd.DataFrame: + """Per-rebalance-date as-of financial coverage among that date's members. + + ``aligned_col`` is the ann_date as-of aligned financial series (NaN where no + report had been disclosed by that date). The denominator (``n_members``) is the + universe members PRESENT IN THE PANEL that date (taken from ``panel`` so the + contract is explicit, not implied by the aligned series' index); of those, we + count how many carry a non-NaN figure. Returns ``[date, n_members, n_covered, + coverage]`` — a data-quality lens on how well the financial path is populated, + NOT a strategy signal. + """ + rows: list[dict] = [] + for date in rebalance_dates: + target = pd.Timestamp(date).normalize() + members = list(universe.members(target)) + try: + cross_syms = set(panel.xs(target, level="date").index) + except KeyError: + cross_syms = set() + n_members = 0 + n_covered = 0 + for sym in members: + if str(sym) not in cross_syms: + continue # not present in the panel that day -> not a denominator member + n_members += 1 + val = aligned_col.get((target, str(sym)), float("nan")) + if pd.notna(val): + n_covered += 1 + coverage = (n_covered / n_members) if n_members else float("nan") + rows.append( + { + "date": target, + "n_members": n_members, + "n_covered": n_covered, + "coverage": coverage, + } + ) + return pd.DataFrame(rows, columns=["date", "n_members", "n_covered", "coverage"]) + + +def _phase2_downgrades(cfg: RootConfig, financial_field: str) -> tuple[str, ...]: + """Base downgrades + phase2-baseline-specific disclosures.""" + extra = ( + f"Financial ann_date coverage is a DATA-QUALITY DIAGNOSTIC on " + f"'{financial_field}' (how well disclosed reports populate the universe " + "as-of); it is NOT the alpha factor in this baseline (the factor is the " + "configured price factor). No financial signal is traded here.", + "Per-period holdings are RECONSTRUCTED read-only (universe -> scores -> " + "constructor.build), the same chain the backtest driver runs; they are a " + "faithful view, not an independent recomputation of returns.", + "This is a SMALL-SCALE baseline (a small index over a short window) for " + "reproducibility/plumbing validation, NOT a performance claim; numbers are " + "not optimized and must not be read as a strategy result.", + ) + return _collect_downgrades(cfg) + extra + + +# --------------------------------------------------------------------------- # +# Orchestration +# --------------------------------------------------------------------------- # +def run_phase2_baseline(config_path: str) -> Phase2Result: + """Run the small-scale real-data baseline and write the phase2 report. + + Raises ``ValueError`` (readable) if pointed at the demo source — the baseline + is meaningless without real PIT / ann_date / tradability data. All writes land + under the configured ``output`` dirs (the report is git-ignored under + ``artifacts/``; the tushare token is never read into the report). + """ + cfg = load_config(config_path) + if cfg.data.source != "tushare": + raise ValueError( + "run-phase2-baseline is a REAL-data experiment and requires " + "data.source='tushare'. The demo/offline path carries no PIT, " + "ann_date, or tradability meaning, so a 'baseline' there would be a " + f"category error. Got data.source={cfg.data.source!r}." + ) + + t0 = time.perf_counter() + log_path = Path(cfg.output.log_dir) / "run_phase2_baseline.log" + logger = _make_logger(log_path, name=_LOGGER_NAME) + logger.info("phase2 baseline start: project=%s", cfg.project.name) + + # --- reuse the exact P0/P1 spine ------------------------------------- # + universe, symbols = _build_universe(cfg, logger) + panel = _load_panel(cfg, symbols, logger) + factor = _build_factor(cfg) + panel = _maybe_enrich_financials(cfg, panel, symbols, factor, logger) + panel = _maybe_enrich_covariates(cfg, panel, symbols, logger) + factor_panel = _compute_factor_panel(cfg, panel, factor, logger) + processed = _process_factors(cfg, factor_panel, panel) + score_panel = _build_scores(processed, EqualWeightAlpha()) + scores = _FrameScores(score_panel) + + constructor = TopNEqualWeight(cfg.portfolio.top_n, long_only=cfg.portfolio.long_only) + execution = SimExecution(fee_rate=cfg.cost.fee_rate, cash_return=cfg.backtest.cash_return) + driver = BacktestDriver( + universe=universe, + scores=scores, + constructor=constructor, + execution=execution, + prices=panel, + rebalance=cfg.backtest.rebalance, + fee_rate=cfg.cost.fee_rate, + initial_nav=cfg.backtest.initial_nav, + cash_return=cfg.backtest.cash_return, + ) + candidate_dates = driver.rebalance_dates() + nav_table = driver.run() + # The diagnostics (coverage / filter hits / holdings) MUST key off the dates + # that were actually held + settled — i.e. nav_table.index — NOT the candidate + # rebalance dates. The driver skips a terminal rebalance with no forward + # holding period (BT-003), so the last candidate date has no NAV/turnover row; + # reporting holdings for it would be a phantom period. + settled_dates = list(nav_table.index) + settled_set = set(settled_dates) + skipped_dates = [d for d in candidate_dates if d not in settled_set] + logger.info("backtest: %d candidate rebalance dates, %d settled rows (skipped %d)", + len(candidate_dates), len(nav_table), len(skipped_dates)) + + ic_mean, ic_ir, q_returns = _factor_analytics(cfg, panel, factor_panel, factor.name) + perf = ( + performance_summary( + nav_table["nav"], periods_per_year=_periods_per_year(cfg.backtest.rebalance) + ) + if not nav_table.empty + else {k: float("nan") for k in ("annual_return", "max_drawdown", "volatility", "sharpe")} + ) + + # --- diagnostics (read-only) ----------------------------------------- # + financial_field = ( + factor.name if isinstance(factor, FinancialFactor) else SUPPORTED_FINANCIAL_FIELDS[0] + ) + if isinstance(factor, FinancialFactor): + diag_col = panel[financial_field] + else: + logger.info( + "diagnostics: fetching '%s' for ann_date coverage report ONLY " + "(NOT the active factor; the strategy factor is '%s')", + financial_field, factor.name, + ) + diag_panel = _maybe_enrich_financials( + cfg, panel, symbols, FinancialFactor(financial_field), logger + ) + diag_col = diag_panel[financial_field] + coverage_by_rebalance = financial_coverage_at_dates( + diag_col, universe, panel, settled_dates + ) + coverage_overall = float(diag_col.notna().mean()) if len(diag_col) else float("nan") + + hits = tradability_hit_stats( + universe, panel, settled_dates, cfg.universe.filters.model_dump() + ) + holdings = reconstruct_holdings(scores, universe, panel, constructor, settled_dates) + + dates = panel.index.get_level_values("date") + result = Phase2Result( + config=cfg, + elapsed_seconds=time.perf_counter() - t0, + first_trade_date=dates.min() if len(dates) else None, + last_trade_date=dates.max() if len(dates) else None, + trade_days=int(pd.Series(dates).nunique()), + panel_rows=len(panel), + panel_symbols=int(panel.index.get_level_values("symbol").nunique()), + universe_summary=summarize_universe( + universe, cfg.universe.type, cfg.data.start, cfg.data.end + ), + financial_field=financial_field, + financial_coverage_overall=coverage_overall, + financial_coverage_by_rebalance=coverage_by_rebalance, + tradability_hits=hits, + rebalance_dates=tuple(settled_dates), + candidate_rebalance_dates=tuple(candidate_dates), + skipped_terminal_dates=tuple(skipped_dates), + holdings=holdings, + factor_name=factor.name, + nav_table=nav_table, + avg_turnover=float(nav_table["turnover"].mean()) if not nav_table.empty else 0.0, + cost_drag=float(nav_table["cost"].sum()) if not nav_table.empty else 0.0, + ic_mean=ic_mean, + ic_ir=ic_ir, + quantile_returns=q_returns, + performance=perf, + downgrades=_phase2_downgrades(cfg, financial_field), + report_path=Path(cfg.output.report_dir) / "phase2_real_baseline.md", + log_path=log_path, + ) + write_phase2_baseline_summary(result) + logger.info( + "phase2 baseline done: ic_mean=%.4f annual_return=%.4f report=%s (%.1fs)", + ic_mean, perf.get("annual_return", float("nan")), result.report_path, + result.elapsed_seconds, + ) + return result diff --git a/qt/pipeline.py b/qt/pipeline.py index ed73efb..63334e8 100644 --- a/qt/pipeline.py +++ b/qt/pipeline.py @@ -127,10 +127,10 @@ def get(self, date: pd.Timestamp, symbols: list[str]) -> pd.Series: return cross.reindex(list(symbols)) -def _make_logger(log_path: Path) -> logging.Logger: +def _make_logger(log_path: Path, name: str = _LOGGER_NAME) -> logging.Logger: """A run-scoped logger that writes to ``log_path`` (and never logs secrets).""" log_path.parent.mkdir(parents=True, exist_ok=True) - logger = logging.getLogger(_LOGGER_NAME) + logger = logging.getLogger(name) logger.setLevel(logging.INFO) # Re-entrant: drop handlers from a previous run so we don't double-write. for handler in list(logger.handlers): diff --git a/qt/reports.py b/qt/reports.py index cc15671..41ef41a 100644 --- a/qt/reports.py +++ b/qt/reports.py @@ -21,6 +21,7 @@ import pandas as pd if TYPE_CHECKING: # avoid a circular import at runtime (pipeline imports reports) + from qt.phase2_baseline import Phase2Result from qt.pipeline import Phase0Result @@ -118,6 +119,234 @@ def write_phase0_summary(result: "Phase0Result") -> Path: return target +# --------------------------------------------------------------------------- # +# Phase 2-1 real-data baseline report (qt.phase2_baseline). +# --------------------------------------------------------------------------- # +_PHASE2_REQUIRED_SECTIONS = ( + "## Data window", + "## Universe / PIT membership", + "## Financial ann_date coverage", + "## Tradability filter hits", + "## Rebalance dates", + "## Holdings per period", + "## Turnover & cost", + "## Factor IC", + "## Quantile returns", + "## Portfolio performance", + "## DOWNGRADES", +) + + +def phase2_baseline_required_sections() -> tuple[str, ...]: + """Section headers the phase2 baseline report MUST contain (single source).""" + return _PHASE2_REQUIRED_SECTIONS + + +def _date_str(value: object) -> str: + """Format a timestamp-ish value as YYYY-MM-DD (or 'n/a').""" + if value is None or (isinstance(value, float) and not math.isfinite(value)): + return "n/a" + try: + return pd.Timestamp(value).strftime("%Y-%m-%d") + except (ValueError, TypeError): + return str(value) + + +def _universe_block(summ: dict) -> str: + """Render the universe / PIT membership summary (loaded vs in-window).""" + if summ.get("pit"): + return ( + f"- type: **index** (point-in-time, survivorship-safe)\n" + f"- loaded snapshots: **{summ.get('n_loaded_snapshots', 0)}** " + f"({_date_str(summ.get('loaded_first'))} → {_date_str(summ.get('loaded_last'))}; " + f"includes the ~370-day pre-start lookback for as-of safety)\n" + f"- in-window membership updates: **{summ.get('n_window_snapshots', 0)}** " + f"(as-of anchor at start: {_date_str(summ.get('anchor_snapshot'))})\n" + f"- distinct names in-window: **{summ.get('distinct_names_in_window', 0)}**\n" + f"- per-snapshot size: {summ.get('min_size', 0)}–{summ.get('max_size', 0)}\n" + f"- avg churn per in-window update: **{summ.get('avg_churn_in', 0.0):.1f} in / " + f"{summ.get('avg_churn_out', 0.0):.1f} out**\n" + ) + return ( + f"- type: **{summ.get('type', '?')}** (NOT point-in-time — survivorship/" + f"look-ahead membership downgrade)\n" + f"- symbols: **{summ.get('distinct_names_in_window', 0)}**\n" + ) + + +def _coverage_block(overall: float, by_reb: pd.DataFrame) -> str: + """Render the ann_date as-of financial coverage diagnostic.""" + head = f"- overall as-of coverage (non-NaN rows): **{_fmt(overall, pct=True)}**\n\n" + if by_reb is None or by_reb.empty: + return head + "_(no rebalance cross-sections)_\n" + table = "| Rebalance date | members | covered | coverage |\n|---|---|---|---|\n" + for _, r in by_reb.iterrows(): + table += ( + f"| {_date_str(r['date'])} | {int(r['n_members'])} | " + f"{int(r['n_covered'])} | {_fmt(float(r['coverage']), pct=True)} |\n" + ) + return head + table + + +def _tradability_block(hits: pd.DataFrame) -> str: + """Render the tradability filter-hit funnel.""" + candidates = hits.attrs.get("candidates", 0) + tradable = hits.attrs.get("tradable", 0) + head = ( + f"- member-days evaluated (over rebalance dates): **{candidates}**\n" + f"- tradable after filters: **{tradable}**\n\n" + ) + table = "| Filter reason | hits (member-days dropped) |\n|---|---|\n" + for reason, row in hits.iterrows(): + table += f"| {reason} | {int(row['hits'])} |\n" + return head + table + + +def _holdings_block(holdings: pd.DataFrame) -> str: + """Render per-period holdings (complete; one line per rebalance date).""" + if holdings is None or holdings.empty: + return "_(no holdings — universe was empty every rebalance)_\n" + lines: list[str] = [] + for date, block in holdings.groupby("date", sort=True): + syms = list(block.sort_values("rank")["symbol"]) + k = len(syms) + w = block["weight"].iloc[0] if k else float("nan") + lines.append(f"- **{_date_str(date)}** ({k} names, w≈{_fmt(float(w))}): {', '.join(syms)}\n") + return "".join(lines) + + +def _per_period_table(nav: pd.DataFrame) -> str: + """Render per-rebalance turnover / cost / net return.""" + if nav is None or nav.empty: + return "_(no settled periods)_\n" + table = "| Rebalance date | turnover | cost | net return | nav |\n|---|---|---|---|---|\n" + for date, r in nav.iterrows(): + table += ( + f"| {_date_str(date)} | {_fmt(float(r['turnover']))} | " + f"{_fmt(float(r['cost']), pct=True)} | {_fmt(float(r['net_return']), pct=True)} | " + f"{_fmt(float(r['nav']))} |\n" + ) + return table + + +def render_phase2_baseline(result: "Phase2Result") -> str: + """Build the phase2 real-baseline markdown (pure; no I/O, no secrets). + + The tushare token / secret file is never echoed; only non-sensitive config + (window, universe type, factor) and computed diagnostics are written. + """ + cfg = result.config + perf = result.performance + lines: list[str] = [] + lines.append("# Phase 2-1 — Real-data Reproducibility Baseline\n") + lines.append( + f"Project: **{cfg.project.name}** · source: **{cfg.data.source}** · " + f"ran in **{result.elapsed_seconds:.1f}s**\n" + ) + lines.append( + "\n> Small-scale REAL (tushare) baseline that runs the existing P0/P1 spine " + "end-to-end. No new factor, no parameter search, not a performance claim — " + "it validates the real-data plumbing and is fully reproducible from the " + "config below.\n" + ) + + lines.append("\n## Config echo\n") + lines.append( + f"- universe: type=`{cfg.universe.type}`, index_code=`{cfg.universe.index_code}`, " + f"top_n=`{cfg.portfolio.top_n}`\n" + f"- factor: `{result.factor_name}` · neutralize=`{cfg.processing.neutralize.enabled}`\n" + f"- filters: `{cfg.universe.filters.model_dump()}`\n" + f"- backtest: rebalance=`{cfg.backtest.rebalance}`, fee_rate=`{cfg.cost.fee_rate}`\n" + ) + + lines.append("\n## Data window\n") + lines.append( + f"- configured: `[{cfg.data.start}, {cfg.data.end}]`\n" + f"- panel calendar: {_date_str(result.first_trade_date)} → " + f"{_date_str(result.last_trade_date)} (**{result.trade_days}** trading days)\n" + f"- panel rows: **{result.panel_rows}**, symbols: **{result.panel_symbols}**\n" + ) + + lines.append("\n## Universe / PIT membership\n") + lines.append(_universe_block(result.universe_summary)) + + lines.append("\n## Financial ann_date coverage\n") + lines.append( + f"_Diagnostic on `{result.financial_field}` (ann_date as-of); NOT the alpha " + f"factor._\n\n" + ) + lines.append( + _coverage_block(result.financial_coverage_overall, result.financial_coverage_by_rebalance) + ) + + lines.append("\n## Tradability filter hits\n") + lines.append(_tradability_block(result.tradability_hits)) + + lines.append("\n## Rebalance dates\n") + reb = ", ".join(_date_str(d) for d in result.rebalance_dates) or "_(none)_" + lines.append( + f"- **{len(result.rebalance_dates)}** settled rebalance dates (held + " + f"settled; these drive holdings / filter hits / coverage below): {reb}\n" + ) + lines.append( + f"- candidate rebalance dates (last trading day of each month): " + f"**{len(result.candidate_rebalance_dates)}**\n" + ) + if result.skipped_terminal_dates: + skipped = ", ".join(_date_str(d) for d in result.skipped_terminal_dates) + lines.append( + f"- skipped (terminal date(s) with no forward holding period, BT-003): " + f"{skipped}\n" + ) + + lines.append("\n## Holdings per period\n") + lines.append( + "_Holdings are listed for the settled rebalance dates only, so they match " + "the NAV/turnover table 1:1 (no phantom terminal period)._\n\n" + ) + lines.append(_holdings_block(result.holdings)) + + lines.append("\n## Turnover & cost\n") + lines.append( + f"- average turnover (per rebalance): **{_fmt(result.avg_turnover)}**\n" + f"- total cost drag: **{_fmt(result.cost_drag, pct=True)}**\n\n" + ) + lines.append(_per_period_table(result.nav_table)) + + lines.append("\n## Factor IC\n") + lines.append( + f"- IC mean: **{_fmt(result.ic_mean)}**\n" + f"- IC_IR (mean/std): **{_fmt(result.ic_ir)}**\n" + ) + + lines.append("\n## Quantile returns\n") + lines.append(_quantile_table(result.quantile_returns) + "\n") + + lines.append("\n## Portfolio performance\n") + lines.append( + f"- annual return: **{_fmt(perf.get('annual_return', float('nan')), pct=True)}**\n" + f"- max drawdown: **{_fmt(perf.get('max_drawdown', float('nan')), pct=True)}**\n" + f"- volatility: **{_fmt(perf.get('volatility', float('nan')), pct=True)}**\n" + f"- sharpe: **{_fmt(perf.get('sharpe', float('nan')))}**\n" + ) + + lines.append("\n## DOWNGRADES (INV-007 — must be disclosed)\n") + for item in result.downgrades: + lines.append(f"- {item}\n") + + lines.append("\n## Artifacts\n") + lines.append(f"- report: `{result.report_path}`\n- log: `{result.log_path}`\n") + return "".join(lines) + + +def write_phase2_baseline_summary(result: "Phase2Result") -> Path: + """Render and write the phase2 baseline report; return the path (SEC-003).""" + target = result.report_path + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(render_phase2_baseline(result), encoding="utf-8") + return target + + # --------------------------------------------------------------------------- # # Repo-root delivery docs (framework spec §10). These describe the run; they # carry no secrets and are regenerable. diff --git a/tests/test_phase2_baseline.py b/tests/test_phase2_baseline.py new file mode 100644 index 0000000..74682bb --- /dev/null +++ b/tests/test_phase2_baseline.py @@ -0,0 +1,345 @@ +"""Phase 2-1 real-data baseline: collectors, demo/real guard, report fields. + +Every test here is NETWORK-FREE (TEST-002): the pure collectors run on synthetic +panels/universes, the demo-guard is checked WITHOUT a tushare call, and the report +renderer runs on a hand-built ``Phase2Result``. No test reads the tushare token. +""" + +from __future__ import annotations + +from pathlib import Path + +import numpy as np +import pandas as pd +import pytest + +from qt.config import load_config +from qt.phase2_baseline import ( + Phase2Result, + financial_coverage_at_dates, + reconstruct_holdings, + summarize_universe, + tradability_hit_stats, +) +from qt.pipeline import _FrameScores +from qt.reports import ( + phase2_baseline_required_sections, + render_phase2_baseline, +) +from portfolio.construct import TopNEqualWeight +from runtime.backtest.driver import BacktestDriver +from runtime.backtest.sim_execution import SimExecution +from universe.index_universe import PITIndexUniverse +from universe.static import StaticUniverse + +_CONFIG = str(Path(__file__).resolve().parents[1] / "config" / "phase2_real_baseline.yaml") + + +# --------------------------------------------------------------------------- # +# Config + demo/real guard (no path confusion) +# --------------------------------------------------------------------------- # +def test_phase2_config_validates_as_real_path(): + cfg = load_config(_CONFIG) + assert cfg.data.source == "tushare" + assert cfg.universe.type == "index" + assert cfg.universe.index_code == "000016.SH" + # the baseline exercises every P1 red-line + assert cfg.universe.filters.suspended and cfg.universe.filters.st + assert cfg.universe.filters.limit_up_down + assert cfg.processing.neutralize.enabled + + +def test_run_phase2_baseline_rejects_demo_source(example_config_path): + # example_config_path is the DEMO config; the baseline must refuse it BEFORE + # any network call (demo carries no PIT/ann_date/tradability meaning). + from qt.phase2_baseline import run_phase2_baseline + + with pytest.raises(ValueError, match="REAL-data|tushare"): + run_phase2_baseline(example_config_path) + + +# --------------------------------------------------------------------------- # +# summarize_universe +# --------------------------------------------------------------------------- # +def _pit_universe() -> PITIndexUniverse: + # two snapshots, one name churns out and one churns in between them. + rows = [ + {"date": "2024-01-31", "symbol": "000001.SZ"}, + {"date": "2024-01-31", "symbol": "000002.SZ"}, + {"date": "2024-01-31", "symbol": "000003.SZ"}, + {"date": "2024-02-29", "symbol": "000001.SZ"}, + {"date": "2024-02-29", "symbol": "000002.SZ"}, + {"date": "2024-02-29", "symbol": "000004.SZ"}, # 003 out, 004 in + ] + return PITIndexUniverse(pd.DataFrame(rows), filters={}) + + +def test_summarize_universe_pit_counts_churn(): + summ = summarize_universe(_pit_universe(), "index", "2024-01-01", "2024-03-01") + assert summ["pit"] is True + assert summ["n_loaded_snapshots"] == 2 + assert summ["n_window_snapshots"] == 2 + assert summ["distinct_names_in_window"] == 4 # 001,002,003,004 + assert summ["min_size"] == 3 and summ["max_size"] == 3 + assert summ["avg_churn_in"] == 1.0 and summ["avg_churn_out"] == 1.0 + + +def test_summarize_universe_excludes_prestart_lookback(): + # 2022-06-30 carries a name 'Y' that is superseded before the window; the + # as-of anchor at start (2023-07-01) is 2022-12-31. 'Y' must NOT count toward + # the in-window distinct names, and the pre-start snapshots must NOT count as + # in-window snapshots (the MEDIUM finding). + rows = [ + {"date": "2022-06-30", "symbol": "A"}, + {"date": "2022-06-30", "symbol": "B"}, + {"date": "2022-06-30", "symbol": "Y"}, # only here -> superseded pre-window + {"date": "2022-12-31", "symbol": "A"}, + {"date": "2022-12-31", "symbol": "B"}, + {"date": "2022-12-31", "symbol": "X"}, # anchor membership at start + {"date": "2024-01-31", "symbol": "A"}, + {"date": "2024-01-31", "symbol": "B"}, + {"date": "2024-01-31", "symbol": "C"}, + {"date": "2024-02-29", "symbol": "A"}, + {"date": "2024-02-29", "symbol": "B"}, + {"date": "2024-02-29", "symbol": "D"}, + ] + uni = PITIndexUniverse(pd.DataFrame(rows), filters={}) + summ = summarize_universe(uni, "index", "2023-07-01", "2024-03-01") + assert summ["n_loaded_snapshots"] == 4 + assert pd.Timestamp(summ["loaded_first"]) == pd.Timestamp("2022-06-30") + assert summ["n_window_snapshots"] == 2 # only the two 2024 snapshots + assert pd.Timestamp(summ["anchor_snapshot"]) == pd.Timestamp("2022-12-31") + # A,B,X (anchor),C,D — but NOT Y (superseded before the window) + assert summ["distinct_names_in_window"] == 5 + + +def test_summarize_universe_static_marks_non_pit(): + uni = StaticUniverse(["000001.SZ", "000002.SZ"], {}) + summ = summarize_universe(uni, "static", "2024-01-01", "2024-12-31") + assert summ["pit"] is False + assert summ["distinct_names_in_window"] == 2 + + +# --------------------------------------------------------------------------- # +# tradability_hit_stats +# --------------------------------------------------------------------------- # +def _flag_panel() -> pd.DataFrame: + # one date, 4 symbols: one normal, one suspended, one ST, one at up-limit. + date = pd.Timestamp("2024-01-31") + syms = ["000001.SZ", "000002.SZ", "000003.SZ", "000004.SZ"] + idx = pd.MultiIndex.from_product([[date], syms], names=["date", "symbol"]) + return pd.DataFrame( + { + "close": [10.0, 10.0, 10.0, 10.0], + "suspended": [False, True, False, False], + "is_st": [False, False, True, False], + "at_up_limit": [False, False, False, True], + "at_down_limit": [False, False, False, False], + }, + index=idx, + ) + + +def test_tradability_hit_stats_counts_each_reason(): + panel = _flag_panel() + date = pd.Timestamp("2024-01-31") + uni = StaticUniverse(["000001.SZ", "000002.SZ", "000003.SZ", "000004.SZ"], {}) + filters = {"suspended": True, "st": True, "limit_up_down": True, "missing_close": True} + hits = tradability_hit_stats(uni, panel, [date], filters) + assert int(hits.loc["suspended", "hits"]) == 1 + assert int(hits.loc["is_st", "hits"]) == 1 + assert int(hits.loc["at_up_limit", "hits"]) == 1 + assert int(hits.loc["missing_close", "hits"]) == 0 + assert hits.attrs["candidates"] == 4 + assert hits.attrs["tradable"] == 1 # only 000001.SZ survives + + +def test_tradability_hit_stats_multiflag_counted_once(): + # A name flagged by MULTIPLE filters must be counted ONCE, in the first + # matching bucket (mirrors apply_tradable_filters' first-match continue), so + # the buckets stay exclusive and sum(hits) == candidates - tradable. + date = pd.Timestamp("2024-01-31") + idx = pd.MultiIndex.from_product([[date], ["000001.SZ"]], names=["date", "symbol"]) + panel = pd.DataFrame( + {"close": [10.0], "suspended": [True], "is_st": [True], + "at_up_limit": [False], "at_down_limit": [False]}, + index=idx, + ) + uni = StaticUniverse(["000001.SZ"], {}) + hits = tradability_hit_stats( + uni, panel, [date], {"suspended": True, "st": True, "limit_up_down": True} + ) + assert int(hits.loc["suspended", "hits"]) == 1 # first match + assert int(hits.loc["is_st", "hits"]) == 0 # NOT double-counted + assert int(hits["hits"].sum()) == hits.attrs["candidates"] - hits.attrs["tradable"] + + +def test_tradability_hit_stats_missing_close_counted(): + panel = _flag_panel().copy() + panel.loc[(pd.Timestamp("2024-01-31"), "000001.SZ"), "close"] = np.nan + uni = StaticUniverse(["000001.SZ"], {}) + hits = tradability_hit_stats(uni, panel, [pd.Timestamp("2024-01-31")], {}) + assert int(hits.loc["missing_close", "hits"]) == 1 + assert hits.attrs["tradable"] == 0 + + +# --------------------------------------------------------------------------- # +# reconstruct_holdings +# --------------------------------------------------------------------------- # +def test_reconstruct_holdings_mirrors_topn_build(): + date = pd.Timestamp("2024-01-31") + syms = ["000001.SZ", "000002.SZ", "000003.SZ"] + panel = pd.DataFrame( + {"close": [1.0, 1.0, 1.0]}, + index=pd.MultiIndex.from_product([[date], syms], names=["date", "symbol"]), + ) + score_panel = pd.Series( + [0.3, 0.1, 0.2], + index=pd.MultiIndex.from_product([[date], syms], names=["date", "symbol"]), + name="score", + ) + uni = StaticUniverse(syms, {}) + holdings = reconstruct_holdings( + _FrameScores(score_panel), uni, panel, TopNEqualWeight(2), [date] + ) + assert len(holdings) == 2 # top_n=2 + held = set(holdings["symbol"]) + assert held == {"000001.SZ", "000003.SZ"} # two highest scores + assert holdings["weight"].tolist() == [0.5, 0.5] + # rank 1 is the highest score (000001.SZ @ 0.3) + top = holdings.sort_values("rank").iloc[0] + assert top["symbol"] == "000001.SZ" + + +def test_holdings_dates_equal_settled_nav_index_not_candidates(): + # Regression for the HIGH finding: the driver SKIPS a terminal rebalance with + # no forward holding period, so the diagnostics must key off nav_table.index + # (settled dates), NOT driver.rebalance_dates() (candidate dates). Holdings + # listed for the skipped terminal date would be a phantom period. + symbols = ["000001.SZ", "000002.SZ", "000003.SZ"] + cal = pd.bdate_range("2024-01-01", "2024-03-29") # spans 3 month-ends + idx = pd.MultiIndex.from_product([cal, symbols], names=["date", "symbol"]) + close = [100.0 + i + j for i in range(len(cal)) for j in range(len(symbols))] + panel = pd.DataFrame({"close": close}, index=idx) + + uni = StaticUniverse(symbols, {}) + constructor = TopNEqualWeight(2) + probe = BacktestDriver( + universe=uni, scores=_FrameScores(pd.Series(dtype=float)), + constructor=constructor, execution=SimExecution(), prices=panel, + ) + candidate_dates = probe.rebalance_dates() # 3 (Jan/Feb/Mar month-ends) + # scores at every candidate date so each settled period holds names. + score_idx = pd.MultiIndex.from_product([candidate_dates, symbols], names=["date", "symbol"]) + score_panel = pd.Series( + [0.3 - j * 0.1 for _ in candidate_dates for j in range(len(symbols))], + index=score_idx, name="score", + ) + scores = _FrameScores(score_panel) + driver = BacktestDriver( + universe=uni, scores=scores, constructor=constructor, + execution=SimExecution(), prices=panel, + ) + nav = driver.run() + + assert len(nav) == len(candidate_dates) - 1 # terminal date skipped (BT-003) + holdings = reconstruct_holdings(scores, uni, panel, constructor, list(nav.index)) + assert set(holdings["date"]) == set(nav.index) + assert candidate_dates[-1] not in set(holdings["date"]) # no phantom terminal + + +# --------------------------------------------------------------------------- # +# financial_coverage_at_dates +# --------------------------------------------------------------------------- # +def test_financial_coverage_counts_non_nan_members(): + date = pd.Timestamp("2024-01-31") + syms = ["000001.SZ", "000002.SZ", "000003.SZ"] + idx = pd.MultiIndex.from_product([[date], syms], names=["date", "symbol"]) + aligned = pd.Series([1.0, np.nan, 3.0], index=idx, name="roe") + uni = StaticUniverse(syms, {}) + cov = financial_coverage_at_dates(aligned, uni, _flag_panel(), [date]) + row = cov.iloc[0] + assert int(row["n_members"]) == 3 + assert int(row["n_covered"]) == 2 + assert row["coverage"] == pytest.approx(2 / 3) + + +# --------------------------------------------------------------------------- # +# render_phase2_baseline — all required sections + no secret leak +# --------------------------------------------------------------------------- # +def _synthetic_result() -> Phase2Result: + cfg = load_config(_CONFIG) # real config (carries the secret FILE PATH) + date = pd.Timestamp("2024-01-31") + nav = pd.DataFrame( + { + "nav": [1.01], + "gross_return": [0.011], + "cost": [0.001], + "turnover": [1.0], + "net_return": [0.01], + }, + index=pd.Index([date], name="date"), + ) + holdings = pd.DataFrame( + {"date": [date, date], "symbol": ["000001.SZ", "000002.SZ"], + "weight": [0.5, 0.5], "rank": [1, 2]} + ) + hits = tradability_hit_stats( + StaticUniverse(["000001.SZ"], {}), _flag_panel(), [date], + {"suspended": True, "st": True, "limit_up_down": True}, + ) + cov = pd.DataFrame( + {"date": [date], "n_members": [2], "n_covered": [1], "coverage": [0.5]} + ) + qret = pd.DataFrame({1: [0.01], 2: [0.02]}, index=[date]) + return Phase2Result( + config=cfg, + elapsed_seconds=12.3, + first_trade_date=date, + last_trade_date=date, + trade_days=1, + panel_rows=2, + panel_symbols=2, + universe_summary=summarize_universe(_pit_universe(), "index", "2024-01-01", "2024-03-01"), + financial_field="roe", + financial_coverage_overall=0.5, + financial_coverage_by_rebalance=cov, + tradability_hits=hits, + rebalance_dates=(date,), + candidate_rebalance_dates=(date, pd.Timestamp("2024-02-29")), + skipped_terminal_dates=(pd.Timestamp("2024-02-29"),), + holdings=holdings, + factor_name="momentum_20", + nav_table=nav, + avg_turnover=1.0, + cost_drag=0.001, + ic_mean=0.05, + ic_ir=0.5, + quantile_returns=qret, + performance={"annual_return": 0.1, "max_drawdown": -0.05, + "volatility": 0.2, "sharpe": 0.5}, + downgrades=("DATA PATH = REAL tushare: ...", "small-scale baseline"), + report_path=Path("artifacts/reports/phase2_real_baseline.md"), + log_path=Path("artifacts/logs/run_phase2_baseline.log"), + ) + + +def test_render_has_all_required_sections(): + md = render_phase2_baseline(_synthetic_result()) + for section in phase2_baseline_required_sections(): + assert section in md, f"missing report section: {section}" + + +def test_render_does_not_leak_secret_file_or_token(): + result = _synthetic_result() + md = render_phase2_baseline(result) + # the secret FILE PATH and token KEY must never be echoed into the report. + assert result.config.data.external_secret_file not in md + assert result.config.data.tushare_token_key not in md + assert "token" not in md.lower() + + +def test_render_reports_holdings_and_rebalance_dates(): + md = render_phase2_baseline(_synthetic_result()) + assert "000001.SZ" in md and "000002.SZ" in md # holdings listed + assert "2024-01-31" in md # rebalance date listed diff --git a/universe/index_universe.py b/universe/index_universe.py index 7a71fe9..0f0173e 100644 --- a/universe/index_universe.py +++ b/universe/index_universe.py @@ -44,6 +44,15 @@ def __init__(self, constituents: pd.DataFrame, filters: dict | None = None) -> N self._snapshots: list[pd.Timestamp] = sorted(self._by_date) self._filters = dict(filters or {}) + def membership_snapshots(self) -> dict[pd.Timestamp, list[str]]: + """Return a copy of ``{snapshot_date: [symbols]}`` (read-only reporting). + + Exposes the raw PIT snapshots so reporting/analytics can summarise the + universe (snapshot count, distinct names, churn) WITHOUT reaching into + private state. It returns copies, so callers cannot mutate the universe. + """ + return {date: list(symbols) for date, symbols in self._by_date.items()} + def members(self, date: pd.Timestamp) -> list[str]: """Constituents of the latest snapshot on or before ``date`` (as-of).""" target = pd.Timestamp(date).normalize()