Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 7 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- **SW 层级可配置** `processing.neutralize.industry_level`(L1/L2/L3,**默认 L1**=31 宽板块,中性化标准 + 小截面自由度更稳)。**关键实证**:旧 tag 年化 −17.6% / SW-L1 −10.2% / SW-L2 −9.3% → **L1≈L2**,−17.6→−10 大跳主因是 **tushare→SW 分类切换**(补 PIT 必然:只有 SW 有 in/out 历史可 PIT 化,旧 tag 无法),**与粒度无关**。(曾误判粒度、默认 L2,经 L2 实测推翻,改默认 L1。)
- 不静默退回 current:无 SW 历史的票 → 行业 NaN,被 neutralize 按截面丢弃;每次运行 **实际 level + PIT 覆盖率**进 phase2 报告。
- 真实 baseline(SW-L1,默认):`run-phase2-baseline` OK,symbols=68,settled=11,**PIT SW-L1 coverage 98.53%**(67/68),IC 0.0083,**annual −10.19%**。
- ✅ 当前质量门:`pytest -p no:cacheprovider` **223 passed**;`ruff` clean;`validate-config`(demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`)OK;`run-phase0`(demo)OK。
- ⚠️ 剩余 P2(已显式披露):日线 only、简版 IC/绩效未走 alphalens/quantstats、demo 路径非真数据。
- 路线图下一步:财务因子组合 / 分钟级 / alphalens·quantstats 接入(architecture.html §11)。
- 🔧 **Phase 2-4 标准分析集成**(**PR #7 OPEN/MERGEABLE,待验收/合并**):alphalens-reloaded + quantstats 接入报告(**report-only cross-check**)。
- `analytics/alphalens_adapter.py`(IC mean/IR + 分位均值)+ `analytics/quantstats_adapter.py`(CAGR/Sharpe/maxDD/vol)薄 adapter,各带 `backend` 字段;`_import_*` 间接层使 import-missing 路径可测;alphalens stdout/warnings 已抑制。
- **简版仍权威**(驱动回测 + cross-check);依赖不可用/报错 → 报告披露 backend(unavailable/error,只记异常类型),**绝不静默假装**。
- **不改 alpha/portfolio/runtime/fills/universe**:P0/P2 交易数字不变(demo ic 0.96/annual 0.84 不变;alphalens IC=简版 IC 吻合),只新增 **Standard analytics** 报告段。
- ✅ 当前质量门:`pytest -p no:cacheprovider` **234 passed**;`ruff` clean;`validate-config`(demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`)OK;`run-phase0`(demo)OK。
- ⚠️ 剩余 P2(已显式披露):日线 only、demo 路径非真数据。
- 路线图下一步:财务因子组合 / 分钟级(architecture.html §11)。
10 changes: 7 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,10 @@ data → universe → factors(特征) → alpha(合成/预测) → portfolio(+ri
- `tushare_covariates.pit_sw_intervals(symbols, level)` 读 `index_member_all` 取每股 SW 行业 `in_date`/`out_date` 区间;`data/clean/pit_industry.py::asof_industry` 按 `[in_date,out_date)` 覆盖该日取行业(改分类日新行业生效,PIT-safe;起始日前成分 carry forward 到窗口开始)。
- **SW 层级可配置** `processing.neutralize.industry_level`(L1/L2/L3,**默认 L1**=31 宽板块,中性化标准 + 小截面自由度更稳)。**关键实证**:旧 tag 年化 −17.6% / SW-L1 −10.2% / SW-L2 −9.3% → **L1≈L2**,大跳的主因是 **tushare→SW 分类切换**(补 PIT 的必然:只有 SW 有 in/out 历史可 PIT 化,旧 tag 无法),**与粒度无关**。(注:曾误以为是粒度、默认 L2,经 L2 实测推翻,改默认 L1。)
- **不静默退回 current**:无 SW 历史的票 → 行业 NaN,被 neutralize 按截面丢弃(neutralize 数学不变,本就丢 NaN 行);每次运行 **实际 level + PIT 覆盖率**进 phase2 报告。
- ✅ 质量门:`pytest` **223 passed**(P0=95 / P1=77 / P2-1=15 / P2-2=22 / P2-3=14);`ruff` clean;`validate-config`(demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余 P2(已显式披露):日线 only、简版 IC/绩效未走 alphalens/quantstats、demo 路径非真数据。
- 路线图下一步:财务因子组合 / 分钟级 / alphalens·quantstats 接入(architecture.html §11)。
- ✅ **Phase 2-4 标准分析集成**(**PR #7 OPEN/MERGEABLE,待验收/合并**):alphalens-reloaded + quantstats 接入报告(**report-only cross-check**)。
- `analytics/alphalens_adapter.py`(IC mean/IR + 分位均值)+ `analytics/quantstats_adapter.py`(CAGR/Sharpe/maxDD/vol)薄 adapter,各带 `backend` 字段。
- **简版 numpy/pandas 仍权威**(驱动回测 + cross-check);依赖不可用/报错 → 报告显式披露 backend(unavailable/error,只记异常**类型**不记消息),**绝不静默假装用了标准库**。
- **不改 alpha/portfolio/runtime/fills/universe**:P0/P2 交易数字不变(demo ic 0.96/annual 0.84 不变;实测 alphalens IC=简版 IC 完全吻合),只新增 **Standard analytics** 报告段。
- ✅ 质量门:`pytest` **234 passed**(P0=97 / P1=77 / P2-1=16 / P2-2=22 / P2-3=14 / P2-4=8);`ruff` clean;`validate-config`(demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`)+ `run-phase0`(demo)均 OK。
- ⚠️ 剩余 P2(已显式披露):日线 only、demo 路径非真数据。
- 路线图下一步:财务因子组合 / 分钟级(architecture.html §11)。
12 changes: 10 additions & 2 deletions RUNBOOK.md
Original file line number Diff line number Diff line change
Expand Up @@ -223,9 +223,17 @@ Resolved in P2-2 / P2-3 (was deferred):
neutralizer drops) — never a silent current-tag fallback; the actual level + PIT coverage
are reported in `phase2_real_baseline.md`.

Resolved in P2-4 (was deferred):

- **Standard analytics** — alphalens-reloaded (IC / quantiles) and quantstats (CAGR /
Sharpe / maxDD / vol) are now computed and shown in the report as a **report-only
cross-check** (`analytics/alphalens_adapter.py`, `analytics/quantstats_adapter.py`).
The simple numpy/pandas metrics remain the AUTHORITATIVE backtest result and drive
the run; the standard tools never alter selection / portfolio / execution. When a
library is unavailable or errors, the report discloses the `backend` (no silent fake).

Still downgraded / deferred (disclosed):

- **Demo path** uses offline `DemoFeed` — NOT real data (no PIT/financial meaning).
- **Static universe** option remains a PIT downgrade (use `type: index` for real).
- **Daily bars only**; **simple IC / performance** (numpy/pandas, not
alphalens-reloaded / quantstats).
- **Daily bars only** (minute-level link deferred).
40 changes: 31 additions & 9 deletions TEST_REPORT.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 (bias-boundary → execution realism → PIT industry)
# TEST_REPORT — Phase 0 + Phase 1 + Phase 2 (bias-boundary → execution realism → PIT industry → standard analytics)

## Commands

Expand All @@ -17,14 +17,14 @@ Run from the repo root with the project python (env `quant_mf`):

| Gate | Command | Result |
|---|---|---|
| Unit + integration | `pytest -q` | **223 passed, 0 failed** |
| Unit + integration | `pytest -q` | **234 passed, 0 failed** |
| Lint | `ruff check .` | **All checks passed** |
| Config validation | `validate-config` (demo + `example_tushare.yaml` + `phase2_real_baseline.yaml`) | exit `0`, prints `OK` |
| End-to-end run | `run-phase0` (demo) | exit `0`, writes `artifacts/reports/phase0_summary.md` |

Counts below are the actual per-file `pytest` numbers (sum = 223).
Counts below are the actual per-file `pytest` numbers (sum = 234).

## Per-file breakdown — Phase 0 core (95)
## Per-file breakdown — Phase 0 core (97)

| Test file | Tests | Area |
|---|---|---|
Expand All @@ -42,7 +42,7 @@ Counts below are the actual per-file `pytest` numbers (sum = 223).
| `test_backtest_driver.py` | 7 | backtest driver |
| `test_analytics_factor.py` | 5 | IC / quantile |
| `test_analytics_performance.py` | 3 | performance metrics |
| `test_phase0_pipeline.py` | 8 | end-to-end pipeline |
| `test_phase0_pipeline.py` | 10 | end-to-end pipeline (+ std-analytics additive / no-leak) |
| `test_bias_audit_report.py` | 6 | bias audit doc (+2: P2-2 + P2-3 disclosures) |

## Per-file breakdown — Phase 1 bias-boundary (79)
Expand All @@ -68,11 +68,11 @@ Counts below are the actual per-file `pytest` numbers (sum = 223).
| `test_tushare_covariates.py` | 5 | stock_basic + daily_basic + index_member_all SW feed (level L1/L2/L3 select, bad level) |
| `test_real_path_config.py` | 4 | demo vs real-path downgrade disclosure (+ PIT industry) |

## Per-file breakdown — Phase 2-1 real-data baseline (15)
## Per-file breakdown — Phase 2-1 real-data baseline (16)

| Test file | Tests | Feature |
|---|---|---|
| `test_phase2_baseline.py` | 15 | collectors, demo/real guard, report-field contract, no-secret-leak, settled-vs-candidate dates, loaded-vs-in-window membership, list_date + PIT-industry coverage |
| `test_phase2_baseline.py` | 16 | collectors, demo/real guard, report-field contract, no-secret-leak, settled-vs-candidate dates, loaded-vs-in-window membership, list_date + PIT-industry coverage, standard-analytics cross-check |

## Per-file breakdown — Phase 2-2 execution realism (22)

Expand All @@ -87,7 +87,14 @@ Counts below are the actual per-file `pytest` numbers (sum = 223).
| Test file | Tests | Red-line / feature |
|---|---|---|
| `test_pit_industry.py` | 12 | SW **as-of** industry (switch at reclassification, carry-forward pre-start, missing → NaN, latest-in_date on overlap) + `enrich_pit_industry` + pipeline wiring (per-date industry, configured SW level passed, no current-tag fallback) + `industry_level` config (default L1, accepts L1/L2/L3, rejects invalid, phase2 config = L1) |
| **Total (P0 + P1 + P2-1 + P2-2 + P2-3)** | **223** | |

## Per-file breakdown — Phase 2-4 standard analytics (8)

| Test file | Tests | Feature |
|---|---|---|
| `test_quantstats_adapter.py` | 4 | quantstats perf metrics + unavailable / error fallback disclosure (no silent fake) |
| `test_alphalens_adapter.py` | 4 | alphalens IC / quantile metrics + unavailable / error fallback + stdout suppression |
| **Total (P0 + P1 + P2-1 + P2-2 + P2-3 + P2-4)** | **234** | |

## Real-data validation (manual, not in CI — TEST-002 keeps the suite network-free)

Expand All @@ -100,6 +107,12 @@ Counts below are the actual per-file `pytest` numbers (sum = 223).

- No test hits the network or reads the tushare token (TEST-002, INV-004): the whole
suite runs on `DemoFeed` / fixtures / monkeypatched SDKs.
- **Optional analytics extras:** `alphalens-reloaded` / `quantstats` are the
`analytics` optional extra in `pyproject.toml` (installed in `quant_mf`). The
adapter success-path tests `pytest.importorskip` them — a clean `.[dev]`-only
environment SKIPS those 3 tests (disclosed) instead of failing; the
fallback/disclosure tests monkeypatch the import and run everywhere. For the
full 234-passed run, install `.[analytics]` too (or use `quant_mf`).
- **P2-2 execution realism (locked by tests):** selection eligibility and execution
feasibility are split. `runtime.fills.simulate_fills` is the cash-coherent
sell-then-buy model — at-up-limit blocks buys, at-down-limit blocks sells,
Expand All @@ -120,7 +133,16 @@ Counts below are the actual per-file `pytest` numbers (sum = 223).
history get NaN (the neutralizer drops them) — never a silent current-tag fallback; the
actual level + PIT coverage are disclosed in the phase2 report. The neutralize math is
unchanged.
- **P2-4 standard analytics (locked by tests):** alphalens-reloaded and quantstats are
thin, report-only adapters (`analytics/alphalens_adapter.py`,
`analytics/quantstats_adapter.py`). The simple numpy/pandas metrics remain the
authoritative backtest result and drive the run; the standard tools are an additive
cross-check that never touches selection / portfolio / execution
(`test_phase0_standard_analytics_is_additive_not_replacing`). Unavailable / erroring
backends are disclosed (`backend` + exception TYPE only, no message) and keep the
simple fallback. Empirically the alphalens IC matched the simple IC exactly on the
demo (0.96), and the demo trading numbers (ic 0.96, annual 0.84) are unchanged.
- A duplicate test-function name across two files was found and renamed during P2-2
(it had been silently shadowing one test in the full-suite run). A second, harmless
duplicate (`test_enrich_does_not_mutate_input` in two files) was verified NOT to drop
a test (per-file sum == full-run total = 217).
a test (per-file sum == full-run total).
90 changes: 90 additions & 0 deletions analytics/alphalens_adapter.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
"""Thin, report-only alphalens-reloaded adapter (P2-4).

Consumes the factor Series + a wide price frame and produces the standard factor
diagnostics (IC mean / IC std / IC-IR, per-quantile mean returns) via
``alphalens``, tagging the ``backend`` it actually used. This is REPORTING ONLY —
it never feeds back into the factor / selection / portfolio, so trading results
are unchanged.

Honesty contract (INV-007): if alphalens is unavailable (``ImportError``) or
raises (small/degenerate cross-sections are common), the result is tagged
``unavailable`` / ``error`` and the caller's simple-pandas ``simple_fallback`` is
carried through — never a silent fake. The ``_import_alphalens`` indirection makes
the unavailable path testable; alphalens' ``print``/warnings are suppressed so the
CLI / report stays clean.
"""

from __future__ import annotations

import contextlib
import io
import math
import warnings

import pandas as pd


def _import_alphalens():
"""Import alphalens (indirection so the ImportError path is testable)."""
import alphalens

return alphalens


def alphalens_factor_metrics(
factor: pd.Series,
prices: pd.DataFrame,
quantiles: int = 5,
period: int = 1,
simple_fallback: dict | None = None,
) -> dict:
"""Standard factor diagnostics from a factor Series + wide price frame.

Args:
factor: MultiIndex(date, symbol) factor values.
prices: wide price frame, index = date, columns = symbol (e.g.
``panel["close"].unstack("symbol")``).
quantiles: number of factor buckets.
period: forward-return horizon (trading days); alphalens labels the column
``f"{period}D"``.
simple_fallback: the authoritative simple-pandas IC metrics, carried
through verbatim when alphalens is unavailable or errors.

Returns:
``{"backend": "alphalens", "ic_mean", "ic_std", "ic_ir", "quantile_mean",
"n_dates"}`` on success; ``{"backend": "unavailable", **fallback}`` or
``{"backend": "error", "error_type", **fallback}`` otherwise.
"""
fallback = dict(simple_fallback or {})
try:
al = _import_alphalens()
except ImportError:
return {"backend": "unavailable", **fallback}

try:
col = f"{period}D"
# alphalens uses level 0 = date, level 1 = asset; rename for clarity and
# suppress its stdout banner / RuntimeWarnings so the CLI stays clean.
fac = factor.copy()
fac.index = fac.index.set_names(["date", "asset"])
with warnings.catch_warnings(), contextlib.redirect_stdout(io.StringIO()):
warnings.simplefilter("ignore")
factor_data = al.utils.get_clean_factor_and_forward_returns(
fac, prices, quantiles=quantiles, periods=(period,), max_loss=1.0
)
ic = al.performance.factor_information_coefficient(factor_data)[col]
mq = al.performance.mean_return_by_quantile(factor_data)[0][col]
ic_mean = float(ic.mean())
ic_std = float(ic.std(ddof=1)) if len(ic) > 1 else float("nan")
ic_ir = ic_mean / ic_std if (math.isfinite(ic_std) and ic_std != 0) else float("nan")
quantile_mean = {int(q): float(v) for q, v in mq.items()}
return {
"backend": "alphalens",
"ic_mean": ic_mean, "ic_std": ic_std, "ic_ir": float(ic_ir),
"quantile_mean": quantile_mean, "n_dates": int(len(ic)),
}
except Exception as exc: # noqa: BLE001 - any alphalens failure -> disclosed fallback
return {"backend": "error", "error_type": type(exc).__name__, **fallback}


__all__ = ["alphalens_factor_metrics"]
10 changes: 5 additions & 5 deletions analytics/factor.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@
(future-looking) returns. The factor layer must never receive them — see
CLAUDE.md invariant #1 and CONTRACTS.md §3.

Implementation note (INV-007 downgrade): the IC / quantile logic here is a
simple numpy/pandas implementation, not alphalens-reloaded. It is deterministic,
dependency-light, and easy to audit. ``alphalens`` is available and may be wired
in later for the HTML report; any such swap must be recorded in the phase0
report. The simple version is what runs in P0.
Implementation note: the IC / quantile logic here is a simple numpy/pandas
implementation — deterministic, dependency-light, easy to audit — and it remains
the AUTHORITATIVE result that drives the run. Since P2-4, alphalens-reloaded is
wired in as a report-only cross-check via ``analytics/alphalens_adapter.py``
(backend disclosed in the report); it never replaces these numbers.

All functions are pure: inputs are never mutated.
"""
Expand Down
9 changes: 5 additions & 4 deletions analytics/performance.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,11 @@
volatility, Sharpe. ``performance_summary`` bundles them into a plain dict for
the phase0 report.

Implementation note (INV-007 downgrade): these are simple, dependency-light
numpy/pandas computations, NOT quantstats/empyrical. ``quantstats`` is available
and may be used later for the richer HTML tearsheet; any such use must be
recorded in the phase0 report. The simple version is what runs in P0.
Implementation note: these are simple, dependency-light numpy/pandas
computations, and they remain the AUTHORITATIVE backtest result that drives the
run. Since P2-4, quantstats is wired in as a report-only cross-check via
``analytics/quantstats_adapter.py`` (backend disclosed in the report); it never
replaces these numbers.

All functions are pure: inputs are never mutated.

Expand Down
Loading