diff --git a/.gitignore b/.gitignore index 1d4c3f6..ce3160d 100644 --- a/.gitignore +++ b/.gitignore @@ -10,7 +10,18 @@ venv/ .env # Data artifacts (quant: keep large data out of git) -artifacts/ +# +# NO TRAILING SLASH on purpose. `artifacts/` matches only a DIRECTORY, so it did +# not match the `artifacts` SYMLINK the git worktrees carry (pointing at the main +# checkout), leaving it untracked-but-addable -- and a `git add -A` duly committed +# a machine-specific absolute-path symlink. Caught by +# test_frozen_bytes_are_untracked_while_the_manifest_is_tracked, which guards the +# split this repo's tamper checks rest on: if the frozen bytes and their hashes +# both lived in git, one commit could move them together. +artifacts +# Local phase0 output. Named separately because `artifacts` does not match it, +# and NO test guards it -- the same accident here would be silent. +artifacts_local_phase0/ *.parquet *.feather *.h5 diff --git a/docs/factors/d5_hand_anchor_record_manifest.json b/docs/factors/d5_hand_anchor_record_manifest.json new file mode 100644 index 0000000..27f581c --- /dev/null +++ b/docs/factors/d5_hand_anchor_record_manifest.json @@ -0,0 +1,23 @@ +{ + "schema": "d5-hand-anchor-record/1", + "path": "artifacts/refactor_baseline/hand_anchors_d2.json", + "sha256": "6e779a3685c7e1edda237bd9a5297aa6601d634b9a27c3c4dd1b0d903f67b6f4", + "size_bytes": 23525, + "why_pinned": "The C5 anchors leg closed 11/11 against THESE bytes. The record carries no provenance of its own (it records no panels_d2 hash and no git SHA), and it lives in a directory documented as never-overwrite while being, unlike the panels, still regenerable -- qt.hand_anchor_rows rewrites it wholesale, which is how the engine comparison was lost on 2026-07-25. Pinning the sha in git is what makes a further overwrite loud on the READ side, and makes changing the expectation a reviewed action visible in git diff.", + "contents": { + "frozen14_rows": 70, + "daily_pending_engine_rows": 20, + "daily_engine_compared_rows": 0, + "all_ok_frozen14": false, + "top_level_keys": [ + "all_ok_frozen14", + "daily_pending_engine", + "elapsed_seconds", + "frozen14", + "seed", + "tolerance" + ] + }, + "what_this_record_does_NOT_contain": "daily_engine_compared is EMPTY (0 rows) and all_ok_daily is absent. The four ops-rewritten daily factors are selected and hand-computed but NOT compared: the companion that compared them was retired in D5 C6, so those 20 rows stay pending and cannot be completed from this tree. Pinning an INCOMPLETE record is fine; reading 'it is pinned' as 'it is complete' is not. The D2 hand-anchor conclusion itself is not lost -- '88 hand anchors, 0 mismatches' is on file in PR #89 and docs/progress/07_factor_layer_refactor.md. What was lost is the ability to re-derive it.", + "all_ok_frozen14_is_false_on_purpose": "all_ok_frozen14 is false: 5 jump_amount_corr_20 rows mismatch because the hand side was truncated to 14:50 by PR #103 while the frozen panels_d2 panel is the pre-truncation one, held unchanged by convention. See docs/factors/pr_c_cutoff_fix_reference_panel.md section 6-4." +} diff --git a/docs/factors/d5_property_test_migration_map.md b/docs/factors/d5_property_test_migration_map.md index f94fe22..a334507 100644 --- a/docs/factors/d5_property_test_migration_map.md +++ b/docs/factors/d5_property_test_migration_map.md @@ -219,3 +219,31 @@ R16 的门是"删旧 runner 测试文件之前，59 条逐条有映射"。本节 真引用、判定"引用了但不存在"。占位符已改成不含双冒号的描述。**这不是误报**——本表的引用格式 就是双冒号加测试名，任何写成那个形状的东西都应该指向一个真实测试。写这一段本身又踩了一次 （复述时把占位符原样抄了回来），第二次才改对。 + +### 删除后复核（C6 删除 PR，2026-07-31） + +改造 PR 拿到的准入证是在**删除前**的树上跑的。删完**又跑了一次**，两次结果并列如下——R16 的门是 +「覆盖数非降」，所以承重的是**第三行**，不是收集总数。 + +| | 删除前（改造 PR 末） | 删除后（本 PR） | +|---|---|---| +| 旧套件文件 / 测试函数 | 10 / **59** | **0 / 0**（已删） | +| 59 条逐条有映射 | **59 / 59** | 不适用（左列已不存在） | +| **本表引用的新套件 node-ID 仍可收集** | **56 / 56** | **56 / 56** ✅ | +| 家族通配数目属实 | 3 / 3 | **3 / 3** ✅ | +| 引用唯一解析 | 全部 | **全部** ✅ | +| 收集总数（背景，非主张） | 2,701 | 2,642 | + +**结论：覆盖数非降成立。** 新套件这一侧**一条没少**——56 条引用删除后仍全部命中，家族通配数目 +不变，无歧义。少掉的 **59** 条全部是被删掉的那 10 个文件里的测试，**没有一条落在别处** +（node-ID 差分：removed 59 / added 0，且 59 条全部匹配 `tests/test_eval_*_runner.py::`）。 + +⚠️ **审计脚本删除后 rc=1，这不是覆盖下降**：它的 [1]/[2] 两节断言「旧套件应有 10 文件 59 条」， +那是**删除前**的口径，删完必然不成立。**判定覆盖是否下降的是 [3]**，它 0 problems。 +把脚本改成"删完也 rc=0"会掩盖这个区分，所以**没有改**——两份原始输出留在 +`tmp/context/cc_c6_verify_only/r16_{BEFORE,AFTER}_delete.txt`。 + +**耐久守卫 `tests/test_property_test_migration_map.py` 的表现符合设计**：删除后 **6 passed + +1 skipped**——`test_no_legacy_runner_test_is_missing_from_the_map` 空过（旧文件已不存在， +它的职责是"只要旧文件还在就不许有没被映射的"），`test_the_claimed_total_matches_the_legacy_files_while_they_exist` +显式 skip。**没有一条因为删除而变红**——一个会因删除变红的守卫，对本表要授权的这个 PR 就是陷阱。 diff --git a/docs/factors/d5_runner_difference_catalogue.md b/docs/factors/d5_runner_difference_catalogue.md index bbeb8a9..f4c688d 100644 --- a/docs/factors/d5_runner_difference_catalogue.md +++ b/docs/factors/d5_runner_difference_catalogue.md @@ -937,9 +937,7 @@ anchors 的结论不受影响，**只重跑 reports 腿**。 manifest 行、**15/15 通过**，`panel_reconcile --verify` 14/14 `max_rel=0.0` ⇒ **面板此刻完好，C5 的结论不受影响**。 -**处置**：修在**下一个 PR（删除 PR）**——那个 PR 的主题正是「让 frozen-forever 名副其实」， -把 `qt.panel_freeze.verify_frozen_panels` 接进 C5 harness 的**三个**读入点正属于它。 -**本 PR 不动 C5 harness。** +**处置：三处已在删除 PR（C6 part 2）接上，见下 §六之四。** ### (3之二) `hand_anchor_rows` 的 payload 只写 pending、不写 compared —— 已知后果，本 PR 不改行为 @@ -967,3 +965,56 @@ C6 退役的三个工具对 11 个旧 runner 的依赖**不是一个量级**： **裁定维持**（owner 2026-07-28 的裁定是「退役重生成能力、基线 frozen-forever」，那是**目的**， 不是从「依赖 11 个 runner」推出来的推论；依赖强弱不改变裁定）。但如实记下： **第三条退役真的放弃了一个未必注定失效的能力**，这一点已单独提给 owner。 + + +## 六之四、C6 删除 PR 的收口（2026-07-31） + +### 已做：三个裸读点全部接上校验 + +| 读入点 | 接了什么 | 失败形态 | +|---|---|---| +| `run_panels_mode` | `verified_frozen_panel_path()` → 按 **git 里的**哈希核该面板 | `ReconciliationError`，点名因子与两个哈希 | +| `run_anchors_mode` | 同上（warmup 网格那次读） | 同上 | +| `load_anchor_rows` | `verify_anchor_record()` → 核 `hand_anchors_d2.json` 的 **sha + 内容清单** | `ReconciliationError` | + +**按因子校验，不是整棵扫**：harness 本来就按因子跑，为读 1 个面板而验 15 个（3.0s vs 0.2s）， +是把校验变成大家会去关掉的东西；**一个因为太慢而被关掉的校验等于没有**。 +**jump 的分叉写在代码注释里**：两棵树都有 `jump_amount_corr_20.parquet` 而它们是**两个不同定义** +（D1 是 v1.0 未截断、pr_c 是 v1.1 已截断），`frozen_panel_path()` 把 jump 路由到 pr_c， +所以校验**按路径**跟着同一个分叉走（不按 factor_id 重新推导，两者因此不会漂）。 + +### 已做：`hand_anchors_d2.json` 用 git 里的 manifest 钉住（方案 C-1） + +新 `docs/factors/d5_hand_anchor_record_manifest.json`（**入 git**，记录本身仍 gitignored）。 +**为什么是 C-1 而不是更强的 C-2**：C5 的 anchors 腿刚刚以 **11/11** 收口，比的正是盘上这份字节； +C-2 需要重跑，**而重跑就会覆盖它** ⇒ C5 的一条腿会变成「对着一份已不存在的记录验过」。 +**所以先钉住 C5 实际跑过的那份**。 + +⚠️ **manifest 明写这份被钉住的记录「有什么、没有什么」，并且清单是被程序核对的、不只是散文**： +`daily_engine_compared` **为 0 行**、`all_ok_daily` 缺席。**钉住一份残缺记录可以，把它说成完整的不行。** +（D2 手算锚的**结论**没丢——「88 行分层手算锚 0 失配」在 PR #89 与 `docs/progress/07` 在案； +丢的是**重新验一次**的能力。）另记：`all_ok_frozen14` 为 false 是**预期**——5 行 jump 失配源于 +手算侧已按 PR #103 截断而冻结的 panels_d2 按约定保持未截断，见 +`pr_c_cutoff_fix_reference_panel.md` §六之四。 + +### 已做：`hand_anchor_rows` 覆盖需显式表态（方案 D-1） + +目标已存在 → **拒绝**，除非 `--allow-overwrite`（或 `--selection` 写别处）。拒绝信息按**差集** +算出「这次跑不会重新产生的键」，所以将来新增键**不靠谁记得**去维护清单；并说明 +`daily_engine_compared` 无法重建、以及覆盖后必须同 commit 更新 manifest（否则读侧会拒新字节）。 +**检查放在 ~7 分钟计算之前**，因为在长跑结束时才说"拒绝"，教出来的是反射性加 `--allow-overwrite`； +`PRODUCED_RECORD_KEYS` 在写出时被断言等于实际 payload 键集，所以它不会与现实漂移。 + +**D-3（把旧 `daily_engine_compared` carry forward）已否决，禁翻烧饼**：它会把**对着旧引擎值算出的** +比较结果拼进**刚重算的** `frozen14` ⇒ **一份内部不自洽的记录，比诚实的缺失更糟**。 +记在案是因为**它很诱人**——把否决过的诱人选项写进案卷，比只写选中的那个有用。 + +### 未做，留后续 PR（C-2 + D-4，是同一件事） + +1. **C-2 provenance**：让 `hand_anchor_rows` 把它读过的 `panels_d2` canonical hash 写进记录， + 读侧核「锚是对着现在要比的这批面板算的」——**把记录绑到它所描述的东西**，而不只是「自钉住以来没变」。 +2. **D-4 搬家**：把该记录移出 `artifacts/refactor_baseline/`。它**可再生**（`hand_anchor_rows` 至今可跑）， + 而那个目录写着「无法再生、绝不覆盖」——**一条对 1/113 文件为假的规则**，正是本仓一直在猎的那类缺陷。 + +两件必须一起做，且那个 PR **要连 anchors 腿一起重验**（C-2 要求重跑，重跑即覆盖 C5 验过的字节）。 +本 PR 主职是删除（已 −9,188 行），再叠文件搬迁 + 路径改 + 文档改，是在最不该出错的 PR 里加出错面。 diff --git a/qt/eval_amp_marginal_anomaly_vol.py b/qt/eval_amp_marginal_anomaly_vol.py deleted file mode 100644 index 0914789..0000000 --- a/qt/eval_amp_marginal_anomaly_vol.py +++ /dev/null @@ -1,515 +0,0 @@ -"""run-eval-amp-marginal-anomaly-vol: the third real factor evaluation (PR-E). - -Reproduces the Changjiang high-frequency-factor series #19 (长江证券《高频因子（十九）》, -reportId 5462994) "amplitude marginal-anomaly relative-volatility" factor as a -first-class :class:`~factors.compute.intraday_derived.AmpMarginalAnomalyVolFactor` and -runs it through the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL -cached A-share data (CSI500, PIT membership) — the same contract-driven loop PR-C / -PR-D used. - -The factor is a DAILY signal derived from 5min bars (themselves DERIVED from the 1min -cache; see ``data.clean.intraday_amp_anomaly.compute_amp_marginal_anomaly_vol``): pool -the trailing 20 trading days of PIT-truncated (14:50) 5min bars, form within-day -``(|Δamp|, r)`` pairs, select the bars whose ``|Δamp| > μ + σ`` of the pooled -``|Δamp|``, and take the ddof=1 std of the RETURNS on those selected bars. It is -executed CLOSE-TO-CLOSE (daily default), so ``is_intraday=False`` (the reasoning is -documented on the factor's spec). - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on a -fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats line). -Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the anomaly-vol -factor adds alpha BEYOND value / low-vol. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_amp_anomaly import ( - AMP_ANOMALY_LOOKBACK_DAYS, - AMP_ANOMALY_MIN_POOL, - AMP_ANOMALY_MIN_SELECTED, - AMP_ANOMALY_SIGMA_K, - compute_amp_marginal_anomaly_vol, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import AmpMarginalAnomalyVolFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_amp_marginal_anomaly_vol" -_REPORT_STEM = "eval_amp_marginal_anomaly_vol" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _AnomalyVolMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw amp-marginal-anomaly-vol - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_amp_anomaly_vol_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - min_pool: int, - min_selected: int, - sigma_k: float, -) -> _AnomalyVolMinuteLoad: - """Compute the raw amp-marginal-anomaly-vol panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is - NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch - closure, so ``stk_mins`` live calls are provably zero (a symbol with no cached - minute simply yields no rows and is disclosed as empty). The 1min -> 5min derivation - happens inside ``compute_amp_marginal_anomaly_vol``. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_amp_marginal_anomaly_vol( - bars, - lookback_days=lookback_days, - min_pool=min_pool, - min_selected=min_selected, - sigma_k=sigma_k, - name=spec.factor_id, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-amp-marginal-anomaly-vol blocked: no requested symbol produced a " - f"cached amp-marginal-anomaly-vol value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms " - "it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - return _AnomalyVolMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, - so ``winsorize`` is declared None (nothing was clipped) even though the config may - toggle it. z-score + industry/size neutralization ARE applied, so they are - declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS section run - so the Predictive axis can be assessed. ``is_exploratory=True``: this is a - reproduction on a shorter window / narrower neutralization than the report, not a - return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-amp-marginal-anomaly-vol requires an 'oos' section (split_date) " - "so the Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the - # run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class AmpAnomalyVolEvalResult: - """Immutable summary of one run-eval-amp-marginal-anomaly-vol run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-amp-marginal-anomaly-vol needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-amp-marginal-anomaly-vol needs data.cache.enabled=true (it reads " - "the persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-amp-marginal-anomaly-vol needs universe.type='index' (PIT " - f"membership, e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-amp-marginal-anomaly-vol expects processing.neutralize.enabled=" - "true (industry + size neutralization, matching the report's neutral column " - "and the EvalConfig declaration)." - ) - - -def run_eval_amp_marginal_anomaly_vol(config_path: str) -> AmpAnomalyVolEvalResult: - """Run the two real amp-marginal-anomaly-vol evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = AmpMarginalAnomalyVolFactor(lookback_days=AMP_ANOMALY_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Amp-marginal-anomaly-vol factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_amp_anomaly_vol_panel( - cfg, symbols, spec, logger, - lookback_days=AMP_ANOMALY_LOOKBACK_DAYS, - min_pool=AMP_ANOMALY_MIN_POOL, - min_selected=AMP_ANOMALY_MIN_SELECTED, - sigma_k=AMP_ANOMALY_SIGMA_K, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return AmpAnomalyVolEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "AmpAnomalyVolEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_amp_marginal_anomaly_vol", -] diff --git a/qt/eval_intraday_amp_cut.py b/qt/eval_intraday_amp_cut.py deleted file mode 100644 index eb12140..0000000 --- a/qt/eval_intraday_amp_cut.py +++ /dev/null @@ -1,538 +0,0 @@ -"""run-eval-intraday-amp-cut: the fifth real factor evaluation (PR-G). - -Reproduces the SECOND factor of the Kaiyuan market-microstructure series #30 (开源证券 -《高频振幅因子的内部切割——市场微观结构系列（30）》, reportId 4988549) — the "日内振幅切割因子" -(§3, thought (2)) — as a first-class -:class:`~factors.compute.intraday_derived.IntradayAmpCutFactor` and runs it through the -FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C / PR-D / PR-E / PR-F used. - -DISTINCT FROM PR-D (``minute_ideal_amp``, the report's flagship, thought (1)) which pools -the trailing 10 days' minutes into ONE set and cuts by minute CLOSE PRICE: this factor -cuts EACH DAY independently by the 1-MINUTE RETURN, produces a daily ``V_day`` series, -takes its trailing-10-valid-day ``V_mean`` / ``V_std``, and combines them CROSS- -SECTIONALLY (z-score each per date, average). See ``data.clean.intraday_amp_cut``. - -The factor is a DAILY signal executed CLOSE-TO-CLOSE (daily default), so -``is_intraday=False`` (the reasoning is documented on the factor's spec). - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the minute -store has no fetch closure — a miss simply yields no rows); the daily / universe / -covariate endpoints go through the shared read-through cache, which on a fully-warmed -cache does zero gap fetches (disclosed via the run-log cache-stats line). Forward returns -are computed ONLY at the evaluator/analytics boundary from ``ctx.price_panel`` — the -factor computation never sees a future return. - -The cross-sectional standardization (report step 4) needs EVERY symbol's ``(V_mean, -V_std)`` present before it can z-score a date. The loader therefore streams one symbol at -a time through ``compute_amp_cut_stats`` (memory-bounded), assembles the full-universe -two-column stats panel, then calls ``combine_amp_cut_cross_section`` ONCE — the CSI500 -covered set is the cross-section (PINNED, disclosed on the factor spec). - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the amplitude-cut -factor adds alpha BEYOND value / low-vol. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_amp_cut import ( - AMP_CUT_LAMBDA, - AMP_CUT_LOOKBACK_DAYS, - AMP_CUT_MIN_CROSS_SECTION, - AMP_CUT_MIN_DAY_MINUTES, - AMP_CUT_MIN_VALID_DAYS, - V_MEAN_COL, - V_STD_COL, - combine_amp_cut_cross_section, - compute_amp_cut_stats, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import IntradayAmpCutFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_intraday_amp_cut" -_REPORT_STEM = "eval_intraday_amp_cut" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol stats -> ONE cross-sectional combine) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _AmpCutMinuteLoad: - """Diagnostics from the cache-only per-symbol stats read + cross-sectional combine.""" - - factor: pd.Series # MultiIndex(date, symbol) final combined intraday_amp_cut - stats_rows: int # rows in the assembled (V_mean, V_std) panel - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite (V_mean, V_std) pair - empty_symbols: tuple[str, ...] # requested but no cached minute / no finite stat - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_amp_cut_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - lam: float, - min_day_minutes: int, - min_valid_days: int, - min_cross_section: int, -) -> _AmpCutMinuteLoad: - """Compute the final amp-cut factor from the minute cache (stats loop + combine). - - Memory-bounded: one symbol's minute history is read, aggregated to its daily - ``(V_mean, V_std)`` stats, and discarded before the next — the multi-year all-symbol - minute panel is NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` - has no fetch closure, so ``stk_mins`` live calls are provably zero (a symbol with no - cached minute simply yields no rows and is disclosed as empty). The per-day cut and - trailing aggregation run inside ``compute_amp_cut_stats``; the CROSS-SECTIONAL z-score - combine (report step 4) is applied ONCE to the assembled full-universe stats panel. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - stats_frames: list[pd.DataFrame] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - stats = compute_amp_cut_stats( - bars, - lookback_days=lookback_days, - lam=lam, - min_day_minutes=min_day_minutes, - min_valid_days=min_valid_days, - ) - # A symbol counts as covered iff it produced >= 1 finite (V_mean, V_std) pair - # (those are the only rows that can enter a date's cross-section). - if not stats.empty and stats[[V_MEAN_COL, V_STD_COL]].notna().all(axis=1).any(): - stats_frames.append(stats) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not stats_frames: - raise ValueError( - "run-eval-intraday-amp-cut blocked: no requested symbol produced a cached " - f"intraday-amp-cut stat over [{cfg.data.start}, {cfg.data.end}]. The minute " - "cache is required (this runner never warms it); check coverage." - ) - stats = pd.concat(stats_frames).sort_index() - # ONE cross-sectional combine over the full covered universe (report step 4); the - # CSI500 covered set is the cross-section (PINNED, disclosed on the spec). - factor = combine_amp_cut_cross_section( - stats, min_cross_section=min_cross_section, name=spec.factor_id - ) - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a stat, %d raw 1min rows " - "read, %d stats rows, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(stats), len(factor), - ) - return _AmpCutMinuteLoad( - factor=factor, - stats_rows=int(len(stats)), - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, - so ``winsorize`` is declared None (nothing was clipped) even though the config may - toggle it. z-score + industry/size neutralization ARE applied, so they are - declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS section run - so the Predictive axis can be assessed. ``is_exploratory=True``: this is a - reproduction on a shorter window / narrower neutralization than the report, not a - return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-intraday-amp-cut requires an 'oos' section (split_date) so the " - "Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the - # run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class AmpCutEvalResult: - """Immutable summary of one run-eval-intraday-amp-cut run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - stats_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-intraday-amp-cut needs data.source='tushare' (real cached A-share " - f"data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-intraday-amp-cut needs data.cache.enabled=true (it reads the " - "persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-intraday-amp-cut needs universe.type='index' (PIT membership, " - f"e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-intraday-amp-cut expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_intraday_amp_cut(config_path: str) -> AmpCutEvalResult: - """Run the two real intraday-amp-cut evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = IntradayAmpCutFactor(lookback_days=AMP_CUT_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Amp-cut factor: cache-only per-symbol stats + ONE cross-sectional combine -> process. - load = _load_amp_cut_panel( - cfg, symbols, spec, logger, - lookback_days=AMP_CUT_LOOKBACK_DAYS, - lam=AMP_CUT_LAMBDA, - min_day_minutes=AMP_CUT_MIN_DAY_MINUTES, - min_valid_days=AMP_CUT_MIN_VALID_DAYS, - min_cross_section=AMP_CUT_MIN_CROSS_SECTION, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return AmpCutEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - stats_rows=load.stats_rows, - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "AmpCutEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_intraday_amp_cut", -] diff --git a/qt/eval_jump_amount_corr.py b/qt/eval_jump_amount_corr.py deleted file mode 100644 index 3acd572..0000000 --- a/qt/eval_jump_amount_corr.py +++ /dev/null @@ -1,502 +0,0 @@ -"""run-eval-jump-amount-corr: the FIRST real factor evaluation (PR-C). - -Reproduces the Kaiyuan report §6 "price-jump turnover correlation" factor as a -first-class :class:`~factors.compute.intraday_derived.JumpAmountCorrFactor` and -runs it through the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL -cached A-share data (CSI500, PIT membership), closing the contract-driven loop. - -The factor is a DAILY signal derived from 1min bars (see -``data.clean.intraday_aggregate.compute_jump_amount_corr``): the trailing-20- -trading-day lagged correlation between the amount at price-jump minutes and the -amount at the strictly-next minute. It is executed CLOSE-TO-CLOSE (daily default), -so ``is_intraday=False`` (the reasoning is documented on the factor's spec). - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on -a fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats -line). Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis -is NOT_ASSESSED) and once with the project's independently-confirmed book -(value_ep / value_bp / volatility_20) so the Incremental axis measures whether -jump-amount-corr adds alpha BEYOND value / low-vol. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_aggregate import ( - JUMP_LOOKBACK_DAYS, - JUMP_MIN_PAIRS, - compute_jump_amount_corr, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import JumpAmountCorrFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_jump_amount_corr" -_REPORT_STEM = "eval_jump_amount_corr" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _JumpMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw jump-amount-corr - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_jump_factor_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - min_pairs: int, -) -> _JumpMinuteLoad: - """Compute the raw jump-amount-corr panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily - factor series, and discarded before the next — the multi-year all-symbol minute - panel is NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` - has no fetch closure, so ``stk_mins`` live calls are provably zero (a symbol - with no cached minute simply yields no rows and is disclosed as empty). - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - # Whole-day READ window; the 14:50 PIT truncation is applied inside - # ``compute_jump_amount_corr`` (as in every sibling minute factor), so the - # post-cutoff bars this read returns are dropped before any pooling. This - # read is NOT the cutoff — a reader who assumes it is will conclude the - # factor is truncated when the compute is not, which is exactly how the - # missing truncation survived review here. - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_jump_amount_corr( - bars, lookback_days=lookback_days, min_pairs=min_pairs, name=spec.factor_id - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-jump-amount-corr blocked: no requested symbol produced a " - f"cached minute jump-amount-corr value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms " - "it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - return _JumpMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the - two ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 - no-op, so ``winsorize`` is declared None (nothing was clipped) even though the - config may toggle it. z-score + industry/size neutralization ARE applied, so - they are declared. ``oos_split`` (from the config's ``oos`` block) makes the - OOS section run so the Predictive axis can be assessed. ``is_exploratory=True``: - this is a reproduction on a shorter window / narrower neutralization than the - report, not a return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-jump-amount-corr requires an 'oos' section (split_date) so the " - "Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report - # (not a screen of our own); the report's own 17-factor screen is a caveat - # noted in the run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class JumpEvalResult: - """Immutable summary of one run-eval-jump-amount-corr run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-jump-amount-corr needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-jump-amount-corr needs data.cache.enabled=true (it reads the " - "persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-jump-amount-corr needs universe.type='index' (PIT membership, " - f"e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-jump-amount-corr expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column " - "and the EvalConfig declaration)." - ) - - -def run_eval_jump_amount_corr(config_path: str) -> JumpEvalResult: - """Run the two real jump-amount-corr evaluations (cache-only) and write reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = JumpAmountCorrFactor(lookback_days=JUMP_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info("eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The - # value factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Jump factor: cache-only per-symbol minute aggregation -> raw -> process. - load = _load_jump_factor_panel( - cfg, symbols, spec, logger, - lookback_days=JUMP_LOOKBACK_DAYS, min_pairs=JUMP_MIN_PAIRS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor - # on the daily trading grid so the analytics boundary has a price for every date. - jump_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - jump_processed = _process_factors(cfg, jump_raw.to_frame(spec.factor_id), panel) - factor_series = jump_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return JumpEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "JumpEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_jump_amount_corr", -] diff --git a/qt/eval_minute_ideal_amplitude.py b/qt/eval_minute_ideal_amplitude.py deleted file mode 100644 index 9759b4a..0000000 --- a/qt/eval_minute_ideal_amplitude.py +++ /dev/null @@ -1,508 +0,0 @@ -"""run-eval-minute-ideal-amplitude: the second real factor evaluation (PR-D). - -Reproduces the Kaiyuan report §30 "minute ideal amplitude" factor -(市场微观结构系列 30) as a first-class -:class:`~factors.compute.intraday_derived.MinuteIdealAmplitudeFactor` and runs it -through the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached -A-share data (CSI500, PIT membership) — the same contract-driven loop PR-C used. - -The factor is a DAILY signal derived from 1min bars (see -``data.clean.intraday_amplitude.compute_minute_ideal_amplitude``): pool the trailing -10 trading days of PIT-truncated (14:50) minutes, rank them by RAW close, and take -the mean-amplitude spread between the top / bottom 25% by close. It is executed -CLOSE-TO-CLOSE (daily default), so ``is_intraday=False`` (the reasoning is -documented on the factor's spec). - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on a -fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats -line). Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether minute ideal -amplitude adds alpha BEYOND value / low-vol. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_amplitude import ( - IDEAL_AMP_LAMBDA, - IDEAL_AMP_LOOKBACK_DAYS, - IDEAL_AMP_MIN_MINUTES, - compute_minute_ideal_amplitude, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import MinuteIdealAmplitudeFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_minute_ideal_amplitude" -_REPORT_STEM = "eval_minute_ideal_amplitude" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _AmpMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw minute-ideal-amplitude - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_minute_ideal_amp_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - lam: float, - min_minutes: int, -) -> _AmpMinuteLoad: - """Compute the raw minute-ideal-amplitude panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily - factor series, and discarded before the next — the multi-year all-symbol minute - panel is NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` - has no fetch closure, so ``stk_mins`` live calls are provably zero (a symbol with - no cached minute simply yields no rows and is disclosed as empty). - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_minute_ideal_amplitude( - bars, - lookback_days=lookback_days, - lam=lam, - min_minutes=min_minutes, - name=spec.factor_id, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-minute-ideal-amplitude blocked: no requested symbol produced a " - f"cached minute ideal-amplitude value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms " - "it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - return _AmpMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 - no-op, so ``winsorize`` is declared None (nothing was clipped) even though the - config may toggle it. z-score + industry/size neutralization ARE applied, so they - are declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS - section run so the Predictive axis can be assessed. ``is_exploratory=True``: this - is a reproduction on a shorter window / narrower neutralization than the report, - not a return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-minute-ideal-amplitude requires an 'oos' section (split_date) " - "so the Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not - # a screen of our own); the report's own factor screen is a caveat noted in - # the run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class MinuteIdealAmpEvalResult: - """Immutable summary of one run-eval-minute-ideal-amplitude run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-minute-ideal-amplitude needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-minute-ideal-amplitude needs data.cache.enabled=true (it reads " - "the persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-minute-ideal-amplitude needs universe.type='index' (PIT " - f"membership, e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-minute-ideal-amplitude expects processing.neutralize.enabled=" - "true (industry + size neutralization, matching the report's neutral " - "column and the EvalConfig declaration)." - ) - - -def run_eval_minute_ideal_amplitude(config_path: str) -> MinuteIdealAmpEvalResult: - """Run the two real minute-ideal-amplitude evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = MinuteIdealAmplitudeFactor(lookback_days=IDEAL_AMP_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Minute ideal-amplitude factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_minute_ideal_amp_panel( - cfg, symbols, spec, logger, - lookback_days=IDEAL_AMP_LOOKBACK_DAYS, - lam=IDEAL_AMP_LAMBDA, - min_minutes=IDEAL_AMP_MIN_MINUTES, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - amp_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - amp_processed = _process_factors(cfg, amp_raw.to_frame(spec.factor_id), panel) - factor_series = amp_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return MinuteIdealAmpEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "MinuteIdealAmpEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_minute_ideal_amplitude", -] diff --git a/qt/eval_peak_interval_kurtosis.py b/qt/eval_peak_interval_kurtosis.py deleted file mode 100644 index b21a08f..0000000 --- a/qt/eval_peak_interval_kurtosis.py +++ /dev/null @@ -1,534 +0,0 @@ -"""run-eval-peak-interval-kurtosis: the sixth real factor evaluation (PR-H). - -Reproduces the SECOND factor of the Kaiyuan market-microstructure series #27 (开源证券 -《高频成交量的峰、岭、谷信息——市场微观结构研究系列（27）》, reportId 4957417, §6) — the -"量峰间隔峰度" factor — as a first-class -:class:`~factors.compute.intraday_derived.PeakIntervalKurtosisFactor` and runs it through -the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C / PR-D / PR-E / PR-F / -PR-G used. - -The factor is a DAILY signal derived DIRECTLY from the 1min cache (see -``data.clean.intraday_peak_interval.compute_peak_interval_kurtosis``): PIT-truncate each -day at 14:50, identify the volume peaks with the taxonomy REUSED from PR-F -(``data.clean.intraday_volume_prv``: same-slot strictly-prior μ+σ eruptive test, mild -1-minute same-session neighbours), measure the gaps between consecutive same-day peaks in -TRADING MINUTES, pool the trailing 20 VALID trading days and take the pool's Fisher -excess (bias-corrected) kurtosis. It is executed CLOSE-TO-CLOSE (daily default), so -``is_intraday=False`` (the reasoning is documented on the factor's spec). The eval CELL is -identical to PR-C..PR-G, so this run is directly comparable to its PR-F sibling — same -peaks, different statistic. - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on a -fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats line). -Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the peak-interval -kurtosis adds alpha BEYOND value / low-vol. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_peak_interval import ( - PEAK_INTERVAL_LOOKBACK_DAYS, - PEAK_INTERVAL_MIN_INTERVALS, - compute_peak_interval_kurtosis, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import PeakIntervalKurtosisFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_peak_interval_kurtosis" -_REPORT_STEM = "eval_peak_interval_kurtosis" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _PeakIntervalMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw peak-interval kurtosis - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_peak_interval_kurtosis_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, - min_intervals: int, -) -> _PeakIntervalMinuteLoad: - """Compute the raw peak-interval-kurtosis panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is - NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch - closure, so ``stk_mins`` live calls are provably zero (a symbol with no cached - minute simply yields no rows and is disclosed as empty). The peak taxonomy (reused - from PR-F) and the interval/kurtosis reduction run entirely inside - ``compute_peak_interval_kurtosis`` on 1min bars. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_peak_interval_kurtosis( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - min_intervals=min_intervals, - name=spec.factor_id, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-peak-interval-kurtosis blocked: no requested symbol produced a " - f"cached peak-interval-kurtosis value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms " - "it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - return _PeakIntervalMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, - so ``winsorize`` is declared None (nothing was clipped) even though the config may - toggle it. z-score + industry/size neutralization ARE applied, so they are - declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS section run - so the Predictive axis can be assessed. ``is_exploratory=True``: this is a - reproduction on a shorter window / narrower neutralization than the report, not a - return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-peak-interval-kurtosis requires an 'oos' section (split_date) so " - "the Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the - # run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class PeakIntervalKurtosisEvalResult: - """Immutable summary of one run-eval-peak-interval-kurtosis run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-peak-interval-kurtosis needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-peak-interval-kurtosis needs data.cache.enabled=true (it reads " - "the persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-peak-interval-kurtosis needs universe.type='index' (PIT " - f"membership, e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-peak-interval-kurtosis expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_peak_interval_kurtosis(config_path: str) -> PeakIntervalKurtosisEvalResult: - """Run the two real peak-interval-kurtosis evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = PeakIntervalKurtosisFactor(lookback_days=PEAK_INTERVAL_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Peak-interval-kurtosis factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_peak_interval_kurtosis_panel( - cfg, symbols, spec, logger, - lookback_days=PEAK_INTERVAL_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - min_intervals=PEAK_INTERVAL_MIN_INTERVALS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return PeakIntervalKurtosisEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "PeakIntervalKurtosisEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_peak_interval_kurtosis", -] diff --git a/qt/eval_peak_ridge_amount_ratio.py b/qt/eval_peak_ridge_amount_ratio.py deleted file mode 100644 index 799bd5c..0000000 --- a/qt/eval_peak_ridge_amount_ratio.py +++ /dev/null @@ -1,609 +0,0 @@ -"""run-eval-peak-ridge-amount-ratio: the eleventh real factor evaluation (PR-M). - -Reproduces the SEVENTH factor of the Kaiyuan market-microstructure series #27 (开源证券 -《高频成交量的峰、岭、谷信息——市场微观结构研究系列（27）》, reportId 4957417, §7.2) — the -"峰岭成交比" factor — as a first-class -:class:`~factors.compute.intraday_derived.PeakRidgeAmountRatioFactor` and runs it through -the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C .. PR-L used. - -THIS IS THE CLOSING FACTOR of the reproduction loop, and it is the one that closes the -FAMILY MAP. The nine prior reproductions covered a COUNT (PR-F volume_peak_count, weak), a -TIMING moment (PR-H peak_interval_kurtosis, null), two price RATIOS (PR-I -valley_relative_vwap, PR-J valley_ridge_vwap_ratio, both PASSING), a price POSITION (PR-L -valley_price_quantile, PASSING and strongest) and a RETURN (PR-K ridge_minute_return, sign -transferred but Reject). Every signal found so far has been a PRICE signal. This factor -carries NO price information whatsoever — both legs are pure traded VALUE — so it is the -one test that distinguishes "only price information survives in this taxonomy" from "the -peak/ridge behavioural split itself carries alpha". A PASS widens the finding to the -taxonomy; a REJECT sharpens it to prices. Either outcome is a legitimate result and is -reported as such. - -The factor is a DAILY signal derived DIRECTLY from the 1min cache (see -``data.clean.intraday_amount_ratio.compute_peak_ridge_amount_ratio``): PIT-truncate each -day at 14:50, classify every visible minute with the taxonomy REUSED from PR-F -(``data.clean.intraday_volume_prv``: same-slot strictly-prior μ+σ eruptive test; a PEAK is -an ISOLATED eruption, a RIDGE is an eruptive minute that is not isolated), total each valid -day's peak and ridge traded amount, and divide the trailing-20-VALID-day SUM of the peak -leg by that of the ridge leg. NOTE THE AGGREGATION: the report specifies a RATIO OF SUMS -here ("计算 20 日量峰总成交额与量岭总成交额，二者做比"), NOT the mean of daily ratios that -§7.1 (PR-J) specifies — the two forms are followed as written in each section. It is -executed CLOSE-TO-CLOSE (daily default), so ``is_intraday=False``. The eval CELL is -identical to PR-C..PR-L, so this run is directly comparable to its siblings. - -PEAK SCARCITY IS MEASURED, NOT ASSUMED — and the asymmetry RUNS THE OTHER WAY THAN PR-J's. -There, ridges were the scarce leg; here PEAKS are, because a peak must erupt AND be -ISOLATED. The valid-day gate therefore asks for >= 5 tradable peak bars against the ridge -leg's >= 10. The runner collects the REALIZED per-day peak-bar distribution, the day -validity rate across the whole universe, and the COUNTERFACTUAL valid-day count at a peak -floor of 10, so the cost of the lowered threshold is a number rather than an assumption. - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the minute -store has no fetch closure — a miss simply yields no rows); the daily / universe / -covariate endpoints go through the shared read-through cache, which on a fully-warmed -cache does zero gap fetches (disclosed via the run-log cache-stats line). Forward returns -are computed ONLY at the evaluator/analytics boundary from ``ctx.price_panel`` — the factor -computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the ratio adds alpha -BEYOND value / low-vol. The with-book run carries a different weight here than for the -price factors: a traded-VALUE mix has no obvious kinship with a value or low-vol bet, so an -Incremental PASS would be a genuinely new exposure rather than a repackaged one — and an -Incremental failure would be the more surprising outcome. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_amount_ratio import ( - PEAK_RIDGE_LOOKBACK_DAYS, - PEAK_RIDGE_MIN_PEAK_BARS, - PEAK_RIDGE_MIN_RIDGE_BARS, - compute_peak_ridge_amount_ratio, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import PeakRidgeAmountRatioFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -# The peak-scarcity disclosure MOVED to qt.factor_eval_disclosures (the single -# home, D5 C4); re-exported here so historical import paths keep working. -from qt.factor_eval_disclosures import PeakCoverage, summarize_peak_coverage -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_peak_ridge_amount_ratio" -_REPORT_STEM = "eval_peak_ridge_amount_ratio" - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _PeakRidgeMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw peak/ridge amount ratio - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - peak_coverage: PeakCoverage - - -def _load_peak_ridge_amount_ratio_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, - min_peak_bars: int, - min_ridge_bars: int, -) -> _PeakRidgeMinuteLoad: - """Compute the raw peak/ridge amount-ratio panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is NEVER - materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch closure, - so ``stk_mins`` live calls are provably zero (a symbol with no cached minute simply - yields no rows and is disclosed as empty). The classification (reused from PR-F) and - the amount-ratio reduction run entirely inside ``compute_peak_ridge_amount_ratio`` on - 1min bars, which read ``volume`` and ``amount``. The per-day bar-count diagnostics are - collected alongside so the peak scarcity can be REPORTED, not assumed; collecting them - does not change the factor. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - diagnostics: list[pd.DataFrame] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_peak_ridge_amount_ratio( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - min_peak_bars=min_peak_bars, - min_ridge_bars=min_ridge_bars, - name=spec.factor_id, - diagnostics_out=diagnostics, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - coverage = summarize_peak_coverage( - diagnostics, - min_peak_bars=min_peak_bars, - min_ridge_bars=min_ridge_bars, - min_classifiable=min_classifiable, - ) - if not series: - raise ValueError( - "run-eval-peak-ridge-amount-ratio blocked: no requested symbol produced a " - f"cached peak/ridge amount-ratio value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms it); " - "check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min rows " - "read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - logger.info("%s", coverage.render()) - return _PeakRidgeMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - peak_coverage=coverage, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price panel - (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as Markdown + - JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 supplies the book - (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the research-style - # dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report( - report: FactorEvalReport, report_dir: Path, stem: str -) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report. - - Surfaces the same comparison quantities PR-K / PR-L extracted (turnover, net long-short - by cost scenario, lag-1 rank autocorrelation + half-life, cross-section size) so PR-M - sits in ONE table with all nine siblings — which matters more for this run than any - other, since the closing deliverable of the loop is the full ten-factor comparison. - ``ic_pearson_mean`` rides alongside the rank ``ic_mean`` to keep testing the PR-K - review's regularity, that a divergence between the two predicts the monotonicity gate - failing. - - ``aligned_spread_*`` is NOT unreliable here: the frozen layer's cost-sign defect only - mis-signs negative-sign factors, and this factor's pre-registered sign is +1. - ``net_long_short_by_cost`` is still surfaced as the primary read, for comparability. - """ - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - coverage = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - ret_risk = _section_payload(report, "return_risk") - stability = _section_payload(report, "stability_cost") - autocorr = dict(stability.get("factor_rank_autocorr_by_lag", {}) or {}) - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_pearson_mean": pred.get("ic_pearson_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": coverage.get("settled_rebalances"), - "effective_samples": coverage.get("effective_samples"), - "span_days": coverage.get("span_days"), - "cross_section_size_mean": coverage.get("cross_section_size_mean"), - "cross_section_size_median": coverage.get("cross_section_size_median"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - "monotonicity_spearman": ret_risk.get("monotonicity_spearman"), - "gross_long_short_mean": ret_risk.get("gross_long_short_mean"), - "net_long_short_by_cost": dict(ret_risk.get("net_long_short_by_cost", {}) or {}), - # sign=+1 -> the aligned spreads are NOT mis-signed for this factor. - "aligned_spread_by_cost": dict(ret_risk.get("aligned_spread_by_cost", {}) or {}), - "long_short_turnover": stability.get("turnover_mean_long_short_legs"), - "rank_autocorr_lag1": autocorr.get(1), - "half_life_periods": stability.get("half_life_periods"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, so - ``winsorize`` is declared None (nothing was clipped) even though the config may toggle - it. z-score + industry/size neutralization ARE applied, so they are declared. - ``oos_split`` (from the config's ``oos`` block) makes the OOS section run so the - Predictive axis can be assessed. ``is_exploratory=True``: this is a reproduction on a - shorter window / narrower neutralization than the report, not a return claim (it caps - the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-peak-ridge-amount-ratio requires an 'oos' section (split_date) so " - "the Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the run's - # prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class PeakRidgeAmountRatioEvalResult: - """Immutable summary of one run-eval-peak-ridge-amount-ratio run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - peak_coverage: PeakCoverage - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-peak-ridge-amount-ratio needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-peak-ridge-amount-ratio needs data.cache.enabled=true (it reads the " - "persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-peak-ridge-amount-ratio needs universe.type='index' (PIT " - f"membership, e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-peak-ridge-amount-ratio expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_peak_ridge_amount_ratio(config_path: str) -> PeakRidgeAmountRatioEvalResult: - """Run the two real peak/ridge amount-ratio evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = PeakRidgeAmountRatioFactor(lookback_days=PEAK_RIDGE_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", - eval_cfg.universe, eval_cfg.oos_split, - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Peak/ridge amount-ratio factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_peak_ridge_amount_ratio_panel( - cfg, symbols, spec, logger, - lookback_days=PEAK_RIDGE_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - min_peak_bars=PEAK_RIDGE_MIN_PEAK_BARS, - min_ridge_bars=PEAK_RIDGE_MIN_RIDGE_BARS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on the - # daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return PeakRidgeAmountRatioEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - peak_coverage=load.peak_coverage, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "PeakCoverage", - "PeakRidgeAmountRatioEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_peak_ridge_amount_ratio", - "summarize_peak_coverage", -] diff --git a/qt/eval_ridge_minute_return.py b/qt/eval_ridge_minute_return.py deleted file mode 100644 index 7c7e151..0000000 --- a/qt/eval_ridge_minute_return.py +++ /dev/null @@ -1,607 +0,0 @@ -"""run-eval-ridge-minute-return: the ninth real factor evaluation (PR-K). - -Reproduces the FIFTH factor of the Kaiyuan market-microstructure series #27 (开源证券 -《高频成交量的峰、岭、谷信息——市场微观结构研究系列（27）》, reportId 4957417, §3) — the -"量岭分钟收益" factor — as a first-class -:class:`~factors.compute.intraday_derived.RidgeMinuteReturnFactor` and runs it through the -FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C .. PR-J used. - -This is the FIFTH reproduction from the same report and it opens a FOURTH statistic -family. The four prior ones covered a COUNT (PR-F, weak), a TIMING moment (PR-H, null) and -two PRICE LEVELS (PR-I / PR-J, both passed). This factor is a RETURN — and it is the -report's ONLY NEGATIVE peak/ridge/valley factor, so the run also tests whether the SIGN -transfers along with the family, not just the magnitude. - -The factor is a DAILY signal derived DIRECTLY from the 1min cache (see -``data.clean.intraday_ridge_return.compute_ridge_minute_return``): PIT-truncate each day at -14:50, classify every visible minute with the taxonomy REUSED from PR-F -(``data.clean.intraday_volume_prv``: same-slot strictly-prior μ+σ eruptive test; a RIDGE is -an eruptive minute that is not an isolated peak), form each minute's return against the -previous VISIBLE bar of the SAME day, sum those returns over the day's ridge bars, and sum -the daily sums over the trailing 20 VALID trading days. It is executed CLOSE-TO-CLOSE -(daily default), so ``is_intraday=False``. The eval CELL is identical to PR-C..PR-J, so -this run is directly comparable to its siblings — same classification, different statistic. - -RIDGE SCARCITY IS MEASURED, NOT ASSUMED, exactly as in PR-J. Ridge bars are structurally -rare (a minute must erupt AND fail the isolation test), and this factor narrows them -further: only ridge bars that CARRY A VALID RETURN count, so the day's first visible bar -drops out even when it is a ridge. The runner therefore collects the realized per-day -distribution of BOTH counts plus the day-validity rate across the whole universe and logs -them, so the return-guard attrition and any coverage regression against PR-J are visible as -numbers rather than hidden behind a threshold. - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the minute -store has no fetch closure — a miss simply yields no rows); the daily / universe / -covariate endpoints go through the shared read-through cache, which on a fully-warmed cache -does zero gap fetches (disclosed via the run-log cache-stats line). Forward returns are -computed ONLY at the evaluator/analytics boundary from ``ctx.price_panel`` — the factor -computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the accumulated -ridge-minute return adds alpha BEYOND value / low-vol. That check matters here because a -trailing sum of returns is a plausible cousin of a REVERSAL signal, and the with-book run -is the one that says whether this is a genuinely new bet. - -⚠️ THIS FACTOR'S PRE-REGISTERED SIGN IS -1, which triggers a KNOWN DEFECT in the frozen -evaluation layer: its ``aligned_spread_*`` fields compute ``sign * (gross - cost)``, which -for a negative sign becomes ``-gross + cost`` — i.e. the trading cost is ADDED BACK rather -than deducted. The frozen layer is deliberately NOT patched here. Any reading of this run's -tradability must use ``net_long_short_by_cost`` (which is sign-agnostic and correct) and -must treat ``aligned_spread_*`` as unreliable. :func:`extract_metrics` therefore surfaces -the cost-scenario net spreads explicitly. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_ridge_return import ( - RIDGE_RETURN_LOOKBACK_DAYS, - RIDGE_RETURN_MIN_RIDGE_BARS, - compute_ridge_minute_return, -) -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import RidgeMinuteReturnFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -# The ridge-scarcity disclosure MOVED to qt.factor_eval_disclosures (the single -# home, D5 C4); re-exported here so historical import paths keep working. -from qt.factor_eval_disclosures import ( - RidgeReturnCoverage, - summarize_ridge_return_coverage, -) -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_ridge_minute_return" -_REPORT_STEM = "eval_ridge_minute_return" - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RidgeReturnMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw ridge-minute-return sum - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - ridge_coverage: RidgeReturnCoverage - - -def _load_ridge_minute_return_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, - min_ridge_bars: int, -) -> _RidgeReturnMinuteLoad: - """Compute the raw ridge-minute-return panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is NEVER - materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch closure, - so ``stk_mins`` live calls are provably zero (a symbol with no cached minute simply - yields no rows and is disclosed as empty). The classification (reused from PR-F) and - the return reduction run entirely inside ``compute_ridge_minute_return`` on 1min bars, - which read ``volume`` and ``close``. The per-day bar-count diagnostics are collected - alongside so the ridge scarcity can be REPORTED, not assumed; collecting them does not - change the factor. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - diagnostics: list[pd.DataFrame] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_ridge_minute_return( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - min_ridge_bars=min_ridge_bars, - name=spec.factor_id, - diagnostics_out=diagnostics, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - coverage = summarize_ridge_return_coverage( - diagnostics, - min_ridge_bars=min_ridge_bars, - min_classifiable=min_classifiable, - ) - if not series: - raise ValueError( - "run-eval-ridge-minute-return blocked: no requested symbol produced a cached " - f"ridge-minute-return value over [{cfg.data.start}, {cfg.data.end}]. The " - "minute cache is required (this runner never warms it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min rows " - "read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - logger.info("%s", coverage.render()) - return _RidgeReturnMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ridge_coverage=coverage, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price panel - (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as Markdown + - JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 supplies the book - (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the research-style - # dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report. - - Beyond the fields PR-C..PR-J surfaced, this also extracts the TRADABILITY comparison - quantities the PR-K analysis needs alongside PR-I / PR-J: the long-short leg turnover, - the net long-short spread at EACH cost scenario, the lag-1 rank autocorrelation with - its implied half-life, and the cross-section size. - - ⚠️ ``net_long_short_by_cost`` is deliberately the tradability field surfaced here. The - frozen layer's ``aligned_spread_*`` computes ``sign * (gross - cost)``, which for this - factor's pre-registered sign of -1 becomes ``-gross + cost`` — costs are added back - rather than deducted, so those fields are UNRELIABLE for a negative-sign factor. The - frozen layer is not patched; the correct, sign-agnostic net spreads are reported - instead. - """ - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - coverage = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - ret_risk = _section_payload(report, "return_risk") - stability = _section_payload(report, "stability_cost") - autocorr = dict(stability.get("factor_rank_autocorr_by_lag", {}) or {}) - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": coverage.get("settled_rebalances"), - "effective_samples": coverage.get("effective_samples"), - "span_days": coverage.get("span_days"), - "cross_section_size_mean": coverage.get("cross_section_size_mean"), - "cross_section_size_median": coverage.get("cross_section_size_median"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - "monotonicity_spearman": ret_risk.get("monotonicity_spearman"), - "gross_long_short_mean": ret_risk.get("gross_long_short_mean"), - # The correct tradability read for a negative-sign factor (see the docstring). - "net_long_short_by_cost": dict(ret_risk.get("net_long_short_by_cost", {}) or {}), - "long_short_turnover": stability.get("turnover_mean_long_short_legs"), - "rank_autocorr_lag1": autocorr.get(1), - "half_life_periods": stability.get("half_life_periods"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, so - ``winsorize`` is declared None (nothing was clipped) even though the config may toggle - it. z-score + industry/size neutralization ARE applied, so they are declared. - ``oos_split`` (from the config's ``oos`` block) makes the OOS section run so the - Predictive axis can be assessed. ``is_exploratory=True``: this is a reproduction on a - shorter window / narrower neutralization than the report, not a return claim (it caps - the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-ridge-minute-return requires an 'oos' section (split_date) so the " - "Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the run's - # prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class RidgeMinuteReturnEvalResult: - """Immutable summary of one run-eval-ridge-minute-return run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - ridge_coverage: RidgeReturnCoverage - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-ridge-minute-return needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-ridge-minute-return needs data.cache.enabled=true (it reads the " - "persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-ridge-minute-return needs universe.type='index' (PIT membership, " - f"e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-ridge-minute-return expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_ridge_minute_return(config_path: str) -> RidgeMinuteReturnEvalResult: - """Run the two real ridge-minute-return evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = RidgeMinuteReturnFactor(lookback_days=RIDGE_RETURN_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Ridge-minute-return factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_ridge_minute_return_panel( - cfg, symbols, spec, logger, - lookback_days=RIDGE_RETURN_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - min_ridge_bars=RIDGE_RETURN_MIN_RIDGE_BARS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on the - # daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - # The tradability read for a NEGATIVE-sign factor: aligned_spread_* is unreliable here - # (frozen-layer defect, see the module docstring), so log the net spreads explicitly. - logger.info( - "net long-short by cost (aligned_spread_* is UNRELIABLE for sign=-1): %s", - no_book_metrics["net_long_short_by_cost"], - ) - - return RidgeMinuteReturnEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - ridge_coverage=load.ridge_coverage, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "RidgeMinuteReturnEvalResult", - "RidgeReturnCoverage", - "evaluate_two_runs", - "extract_metrics", - "run_eval_ridge_minute_return", - "summarize_ridge_return_coverage", -] diff --git a/qt/eval_valley_price_quantile.py b/qt/eval_valley_price_quantile.py deleted file mode 100644 index 4703e7c..0000000 --- a/qt/eval_valley_price_quantile.py +++ /dev/null @@ -1,640 +0,0 @@ -"""run-eval-valley-price-quantile: the tenth real factor evaluation (PR-L). - -Reproduces the SIXTH factor of the Kaiyuan market-microstructure series #27 (开源证券 -《高频成交量的峰、岭、谷信息——市场微观结构研究系列（27）》, reportId 4957417, §5) — the -"量谷加权价格分位点" factor — as a first-class -:class:`~factors.compute.intraday_derived.ValleyPriceQuantileFactor` and runs it through -the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C .. PR-K used. - -WHY THIS FACTOR, NOW: the five prior reproductions from this report covered a COUNT (PR-F -volume_peak_count, weak), a TIMING moment (PR-H peak_interval_kurtosis, null), two price -RATIOS (PR-I valley_relative_vwap and PR-J valley_ridge_vwap_ratio, both passing) and a -RETURN (PR-K ridge_minute_return, sign transferred but Reject). This factor is a price -POSITION — where in the day's range the valley VWAP sits — so it tests whether PR-I/PR-J -succeeded because of the ratio FORM or because of price-level INFORMATION generally. - -SAME MACHINE as PR-F .. PR-K: the minute classification is REUSED from -``data/clean/intraday_volume_prv.py`` (not re-implemented, and not modified by this PR), -so a different verdict here cannot be an artefact of a differently implemented taxonomy. - -TWO STRUCTURAL DIFFERENCES from its siblings, both of which this runner is built around: - - 1. THE PRICE RANGE READS THE PREVIOUS DAY'S CLOSE. Pinned to the last VISIBLE (<=14:50) - raw close of the previous trading day, so every price in the factor comes from ONE - source under ONE visibility rule. That is a DISCLOSED DEVIATION from the report, - which uses the true previous daily close. - 2. THE FACTOR IS REVERSAL-NEUTRALIZED, which makes it CROSS-SECTIONAL. Like PR-G, the - runner streams one symbol at a time through ``compute_valley_price_quantile_stats`` - (memory-bounded), assembles the full-universe raw panel, and then residualizes ONCE - against the 20-day reversal. The reversal is taken at T-1 — - ``rev20 = -(close_{d-1}/close_{d-21} - 1)`` — because the report's naive ``close_d`` - form is 15:00 information and WOULD be a lookahead at our 14:50 decision. Those - closes come from the FRONT-ADJUSTED daily panel this runner already loads; NO new - data source is introduced. - -The runner REPORTS the before/after-neutralization IC exposure and the day-validity / -cross-section coverage, so a neutralization that silently ate the signal (or a -cross-section too thin to regress) shows up as a number rather than being hidden. - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the minute -store has no fetch closure — a miss simply yields no rows); the daily / universe / -covariate endpoints go through the shared read-through cache, which on a fully-warmed -cache does zero gap fetches (disclosed via the run-log cache-stats line). Forward returns -are computed ONLY at the evaluator/analytics boundary from ``ctx.price_panel`` — the -factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the price quantile adds -alpha BEYOND value / low-vol. That check matters more than usual here: a within-range -price POSITION is a plausible cousin of both a value signal and a reversal signal, and -although the factor is already reversal-neutralized, the with-book run is what says -whether it is a genuinely new bet. - -⚠️ SIGN = +1, so this factor is NOT affected by the frozen layer's aligned-spread cost -defect (which mis-signs costs only for negative-sign factors). The ``aligned_spread_*`` -fields may be read directly here — unlike PR-K — though ``net_long_short_by_cost`` remains -the primary tradability read for comparability with its siblings. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_valley_quantile import ( - VALLEY_QUANTILE_LOOKBACK_DAYS, - VALLEY_QUANTILE_MIN_CROSS_SECTION, - VALLEY_QUANTILE_MIN_VALLEY_BARS, - VALLEY_QUANTILE_REVERSAL_DAYS, - compute_valley_price_quantile_stats, - residualize_on_reversal, - reversal_20, -) -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import ValleyPriceQuantileFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -# The neutralization disclosure MOVED to qt.factor_eval_disclosures (the single -# home, D5 C4); re-exported here so historical import paths keep working. -from qt.factor_eval_disclosures import ( - NeutralizationCoverage, - summarize_neutralization, -) -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_valley_price_quantile" -_REPORT_STEM = "eval_valley_price_quantile" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol stats -> ONE cross-sectional residualization) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _ValleyQuantileMinuteLoad: - """Diagnostics from the cache-only per-symbol read + the neutralization.""" - - factor: pd.Series # MultiIndex(date, symbol) RESIDUALIZED factor (the shipped one) - raw: pd.Series # MultiIndex(date, symbol) RAW trailing-mean quantile (pre-neutral) - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite RAW value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - coverage: NeutralizationCoverage - - -def _load_valley_price_quantile_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - panel: pd.DataFrame, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, - min_valley_bars: int, - min_cross_section: int, - reversal_days: int, -) -> _ValleyQuantileMinuteLoad: - """Compute the factor from the minute cache: per-symbol stats then ONE residualization. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily raw - quantile series, and discarded before the next — the multi-year all-symbol minute - panel is NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no - fetch closure, so ``stk_mins`` live calls are provably zero (a symbol with no cached - minute simply yields no rows and is disclosed as empty). - - The REVERSAL comes from ``panel['close']``, which :func:`qt.pipeline._load_panel` has - already FRONT-ADJUSTED — required, because the ratio spans 20 trading days. Taking it - at T-1 is what keeps day d's 15:00 close out of day d's factor value. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_valley_price_quantile_stats( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - min_valley_bars=min_valley_bars, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-valley-price-quantile blocked: no requested symbol produced a " - f"cached valley-price-quantile value over [{cfg.data.start}, {cfg.data.end}]. " - "The minute cache is required (this runner never warms it); check coverage." - ) - raw = pd.concat(series).sort_index() - - # ONE cross-sectional reversal neutralization over the assembled universe panel. - # The closes are the panel's FRONT-ADJUSTED closes -- no new data source. - rev = reversal_20(panel[["close"]], days=reversal_days) - factor = residualize_on_reversal( - raw, rev, min_cross_section=min_cross_section, name=spec.factor_id - ) - coverage = summarize_neutralization( - raw, rev, factor, min_cross_section=min_cross_section - ) - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min rows " - "read, %d raw factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(raw), - ) - logger.info( - "reversal neutralization (T-1 basis, %d-day span): raw=%d rev-paired=%d " - "residual=%d rows; dates %d/%d residualized; cross-section min/median/max=" - "%d/%.1f/%d; mean per-date Spearman(raw, rev20)=%.6f", - reversal_days, coverage.raw_rows, coverage.rev_rows, coverage.residual_rows, - coverage.dates_residualized, coverage.dates_total, - coverage.cross_section_min, coverage.cross_section_median, - coverage.cross_section_max, coverage.raw_rev_spearman_mean, - ) - return _ValleyQuantileMinuteLoad( - factor=factor, - raw=raw, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - coverage=coverage, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price panel - (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as Markdown + - JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 supplies the - book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the research-style - # dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report. - - Surfaces the same comparison quantities PR-K extracted (turnover, net long-short by - cost scenario, lag-1 rank autocorrelation + half-life, cross-section size) so PR-L - sits in one table with PR-I / PR-J / PR-K, PLUS ``ic_pearson_mean`` alongside the rank - ``ic_mean``: the PR-K review found that a divergence between the two predicts the - monotonicity gate failing, and this run is the first test of that regularity on a - POSITIVE-sign factor (the four prior cases were all sign=-1). - - Unlike PR-K, ``aligned_spread_*`` is NOT unreliable here: the frozen layer's - cost-sign defect only mis-signs negative-sign factors, and this factor's pre-registered - sign is +1. ``net_long_short_by_cost`` is still surfaced as the primary read, for - sibling comparability. - """ - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - coverage = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - ret_risk = _section_payload(report, "return_risk") - stability = _section_payload(report, "stability_cost") - autocorr = dict(stability.get("factor_rank_autocorr_by_lag", {}) or {}) - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_pearson_mean": pred.get("ic_pearson_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": coverage.get("settled_rebalances"), - "effective_samples": coverage.get("effective_samples"), - "span_days": coverage.get("span_days"), - "cross_section_size_mean": coverage.get("cross_section_size_mean"), - "cross_section_size_median": coverage.get("cross_section_size_median"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - "monotonicity_spearman": ret_risk.get("monotonicity_spearman"), - "gross_long_short_mean": ret_risk.get("gross_long_short_mean"), - "net_long_short_by_cost": dict(ret_risk.get("net_long_short_by_cost", {}) or {}), - # sign=+1 -> the aligned spreads are NOT mis-signed for this factor. - "aligned_spread_by_cost": dict(ret_risk.get("aligned_spread_by_cost", {}) or {}), - "long_short_turnover": stability.get("turnover_mean_long_short_legs"), - "rank_autocorr_lag1": autocorr.get(1), - "half_life_periods": stability.get("half_life_periods"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, so - ``winsorize`` is declared None (nothing was clipped) even though the config may toggle - it. z-score + industry/size neutralization ARE applied, so they are declared. - ``oos_split`` (from the config's ``oos`` block) makes the OOS section run so the - Predictive axis can be assessed. ``is_exploratory=True``: this is a reproduction on a - shorter window / narrower neutralization than the report, not a return claim (it caps - the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-valley-price-quantile requires an 'oos' section (split_date) so the " - "Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the run's - # prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class ValleyPriceQuantileEvalResult: - """Immutable summary of one run-eval-valley-price-quantile run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - neutralization: NeutralizationCoverage - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-valley-price-quantile needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-valley-price-quantile needs data.cache.enabled=true (it reads the " - "persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-valley-price-quantile needs universe.type='index' (PIT membership, " - f"e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-valley-price-quantile expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_valley_price_quantile(config_path: str) -> ValleyPriceQuantileEvalResult: - """Run the two real valley-price-quantile evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = ValleyPriceQuantileFactor(lookback_days=VALLEY_QUANTILE_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Valley-price-quantile: cache-only per-symbol stats + ONE reversal neutralization. - load = _load_valley_price_quantile_panel( - cfg, symbols, spec, panel, logger, - lookback_days=VALLEY_QUANTILE_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - min_valley_bars=VALLEY_QUANTILE_MIN_VALLEY_BARS, - min_cross_section=VALLEY_QUANTILE_MIN_CROSS_SECTION, - reversal_days=VALLEY_QUANTILE_REVERSAL_DAYS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on the - # daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - logger.info( - "net long-short by cost: %s", no_book_metrics["net_long_short_by_cost"] - ) - # The PR-K review's regularity, tested here on a POSITIVE-sign factor for the first - # time: a divergence between the rank IC and the Pearson IC predicted the monotonicity - # gate failing on all four prior (negative-sign) factors. - logger.info( - "ic_mean(rank)=%s vs ic_pearson_mean=%s; monotonicity_spearman=%s", - no_book_metrics["ic_mean"], no_book_metrics["ic_pearson_mean"], - no_book_metrics["monotonicity_spearman"], - ) - - return ValleyPriceQuantileEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - neutralization=load.coverage, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "NeutralizationCoverage", - "ValleyPriceQuantileEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_valley_price_quantile", - "summarize_neutralization", -] diff --git a/qt/eval_valley_relative_vwap.py b/qt/eval_valley_relative_vwap.py deleted file mode 100644 index 01ba042..0000000 --- a/qt/eval_valley_relative_vwap.py +++ /dev/null @@ -1,540 +0,0 @@ -"""run-eval-valley-relative-vwap: the seventh real factor evaluation (PR-I). - -Reproduces the THIRD factor of the Kaiyuan market-microstructure series #27 (开源证券 -《高频成交量的峰、岭、谷信息——市场微观结构研究系列（27）》, reportId 4957417, §4) — the -"量谷相对加权价格" factor — as a first-class -:class:`~factors.compute.intraday_derived.ValleyRelativeVwapFactor` and runs it through -the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C .. PR-H used. - -This is the THIRD reproduction from the same report, and the first from a DIFFERENT -FAMILY: PR-F counted volume peaks and PR-H measured their timing (both null results); -this factor prices the VALLEYS. The minute classification is REUSED verbatim, so a -different verdict here cannot be an artefact of a differently-implemented taxonomy. - -The factor is a DAILY signal derived DIRECTLY from the 1min cache (see -``data.clean.intraday_valley_vwap.compute_valley_relative_vwap``): PIT-truncate each day -at 14:50, classify every visible minute with the taxonomy REUSED from PR-F -(``data.clean.intraday_volume_prv``: same-slot strictly-prior μ+σ eruptive test; a VALLEY -is a classifiable non-eruptive minute), take each valid day's ratio of the valley VWAP to -the WHOLE VISIBLE DAY's VWAP (both via the Σamount/Σvolume identity), and average that -ratio over the trailing 20 VALID trading days. It is executed CLOSE-TO-CLOSE (daily -default), so ``is_intraday=False`` (the reasoning is documented on the factor's spec). -The eval CELL is identical to PR-C..PR-H, so this run is directly comparable to its -PR-F / PR-H siblings — same classification, different statistic and different family. - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on a -fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats line). -Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the valley-relative -VWAP adds alpha BEYOND value / low-vol. That check matters more than usual here: a -relative PRICE LEVEL is a plausible cousin of a value signal, so the with-book run is -the one that says whether this is a new bet. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_valley_vwap import ( - VALLEY_VWAP_LOOKBACK_DAYS, - VALLEY_VWAP_MIN_VALLEY_BARS, - compute_valley_relative_vwap, -) -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import ValleyRelativeVwapFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_valley_relative_vwap" -_REPORT_STEM = "eval_valley_relative_vwap" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _ValleyVwapMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw valley-relative VWAP - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_valley_relative_vwap_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, - min_valley_bars: int, -) -> _ValleyVwapMinuteLoad: - """Compute the raw valley-relative-VWAP panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is - NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch - closure, so ``stk_mins`` live calls are provably zero (a symbol with no cached - minute simply yields no rows and is disclosed as empty). The classification (reused - from PR-F) and the VWAP-ratio reduction run entirely inside - ``compute_valley_relative_vwap`` on 1min bars, which read ``volume`` and ``amount``. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_valley_relative_vwap( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - min_valley_bars=min_valley_bars, - name=spec.factor_id, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-valley-relative-vwap blocked: no requested symbol produced a " - f"cached valley-relative-VWAP value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms " - "it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - return _ValleyVwapMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, - so ``winsorize`` is declared None (nothing was clipped) even though the config may - toggle it. z-score + industry/size neutralization ARE applied, so they are - declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS section run - so the Predictive axis can be assessed. ``is_exploratory=True``: this is a - reproduction on a shorter window / narrower neutralization than the report, not a - return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-valley-relative-vwap requires an 'oos' section (split_date) so " - "the Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the - # run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class ValleyRelativeVwapEvalResult: - """Immutable summary of one run-eval-valley-relative-vwap run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-valley-relative-vwap needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-valley-relative-vwap needs data.cache.enabled=true (it reads " - "the persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-valley-relative-vwap needs universe.type='index' (PIT " - f"membership, e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-valley-relative-vwap expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_valley_relative_vwap(config_path: str) -> ValleyRelativeVwapEvalResult: - """Run the two real valley-relative-VWAP evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = ValleyRelativeVwapFactor(lookback_days=VALLEY_VWAP_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Valley-relative-VWAP factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_valley_relative_vwap_panel( - cfg, symbols, spec, logger, - lookback_days=VALLEY_VWAP_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - min_valley_bars=VALLEY_VWAP_MIN_VALLEY_BARS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return ValleyRelativeVwapEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "ValleyRelativeVwapEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_valley_relative_vwap", -] diff --git a/qt/eval_valley_ridge_vwap_ratio.py b/qt/eval_valley_ridge_vwap_ratio.py deleted file mode 100644 index 6cecce7..0000000 --- a/qt/eval_valley_ridge_vwap_ratio.py +++ /dev/null @@ -1,573 +0,0 @@ -"""run-eval-valley-ridge-vwap-ratio: the eighth real factor evaluation (PR-J). - -Reproduces the FOURTH factor of the Kaiyuan market-microstructure series #27 (开源证券 -《高频成交量的峰、岭、谷信息——市场微观结构研究系列（27）》, reportId 4957417, §7.1) — the -"谷岭加权价格比" factor — as a first-class -:class:`~factors.compute.intraday_derived.ValleyRidgeVwapRatioFactor` and runs it through -the FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C .. PR-I used. - -This is the FOURTH reproduction from the same report and the direct ROBUSTNESS TEST of -PR-I. PR-I (valley VWAP / whole-visible-day VWAP) is the strongest factor of this loop — -both assessed axes PASS. PR-J keeps everything else fixed and swaps only the DENOMINATOR -to the RIDGE VWAP, contrasting the two behavioural groups head-on. If the price-level -signal survives the swap it is a property of the FAMILY; if it does not, PR-I depended on -its specific denominator. Either outcome is a legitimate result and is reported as such. - -The factor is a DAILY signal derived DIRECTLY from the 1min cache (see -``data.clean.intraday_valley_ridge_vwap.compute_valley_ridge_vwap_ratio``): PIT-truncate -each day at 14:50, classify every visible minute with the taxonomy REUSED from PR-F -(``data.clean.intraday_volume_prv``: same-slot strictly-prior μ+σ eruptive test; a VALLEY -is a classifiable non-eruptive minute, a RIDGE is an eruptive minute that is not an -isolated peak), take each valid day's ratio of the valley VWAP to the ridge VWAP (both -via the Σamount/Σvolume identity), and average that ratio over the trailing 20 VALID -trading days. It is executed CLOSE-TO-CLOSE (daily default), so ``is_intraday=False``. -The eval CELL is identical to PR-C..PR-I, so this run is directly comparable to its -siblings — same classification, same VWAP identity, different denominator. - -RIDGE SCARCITY IS MEASURED, NOT ASSUMED. Ridge bars are structurally far rarer than -valley bars (a minute must erupt AND fail the isolation test), which is why the valid-day -gate asks for >= 10 tradable ridge bars against the valley leg's >= 20. The runner -therefore collects the REALIZED per-day ridge-bar distribution and the day-validity rate -across the whole universe and logs them, so a coverage regression against PR-I is visible -as a number rather than hidden behind a lower threshold. - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on a -fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats line). -Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the ratio adds alpha -BEYOND value / low-vol. As with PR-I that check matters more than usual: a relative PRICE -LEVEL is a plausible cousin of a value signal, so the with-book run is the one that says -whether this is a new bet. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_valley_ridge_vwap import ( - VALLEY_RIDGE_LOOKBACK_DAYS, - VALLEY_RIDGE_MIN_RIDGE_BARS, - VALLEY_RIDGE_MIN_VALLEY_BARS, - compute_valley_ridge_vwap_ratio, -) -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import ValleyRidgeVwapRatioFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -# The ridge-scarcity disclosure MOVED to qt.factor_eval_disclosures (the single -# home, D5 C4); re-exported here so historical import paths keep working. -from qt.factor_eval_disclosures import RidgeCoverage, summarize_ridge_coverage -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_valley_ridge_vwap_ratio" -_REPORT_STEM = "eval_valley_ridge_vwap_ratio" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _ValleyRidgeMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw valley/ridge VWAP ratio - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - ridge_coverage: RidgeCoverage - - -def _load_valley_ridge_vwap_ratio_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, - min_valley_bars: int, - min_ridge_bars: int, -) -> _ValleyRidgeMinuteLoad: - """Compute the raw valley/ridge VWAP-ratio panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is - NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch - closure, so ``stk_mins`` live calls are provably zero (a symbol with no cached - minute simply yields no rows and is disclosed as empty). The classification (reused - from PR-F) and the VWAP-ratio reduction run entirely inside - ``compute_valley_ridge_vwap_ratio`` on 1min bars, which read ``volume`` and - ``amount``. The per-day bar-count diagnostics are collected alongside so the ridge - scarcity can be REPORTED, not assumed; collecting them does not change the factor. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - diagnostics: list[pd.DataFrame] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_valley_ridge_vwap_ratio( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - min_valley_bars=min_valley_bars, - min_ridge_bars=min_ridge_bars, - name=spec.factor_id, - diagnostics_out=diagnostics, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - coverage = summarize_ridge_coverage( - diagnostics, - min_ridge_bars=min_ridge_bars, - min_valley_bars=min_valley_bars, - min_classifiable=min_classifiable, - ) - if not series: - raise ValueError( - "run-eval-valley-ridge-vwap-ratio blocked: no requested symbol produced a " - f"cached valley/ridge VWAP-ratio value over [{cfg.data.start}, " - f"{cfg.data.end}]. The minute cache is required (this runner never warms " - "it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - logger.info("%s", coverage.render()) - return _ValleyRidgeMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ridge_coverage=coverage, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, - so ``winsorize`` is declared None (nothing was clipped) even though the config may - toggle it. z-score + industry/size neutralization ARE applied, so they are - declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS section run - so the Predictive axis can be assessed. ``is_exploratory=True``: this is a - reproduction on a shorter window / narrower neutralization than the report, not a - return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-valley-ridge-vwap-ratio requires an 'oos' section (split_date) so " - "the Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the - # run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class ValleyRidgeVwapRatioEvalResult: - """Immutable summary of one run-eval-valley-ridge-vwap-ratio run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - ridge_coverage: RidgeCoverage - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-valley-ridge-vwap-ratio needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-valley-ridge-vwap-ratio needs data.cache.enabled=true (it reads " - "the persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-valley-ridge-vwap-ratio needs universe.type='index' (PIT " - f"membership, e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-valley-ridge-vwap-ratio expects " - "processing.neutralize.enabled=true (industry + size neutralization, " - "matching the report's neutral column and the EvalConfig declaration)." - ) - - -def run_eval_valley_ridge_vwap_ratio(config_path: str) -> ValleyRidgeVwapRatioEvalResult: - """Run the two real valley/ridge VWAP-ratio evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = ValleyRidgeVwapRatioFactor(lookback_days=VALLEY_RIDGE_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Valley/ridge VWAP-ratio factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_valley_ridge_vwap_ratio_panel( - cfg, symbols, spec, logger, - lookback_days=VALLEY_RIDGE_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - min_valley_bars=VALLEY_RIDGE_MIN_VALLEY_BARS, - min_ridge_bars=VALLEY_RIDGE_MIN_RIDGE_BARS, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return ValleyRidgeVwapRatioEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - ridge_coverage=load.ridge_coverage, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "RidgeCoverage", - "ValleyRidgeVwapRatioEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_valley_ridge_vwap_ratio", - "summarize_ridge_coverage", -] diff --git a/qt/eval_volume_peak_count.py b/qt/eval_volume_peak_count.py deleted file mode 100644 index 43ab2fb..0000000 --- a/qt/eval_volume_peak_count.py +++ /dev/null @@ -1,522 +0,0 @@ -"""run-eval-volume-peak-count: the fourth real factor evaluation (PR-F). - -Reproduces the Kaiyuan market-microstructure series #27 (开源证券《高频成交量的峰、岭、 -谷信息——市场微观结构研究系列（27）》, reportId 4957417) flagship "volume-peak-minute-count" -factor as a first-class -:class:`~factors.compute.intraday_derived.VolumePeakCountFactor` and runs it through the -FROZEN :class:`~analytics.eval.StandardFactorEvaluator` on REAL cached A-share data -(CSI500, PIT membership) — the same contract-driven loop PR-C / PR-D / PR-E used. - -The factor is a DAILY signal derived DIRECTLY from the 1min cache (no coarser resample; -see ``data.clean.intraday_volume_prv.compute_volume_peak_count``): PIT-truncate each day -at 14:50, classify every visible minute against its SAME-SLOT strictly-prior 20-day -baseline (eruptive if ``vol > μ + σ`` else mild), mark the eruptive minutes whose both -1-minute same-session neighbours are mild as PEAKS, and count the peak minutes over the -trailing 20 VALID trading days. It is executed CLOSE-TO-CLOSE (daily default), so -``is_intraday=False`` (the reasoning is documented on the factor's spec). - -CACHE-ONLY: every input is read from the persistent tushare cache -(``artifacts/cache/tushare/v1``). The minute read is provably live-call-free (the -minute store has no fetch closure — a miss simply yields no rows); the daily / -universe / covariate endpoints go through the shared read-through cache, which on a -fully-warmed cache does zero gap fetches (disclosed via the run-log cache-stats line). -Forward returns are computed ONLY at the evaluator/analytics boundary from -``ctx.price_panel`` — the factor computation never sees a future return. - -The evaluator is run TWICE: once with NO known-factor book (the Incremental axis is -NOT_ASSESSED) and once with the project's independently-confirmed book (value_ep / -value_bp / volatility_20) so the Incremental axis measures whether the volume-peak-count -factor adds alpha BEYOND value / low-vol. -""" - -from __future__ import annotations - -import time -from dataclasses import dataclass -from pathlib import Path - -import pandas as pd - -from analytics.eval import ( - EvalConfig, - EvalContext, - FactorEvalReport, - StandardFactorEvaluator, -) -from analytics.eval.figures import render_factor_dashboard -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_cache import READ_COLUMNS -from data.cache.intraday_parquet_store import IntradayParquetStore -from data.clean.intraday_schema import RAW_INTRADAY_FREQ, normalize_intraday_bars -from data.clean.intraday_volume_prv import ( - VOLUME_PRV_BASELINE_DAYS, - VOLUME_PRV_BASELINE_MIN_OBS, - VOLUME_PRV_LOOKBACK_DAYS, - VOLUME_PRV_MIN_CLASSIFIABLE, - VOLUME_PRV_MIN_VALID_DAYS, - VOLUME_PRV_SIGMA_K, - compute_volume_peak_count, -) -from data.clean.schema import CORE_COLUMNS, DATE_LEVEL -from factors.compute.intraday_derived import VolumePeakCountFactor -from factors.spec import FactorSpec -from qt.config import RootConfig, load_config -from qt.exec_basis_eval import ExecBasisEvaluation, run_exec_basis_evaluation -from qt.pipeline import ( - _build_cache, - _build_universe, - _load_panel, - _log_run_cache_stats, - _make_logger, - _maybe_enrich_covariates, - _maybe_enrich_value, - _process_factors, -) - -_LOGGER_NAME = "qt.eval_volume_peak_count" -_REPORT_STEM = "eval_volume_peak_count" - - -# --------------------------------------------------------------------------- # -# Minute loading (cache-only, per-symbol -> memory-bounded) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _VolumePeakMinuteLoad: - """Diagnostics from the cache-only per-symbol minute read + aggregation.""" - - factor: pd.Series # MultiIndex(date, symbol) raw volume-peak-count - requested: int - covered: tuple[str, ...] # symbols that produced >= 1 finite factor value - empty_symbols: tuple[str, ...] # requested but no cached minute / no value - raw_rows: int - live_calls: int # provably 0 (store read has no fetch closure) - - -def _load_volume_peak_count_panel( - cfg: RootConfig, - symbols: list[str], - spec: FactorSpec, - logger, - *, - lookback_days: int, - baseline_days: int, - baseline_min_obs: int, - sigma_k: float, - min_valid_days: int, - min_classifiable: int, -) -> _VolumePeakMinuteLoad: - """Compute the raw volume-peak-count panel per symbol from the minute cache. - - Memory-bounded: one symbol's minute history is read, aggregated to its daily factor - series, and discarded before the next — the multi-year all-symbol minute panel is - NEVER materialized. Read-only: :meth:`IntradayParquetStore.read_range` has no fetch - closure, so ``stk_mins`` live calls are provably zero (a symbol with no cached - minute simply yields no rows and is disclosed as empty). The same-slot baseline / - peak taxonomy runs entirely inside ``compute_volume_peak_count`` on 1min bars. - """ - root = cfg.data.cache.root_dir - store = IntradayParquetStore(root) - start = pd.Timestamp(cfg.data.start).normalize() - end = pd.Timestamp(cfg.data.end).normalize() + pd.Timedelta("23:59:59") - - series: list[pd.Series] = [] - covered: list[str] = [] - empty: list[str] = [] - raw_rows = 0 - for i, sym in enumerate(symbols): - part = store.read_range(INTRADAY_ENDPOINT, sym, RAW_INTRADAY_FREQ, start, end) - if part.empty: - empty.append(sym) - continue - raw_rows += len(part) - bars = normalize_intraday_bars( - part.rename(columns={"bar_end": "time"})[READ_COLUMNS], - freq=RAW_INTRADAY_FREQ, - ) - s = compute_volume_peak_count( - bars, - lookback_days=lookback_days, - baseline_days=baseline_days, - baseline_min_obs=baseline_min_obs, - sigma_k=sigma_k, - min_valid_days=min_valid_days, - min_classifiable=min_classifiable, - name=spec.factor_id, - ) - if s.notna().any(): - series.append(s) - covered.append(sym) - else: - empty.append(sym) - if (i + 1) % 100 == 0: - logger.info( - "minute aggregation: %d/%d symbols processed (%d with a value)", - i + 1, len(symbols), len(covered), - ) - - if not series: - raise ValueError( - "run-eval-volume-peak-count blocked: no requested symbol produced a cached " - f"volume-peak-count value over [{cfg.data.start}, {cfg.data.end}]. The " - "minute cache is required (this runner never warms it); check coverage." - ) - factor = pd.concat(series).sort_index() - logger.info( - "minute aggregation (cache-only): %d/%d symbols with a value, %d raw 1min " - "rows read, %d factor rows, stk_mins_live_calls=0", - len(covered), len(symbols), raw_rows, len(factor), - ) - return _VolumePeakMinuteLoad( - factor=factor, - requested=len(symbols), - covered=tuple(covered), - empty_symbols=tuple(empty), - raw_rows=raw_rows, - live_calls=0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (network-free seam: given panels, run the two evaluations) -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class _RunReports: - """The two evaluation reports (no-book / with-book) + their file paths.""" - - no_book: FactorEvalReport - with_book: FactorEvalReport - no_book_md: Path - no_book_json: Path - with_book_md: Path - with_book_json: Path - no_book_dashboard: Path - with_book_dashboard: Path - - -def evaluate_two_runs( - factor_panel: pd.Series | pd.DataFrame, - spec: FactorSpec, - eval_cfg: EvalConfig, - price_panel: pd.DataFrame, - book: pd.DataFrame, - *, - universe_symbols: tuple[str, ...], - fee_rate: float, - report_dir: Path, - stem: str = _REPORT_STEM, -) -> _RunReports: - """Run the StandardFactorEvaluator TWICE (no book / with book) and write reports. - - This is the network-free seam: given the PROCESSED factor panel, the qfq price - panel (for forward returns), and the PROCESSED known-factor book, it does the two - ``evaluate`` calls and writes ``{stem}_no_book`` / ``{stem}_with_book`` as - Markdown + JSON. Run 1 omits ``known_factors`` (Incremental NOT_ASSESSED); run 2 - supplies the book (Incremental measured). - """ - evaluator = StandardFactorEvaluator() - report_dir.mkdir(parents=True, exist_ok=True) - - ctx_no_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - ) - # evaluate_with_ir yields the SAME report as evaluate() plus the IR the - # research-style dashboard needs (per-period IC + quantile return series). - report_no_book, ir_no_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_no_book - ) - - ctx_with_book = EvalContext( - price_panel=price_panel, - universe_symbols=universe_symbols, - fee_rate=fee_rate, - known_factors=book, - ) - report_with_book, ir_with_book = evaluator.evaluate_with_ir( - factor_panel, spec, eval_cfg, ctx_with_book - ) - - nb_md, nb_json = _write_report(report_no_book, report_dir, f"{stem}_no_book") - wb_md, wb_json = _write_report(report_with_book, report_dir, f"{stem}_with_book") - nb_png = render_factor_dashboard( - report_no_book, ir_no_book, report_dir / f"{stem}_no_book_dashboard.png" - ) - wb_png = render_factor_dashboard( - report_with_book, ir_with_book, report_dir / f"{stem}_with_book_dashboard.png" - ) - return _RunReports( - no_book=report_no_book, - with_book=report_with_book, - no_book_md=nb_md, - no_book_json=nb_json, - with_book_md=wb_md, - with_book_json=wb_json, - no_book_dashboard=nb_png, - with_book_dashboard=wb_png, - ) - - -def _write_report(report: FactorEvalReport, report_dir: Path, stem: str) -> tuple[Path, Path]: - """Write ``report`` as deterministic Markdown + JSON; return the two paths.""" - md_path = report_dir / f"{stem}.md" - json_path = report_dir / f"{stem}.json" - md_path.write_text(report.render(), encoding="utf-8") - json_path.write_text(report.to_json(), encoding="utf-8") - return md_path, json_path - - -# --------------------------------------------------------------------------- # -# Metric extraction (for the CLI line + the handoff) -# --------------------------------------------------------------------------- # -def _section_payload(report: FactorEvalReport, name: str) -> dict: - section = report.by_name().get(name) - return dict(getattr(section, "payload", {}) or {}) - - -def extract_metrics(report: FactorEvalReport) -> dict: - """Pull the headline verdict + gated metrics out of a finished report.""" - verdict = report.require_verdict() - pred = _section_payload(report, "predictive_power") - purity = _section_payload(report, "data_coverage") - incr = _section_payload(report, "purity") - return { - "deployment": verdict.verdict, - "predictive": verdict.predictive.verdict, - "incremental": verdict.incremental.verdict, - "tradable": verdict.tradable.verdict, - "ic_mean": pred.get("ic_mean"), - "ic_ir": pred.get("ic_ir"), - "ic_ir_ci_low": pred.get("ic_ir_ci_low"), - "ic_ir_ci_high": pred.get("ic_ir_ci_high"), - "ic_ir_ci_n_eff": pred.get("ic_ir_ci_n_eff"), - "ic_win_rate": pred.get("ic_win_rate"), - "ic_nw_t": pred.get("ic_nw_t"), - "settled_rebalances": purity.get("settled_rebalances"), - "effective_samples": purity.get("effective_samples"), - "span_days": purity.get("span_days"), - "incremental_ic_ir": incr.get("incremental_ic_ir"), - "incremental_ic_ir_ci_low": incr.get("incremental_ic_ir_ci_low"), - "incremental_ic_ir_ci_high": incr.get("incremental_ic_ir_ci_high"), - "incremental_ic_ir_ci_n_eff": incr.get("incremental_ic_ir_ci_n_eff"), - "incremental_ic_mean": incr.get("incremental_ic_mean"), - } - - -# --------------------------------------------------------------------------- # -# EvalConfig construction (HONEST provenance of what the runner actually did) -# --------------------------------------------------------------------------- # -def _build_eval_config(cfg: RootConfig) -> EvalConfig: - """Build the per-run EvalConfig, declaring EXACTLY what the pipeline applied. - - The declarations must not overstate: this codebase's winsorize step is a P0 no-op, - so ``winsorize`` is declared None (nothing was clipped) even though the config may - toggle it. z-score + industry/size neutralization ARE applied, so they are - declared. ``oos_split`` (from the config's ``oos`` block) makes the OOS section run - so the Predictive axis can be assessed. ``is_exploratory=True``: this is a - reproduction on a shorter window / narrower neutralization than the report, not a - return claim (it caps the deployment label at Watch). - """ - if cfg.oos is None: - raise ValueError( - "run-eval-volume-peak-count requires an 'oos' section (split_date) so the " - "Predictive axis has an out-of-sample split to assess; add e.g. " - "oos: {split_date: '2024-01-01'}." - ) - return EvalConfig( - universe=cfg.universe.index_code or cfg.universe.type, - universe_is_pit=cfg.universe.type == "index", - start=cfg.data.start, - end=cfg.data.end, - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=int(cfg.analytics.quantiles), - cost_scenarios=(1.0, 2.0, 4.0), - oos_split=cfg.oos.split_date, - # winsorize is a P0 no-op in this codebase -> declare None (nothing clipped). - winsorize=None, - standardize="zscore" if cfg.processing.standardize.enabled else None, - neutralization=("industry", "size") if cfg.processing.neutralize.enabled else (), - industry_level=cfg.processing.neutralize.industry_level, - tuned=False, - # We evaluated ONE pre-registered factor whose sign came from the report (not a - # screen of our own); the report's own factor screen is a caveat noted in the - # run's prose, not our multiple-testing background. - n_factors_screened=1, - data_snapshot_id=cfg.data.cache.root_dir, - ) - - -# --------------------------------------------------------------------------- # -# Result container + the full glue -# --------------------------------------------------------------------------- # -@dataclass(frozen=True) -class VolumePeakCountEvalResult: - """Immutable summary of one run-eval-volume-peak-count run.""" - - config: RootConfig - spec: FactorSpec - requested_symbols: int - covered_symbols: int - empty_symbols: int - factor_rows: int - minute_raw_rows: int - minute_live_calls: int - no_book_metrics: dict - with_book_metrics: dict - reports: _RunReports - exec_basis: ExecBasisEvaluation - log_path: Path - elapsed: float - - -def _check_preconditions(cfg: RootConfig) -> None: - """Fail readably if the config cannot drive a real, cache-only CSI500 eval.""" - if cfg.data.source != "tushare": - raise ValueError( - "run-eval-volume-peak-count needs data.source='tushare' (real cached " - f"A-share data); got {cfg.data.source!r}." - ) - if not cfg.data.cache.enabled: - raise ValueError( - "run-eval-volume-peak-count needs data.cache.enabled=true (it reads the " - "persistent tushare cache and never warms live)." - ) - if cfg.universe.type != "index": - raise ValueError( - "run-eval-volume-peak-count needs universe.type='index' (PIT membership, " - f"e.g. 000905.SH for CSI500); got {cfg.universe.type!r}." - ) - if not cfg.processing.neutralize.enabled: - raise ValueError( - "run-eval-volume-peak-count expects processing.neutralize.enabled=true " - "(industry + size neutralization, matching the report's neutral column and " - "the EvalConfig declaration)." - ) - - -def run_eval_volume_peak_count(config_path: str) -> VolumePeakCountEvalResult: - """Run the two real volume-peak-count evaluations (cache-only) + reports.""" - cfg = load_config(config_path) - _check_preconditions(cfg) - - log_path = Path(cfg.output.log_dir) / f"{_REPORT_STEM}.log" - logger = _make_logger(log_path, name=_LOGGER_NAME) - started = time.monotonic() - - factor = VolumePeakCountFactor(lookback_days=VOLUME_PRV_LOOKBACK_DAYS) - spec = factor.spec - eval_cfg = _build_eval_config(cfg) - logger.info( - "eval config: %s rebalance=daily oos_split=%s", eval_cfg.universe, eval_cfg.oos_split - ) - - cache = _build_cache(cfg) - universe, symbols = _build_universe(cfg, logger, cache) - panel = _load_panel(cfg, symbols, logger, cache) - - # Book (value_ep / value_bp / volatility_20): enrich, compute, process. The value - # factors need daily_basic pe/pb; volatility_20 needs close. - book_factors = _build_book_factors() - panel = _maybe_enrich_value(cfg, panel, symbols, book_factors, logger, cache) - panel = _maybe_enrich_covariates(cfg, panel, symbols, logger, cache) - _log_run_cache_stats(cache, logger) # daily/universe/covariate gap-fetches (warm -> 0) - - panel_dates = pd.Index( - pd.unique(panel.index.get_level_values(DATE_LEVEL)), name=DATE_LEVEL - ) - book_raw = pd.concat( - [f.compute(panel).rename(f.name) for f in book_factors], axis=1 - ) - book_processed = _process_factors(cfg, book_raw, panel) - - # Volume-peak-count factor: cache-only per-symbol aggregation -> raw -> process. - load = _load_volume_peak_count_panel( - cfg, symbols, spec, logger, - lookback_days=VOLUME_PRV_LOOKBACK_DAYS, - baseline_days=VOLUME_PRV_BASELINE_DAYS, - baseline_min_obs=VOLUME_PRV_BASELINE_MIN_OBS, - sigma_k=VOLUME_PRV_SIGMA_K, - min_valid_days=VOLUME_PRV_MIN_VALID_DAYS, - min_classifiable=VOLUME_PRV_MIN_CLASSIFIABLE, - ) - # A minute date with no daily bar cannot have a forward return; keep the factor on - # the daily trading grid so the analytics boundary has a price for every date. - factor_raw = load.factor[ - load.factor.index.get_level_values(DATE_LEVEL).isin(panel_dates) - ] - factor_processed = _process_factors(cfg, factor_raw.to_frame(spec.factor_id), panel) - factor_series = factor_processed[spec.factor_id] - - price_panel = panel[CORE_COLUMNS] - reports = evaluate_two_runs( - factor_series, - spec, - eval_cfg, - price_panel, - book_processed, - universe_symbols=tuple(symbols), - fee_rate=float(cfg.cost.fee_rate), - report_dir=Path(cfg.output.report_dir), - ) - - # Second evaluation basis (task card §2.3): the SAME factor values scored on - # the 14:51 VWAP exec-to-exec return instead of close-to-close. The reports - # above are the close_to_close control and are left untouched. - exec_basis = run_exec_basis_evaluation( - factor_series, - spec, - eval_cfg, - book_processed, - cfg=cfg, - panel=panel, - symbols=symbols, - logger=logger, - report_dir=Path(cfg.output.report_dir), - stem=_REPORT_STEM, - ) - - no_book_metrics = extract_metrics(reports.no_book) - with_book_metrics = extract_metrics(reports.with_book) - logger.info( - "verdict no-book: %s (predictive=%s); with-book: %s (incremental=%s)", - no_book_metrics["deployment"], no_book_metrics["predictive"], - with_book_metrics["deployment"], with_book_metrics["incremental"], - ) - - return VolumePeakCountEvalResult( - config=cfg, - spec=spec, - requested_symbols=load.requested, - covered_symbols=len(load.covered), - empty_symbols=len(load.empty_symbols), - factor_rows=int(len(factor_series)), - minute_raw_rows=load.raw_rows, - minute_live_calls=load.live_calls, - no_book_metrics=no_book_metrics, - with_book_metrics=with_book_metrics, - reports=reports, - exec_basis=exec_basis, - log_path=log_path, - elapsed=time.monotonic() - started, - ) - - -def _build_book_factors() -> list: - """Instantiate the confirmed book (value_ep / value_bp / volatility_20).""" - from factors.compute.candidates import ValueFactor, VolatilityFactor - - return [ - ValueFactor("value_ep"), - ValueFactor("value_bp"), - VolatilityFactor(window=20), - ] - - -__all__ = [ - "VolumePeakCountEvalResult", - "evaluate_two_runs", - "extract_metrics", - "run_eval_volume_peak_count", -] diff --git a/qt/factor_eval_reconcile.py b/qt/factor_eval_reconcile.py index 8c8b1a5..2258559 100644 --- a/qt/factor_eval_reconcile.py +++ b/qt/factor_eval_reconcile.py @@ -203,6 +203,12 @@ class is CEILED at ``(lookback_depth - 1) x |frozen symbols|`` cells FACTORS as REPORT_FACTOR_NAMES, FrozenExecBaseline, ) +from qt.hand_anchor_manifest import ( + DEFAULT_MANIFEST as ANCHOR_RECORD_MANIFEST, + AnchorRecordMismatch, + verify_anchor_record, +) +from qt.panel_freeze import FrozenPanelMismatch, verify_frozen_panel_file # --------------------------------------------------------------------------- # # Constants (each pre-registered in the handoff §3 / catalogue §七/§七之二) @@ -1392,6 +1398,39 @@ def frozen_panel_path(factor_id: str, repo_root: Path) -> Path: return Path(repo_root) / FROZEN_PANELS_DIR / f"{factor_id}.parquet" +def verified_frozen_panel_path( + factor_id: str, repo_root: Path, *, required: bool = True +) -> Path | None: + """``frozen_panel_path`` + the git-tracked hash check, as THE only gateway. + + Every read of a frozen panel in this module goes through here, so "did we + check this one?" cannot differ between read sites. Until D5 C6 nothing + checked them at all: the harness read the parquet directly, and the frozen + directory is demonstrably writable (one file in it was overwritten on + 2026-07-25), so a drifted baseline would have reconciled silently. + + The mismatch is re-raised as ``ReconciliationError`` because that is what the + CLI's error handling knows how to print; a bare RuntimeError would surface + as a traceback. + + THE ONLY GATEWAY: ``frozen_panel_path`` must not be called anywhere else in + this module, and a test enforces that by AST. Otherwise "did we verify this + read?" becomes a per-site question, and the site that forgets is exactly the + one that will not announce itself. + """ + path = frozen_panel_path(factor_id, repo_root) + if not path.exists() and not required: + # The anchors leg tolerates an absent panel (only bounded factors have a + # warmup grid to read). Absence is not drift; a missing file that WAS + # required is caught inside the verifier. + return None + try: + verify_frozen_panel_file(path, factor_id, repo_root=repo_root) + except FrozenPanelMismatch as exc: + raise ReconciliationError(str(exc)) from exc + return path + + # --------------------------------------------------------------------------- # # anchors mode: the service path vs the hand-computed anchor rows # --------------------------------------------------------------------------- # @@ -1463,10 +1502,20 @@ def classify_anchor_row( def load_anchor_rows(factor_id: str, repo_root: Path) -> list[dict]: - """The factor's hand-anchor rows (placeholder rows carry no date/symbol).""" + """The factor's hand-anchor rows (placeholder rows carry no date/symbol). + + The record is verified against its git-tracked manifest BEFORE it is parsed. + It is the one file under ``artifacts/refactor_baseline/`` that no manifest + covered, and also the one that actually got overwritten (2026-07-25), so + "the anchors leg read whatever was on disk" was not hypothetical. + """ path = Path(repo_root) / ANCHORS_JSON if not path.exists(): raise ReconciliationError(f"hand anchors not found: {path}") + try: + verify_anchor_record(path, Path(repo_root) / ANCHOR_RECORD_MANIFEST) + except AnchorRecordMismatch as exc: + raise ReconciliationError(str(exc)) from exc payload = json.loads(path.read_text(encoding="utf-8")) return [ row @@ -1539,7 +1588,7 @@ def run_panels_mode(config_path: str, factor_id: str, repo_root: Path) -> PanelD f"cache-only violated: stk_mins_live_calls={live_calls}. ABORT." ) factor = factor_registry.build(factor_id) - frozen = pd.read_parquet(frozen_panel_path(factor_id, repo_root)) + frozen = pd.read_parquet(verified_frozen_panel_path(factor_id, repo_root)) from factors.materialize import stores_intermediate is_cross = stores_intermediate(factor) @@ -1704,8 +1753,8 @@ def run_anchors_mode(config_path: str, factor_id: str, repo_root: Path) -> Ancho if is_minute and not pooled: # Bounded factors: the same warmup boundary as the panels leg — the # frozen grid's first (lookback_depth - 1) trading dates. - panel_path = frozen_panel_path(factor_id, repo_root) - if panel_path.exists(): + panel_path = verified_frozen_panel_path(factor_id, repo_root, required=False) + if panel_path is not None: grid = pd.Index( pd.unique(pd.to_datetime(pd.read_parquet(panel_path)["date"])) ).sort_values() diff --git a/qt/hand_anchor_manifest.py b/qt/hand_anchor_manifest.py new file mode 100644 index 0000000..50486f6 --- /dev/null +++ b/qt/hand_anchor_manifest.py @@ -0,0 +1,173 @@ +"""Integrity check for the D2 hand-anchor record (``hand_anchors_d2.json``). + +The record is the anchors leg's input: ``factor_eval_reconcile.load_anchor_rows`` +reads its ``frozen14`` rows and compares service values against the hand-computed +ones. Until D5 C6 nothing checked it at all, and it is the ONE file in +``artifacts/refactor_baseline/`` that no manifest covered -- while also being the +one file in there that demonstrably got overwritten (2026-07-25, by a +``qt.hand_anchor_rows`` rerun, which is how the engine comparison was lost). + +Why a plain sha256 rather than the panels' canonical-content hash: the panels are +parquet, where writer metadata makes byte equality too strict to be the +authority. This is JSON written by one writer; its bytes ARE canonical, exactly +as for the frozen exec artifacts. + +Where the expectation lives +--------------------------- +``docs/factors/d5_hand_anchor_record_manifest.json`` -- in git, while the record +itself is gitignored. Same split, same reason, as everywhere else here: whoever +overwrites the record cannot also move the expectation without it appearing in +``git diff``. + +The manifest also states the record's INVENTORY, and the inventory is checked, +not merely written down. That matters because the pinned record is incomplete: +``daily_engine_compared`` is empty and cannot be refilled (the companion was +retired in C6). Pinning an incomplete record is fine. Letting a later reader take +"it is pinned" for "it is complete" is not, so the shortfall is a field in the +manifest and an assertion here rather than a sentence someone might not read. + +RANGE OF THE INVENTORY CHECK -- read this before relying on it +-------------------------------------------------------------- +Measured from the CURRENT pinned record (70 / 20 / 0 / False), not reasoned: + +* it bites wherever the recorded SHAPE moves, which is more than row counts: + the three counts, the KEY SET, and the ``all_ok_frozen14`` verdict flag are + all compared. A record that gains ``daily_engine_compared`` rows, grows an + ``all_ok_daily`` key, or flips that flag is refused even with a matching sha; +* it contributes NOTHING when all five stay identical -- which is what a plain + ``qt.hand_anchor_rows`` rerun produces today. Same seed and same stratified + selection give the same counts and key set; the same inputs give the same + flag; only the values inside move. Such a rerun is REFUSED while the sha is + stale and ACCEPTED the moment the sha is honestly refreshed, so on that path + **the protection is the human reading the git diff, not this check**. + (A rerun whose verdict flag DID flip would be caught -- the exemption is the + unchanged shape, not the act of rerunning.) + +The direction that motivated the check -- ``daily_engine_compared`` silently +going to zero, the 2026-07-25 accident -- is no longer reachable FROM HERE: that +loss already happened, and what is pinned is the state after it. The field is +already on the floor. The check would catch that shape on a record that still +had those rows; this record does not. + +Both facts are stated because either alone misleads. "The inventory is checked" +invites more confidence than it earns on the rerun path; "the inventory is +useless" is false in the other direction. + +A stronger form exists and is deliberately NOT built yet: have +``hand_anchor_rows`` record the ``panels_d2`` canonical hashes it computed +against, so the record is tied to the panels it describes rather than merely +being unchanged. That needs a rerun, and a rerun would overwrite the very bytes +the C5 anchors leg was verified against. It belongs with the follow-up that +re-verifies that leg (design note: C-2/D-4). +""" + +from __future__ import annotations + +import hashlib +import json +from dataclasses import dataclass +from pathlib import Path + +#: Git-tracked expectation for the record. +DEFAULT_MANIFEST = "docs/factors/d5_hand_anchor_record_manifest.json" +#: Manifest schema tag; bumping it invalidates every reader, intentionally. +MANIFEST_SCHEMA = "d5-hand-anchor-record/1" +#: Inventory fields the manifest states and this module re-derives and compares. +INVENTORY_FIELDS = ( + "frozen14_rows", + "daily_pending_engine_rows", + "daily_engine_compared_rows", + "all_ok_frozen14", + # ``top_level_keys`` was STATED in the manifest but never compared, so a + # record could grow a key -- ``all_ok_daily``, say -- and be accepted with a + # refreshed sha, while the manifest's most prominent prose says that key is + # ABSENT. "Pinned is not complete" was resting on prose, and prose stops + # nothing. Counts alone cannot see a key appear or vanish; the key SET can. + "top_level_keys", +) + + +class AnchorRecordMismatch(RuntimeError): + """The hand-anchor record is not the one the git-tracked manifest describes.""" + + +@dataclass(frozen=True) +class AnchorRecordCheck: + """What was verified (no secrets; counts and a hash only).""" + + path: Path + sha256: str + inventory: dict + + +def record_inventory(payload: dict) -> dict: + """The inventory the manifest states, re-derived from the record itself.""" + return { + "frozen14_rows": len(payload.get("frozen14", []) or []), + "daily_pending_engine_rows": len(payload.get("daily_pending_engine", []) or []), + "daily_engine_compared_rows": len(payload.get("daily_engine_compared", []) or []), + "all_ok_frozen14": payload.get("all_ok_frozen14"), + "top_level_keys": sorted(payload), + } + + +def verify_anchor_record( + record_path: Path | str, + manifest_path: Path | str, +) -> AnchorRecordCheck: + """Verify the record's bytes AND its inventory against the git-tracked manifest. + + Raises :class:`AnchorRecordMismatch` on any disagreement. The two halves catch + different things and neither subsumes the other: + + * the sha catches ANY byte change, including the one the inventory cannot see + -- a rerun that keeps every count identical while the values move; + * the inventory catches a manifest refreshed to match a record whose SHAPE + changed: rows gained or lost, a key appearing or vanishing. + + See the module docstring's RANGE section for what this does NOT cover from + the currently pinned record. Short version: on the rerun-plus-refreshed-sha + path the inventory adds nothing, and the reviewed git diff is the guard. + """ + record_path = Path(record_path) + manifest_path = Path(manifest_path) + if not manifest_path.exists(): + raise AnchorRecordMismatch( + f"no hand-anchor manifest at {manifest_path}; the record cannot be " + "verified, and an unverified anchors input is how a drifted record " + "reconciles silently." + ) + manifest = json.loads(manifest_path.read_text(encoding="utf-8")) + if manifest.get("schema") != MANIFEST_SCHEMA: + raise AnchorRecordMismatch( + f"manifest schema {manifest.get('schema')!r} != {MANIFEST_SCHEMA!r}" + ) + if not record_path.exists(): + raise AnchorRecordMismatch(f"hand-anchor record not found: {record_path}") + + payload_bytes = record_path.read_bytes() + actual = hashlib.sha256(payload_bytes).hexdigest() + expected = str(manifest.get("sha256", "")) + if actual != expected: + raise AnchorRecordMismatch( + f"{record_path.name}: sha256 {actual[:16]} != the git-tracked " + f"{expected[:16]}. The record changed since it was pinned. If a " + "`qt.hand_anchor_rows` rerun was intended, update " + f"{manifest_path.name} in the same commit so the change is reviewed " + "-- and note that a rerun does NOT restore daily_engine_compared." + ) + + inventory = record_inventory(json.loads(payload_bytes.decode("utf-8"))) + stated = manifest.get("contents", {}) + for field in INVENTORY_FIELDS: + if field not in stated: + raise AnchorRecordMismatch( + f"{manifest_path.name} states no {field!r}; the inventory is what " + "keeps 'pinned' from being read as 'complete'." + ) + if stated[field] != inventory[field]: + raise AnchorRecordMismatch( + f"{record_path.name}: {field} is {inventory[field]!r} but " + f"{manifest_path.name} states {stated[field]!r}." + ) + return AnchorRecordCheck(path=record_path, sha256=actual, inventory=inventory) diff --git a/qt/hand_anchor_rows.py b/qt/hand_anchor_rows.py index 725cb04..0af20fc 100644 --- a/qt/hand_anchor_rows.py +++ b/qt/hand_anchor_rows.py @@ -151,7 +151,12 @@ def provider(sym: str) -> pd.Series | None: return provider -def run_hand_anchors(out_path: Path) -> int: +def run_hand_anchors(out_path: Path, *, allow_overwrite: bool = False) -> int: + # BEFORE the work, not after it: this run takes minutes, and a refusal that + # arrives at the end teaches people to pass --allow-overwrite by reflex. + H.check_overwrite_allowed( + out_path, H.PRODUCED_RECORD_KEYS, allow_overwrite=allow_overwrite + ) started = time.monotonic() rng = np.random.RandomState(SEED) results: list[dict] = [] @@ -454,6 +459,13 @@ def compare(factor_id, cls, key, hand, note=""): "daily_pending_engine": daily_rows, "all_ok_frozen14": all(r["ok"] for r in results), } + if frozenset(payload) != H.PRODUCED_RECORD_KEYS: + raise RuntimeError( + "payload keys drifted from PRODUCED_RECORD_KEYS " + f"({sorted(frozenset(payload) ^ H.PRODUCED_RECORD_KEYS)}); the " + "overwrite guard reasons about that set BEFORE this point, so a new " + "key must be declared there or it is protected by nothing." + ) out_path.write_text(json.dumps(payload, indent=2), encoding="utf-8") n_bad = sum(1 for r in results if not r["ok"]) print(f"hand anchors (frozen 14): {len(results)} rows, {n_bad} mismatches -> {out_path}") diff --git a/qt/hand_anchors_d2.py b/qt/hand_anchors_d2.py index 8e5732f..e839389 100644 --- a/qt/hand_anchors_d2.py +++ b/qt/hand_anchors_d2.py @@ -6,11 +6,13 @@ that imports the engine inherits the engine's bugs); a runtime guard at the bottom of the module raises if any such module is loaded. The engine side of each comparison is read FROM DISK (the ``panels_d2`` parquet files the D2 -reconciliation wrote) for the 14 frozen factors; the four ops-rewritten daily +reconciliation wrote) for the 14 frozen factors. The four ops-rewritten daily factors without a frozen panel (momentum/reversal/liquidity/overnight_mom) are -compared by the companion ``qt.hand_anchors_engine_values`` (which may import -the engine — the INDEPENDENCE requirement binds the hand computation, not the -comparer). +selected and hand-computed here, but are NO LONGER COMPARED: the companion that +did it, ``qt.hand_anchors_engine_values``, had its engine-side rebuild retired +in D5 C6, so those rows stay in ``daily_pending_engine``. The independence rule +that let the companion import the engine still reads correctly — it binds the +hand computation, not the comparer — it simply has nothing left to license. Stratified sampling (R12): per factor, one row from each APPLICABLE boundary class — (a) the warm-up END (the symbol's first finite value; verified by hand @@ -45,6 +47,7 @@ class — (a) the warm-up END (the symbol's first finite value; verified by hand from __future__ import annotations import argparse +import json import sys from pathlib import Path @@ -75,6 +78,82 @@ class — (a) the warm-up END (the symbol's first finite value; verified by hand ) +#: The keys ``qt.hand_anchor_rows`` writes. Declared here so the overwrite +#: refusal can run BEFORE the ~7 minutes of hand computation instead of after +#: it -- being told "refused" at the end of a long run is how people learn to +#: pass the override reflexively. The writer asserts its payload equals this +#: set, so the constant cannot drift from what is actually produced. +PRODUCED_RECORD_KEYS = frozenset({ + "seed", + "tolerance", + "elapsed_seconds", + "frozen14", + "daily_pending_engine", + "all_ok_frozen14", +}) + + +class RecordOverwriteRefused(RuntimeError): + """Refusing to overwrite an existing hand-anchor record without intent.""" + + +def check_overwrite_allowed( + out_path: Path, produced_keys: frozenset[str], *, allow_overwrite: bool +) -> None: + """Refuse to clobber an existing record unless the caller says so. + + This tool rewrites the record WHOLESALE, and the keys it produces are not + all the keys the file can hold. On 2026-07-25 that is exactly how the + engine-side comparison was destroyed: a rerun replaced a file containing + ``daily_engine_compared`` with one that does not, silently, and the + companion that could refill it has since been retired. + + So the refusal is not "you may not write" -- the tool must be able to do its + job -- it is "say so, and here is what this run will not put back". The keys + at risk are computed by DIFFERENCE against what this run actually produces, + not from a hard-coded list, so a future key gets the same protection without + anyone remembering to add it. + + Deliberately NOT a merge: carrying the old ``daily_engine_compared`` forward + onto a freshly recomputed ``frozen14`` would splice a comparison made against + older engine values into a record describing new ones -- internally + inconsistent, which is worse than honestly missing (lead ruling, D-3 + rejected). + """ + if allow_overwrite or not out_path.exists(): + return + try: + existing = json.loads(out_path.read_text(encoding="utf-8")) + except (OSError, json.JSONDecodeError): + existing = {} + at_risk = sorted(k for k in existing if k not in produced_keys) + if at_risk: + detail = ( + f"it holds {at_risk}, which this run does NOT produce and would " + "therefore drop" + ) + if "daily_engine_compared" in at_risk: + detail += ( + " -- and `daily_engine_compared` CANNOT be rebuilt, because the " + "companion that produced it was retired in D5 C6" + ) + else: + detail = ( + "it holds no key this run would drop, but it is still a full rewrite " + "of a file the anchors reconciliation leg reads. (Its " + "`daily_engine_compared` is ALREADY empty and this run will not " + "restore it; that companion was retired in D5 C6.)" + ) + raise RecordOverwriteRefused( + f"refusing to overwrite the existing hand-anchor record at {out_path}: " + f"{detail}. Re-run with --allow-overwrite if that is intended, or pass " + "--selection to write elsewhere; after an intended " + "overwrite, update docs/factors/d5_hand_anchor_record_manifest.json in " + "the same commit, or the anchors reconciliation leg will refuse the new " + "bytes." + ) + + def pending_engine_line(n_pending: int) -> str: """The run-summary line about daily rows awaiting an engine comparison.""" return f"daily rows pending engine comparison: {n_pending} — {ENGINE_COMPARISON_POINTER}" @@ -718,11 +797,32 @@ def assert_no_engine_imports() -> None: def main(argv: list[str] | None = None) -> int: parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) parser.add_argument("--selection", default=str(OUT_JSON)) + parser.add_argument( + "--allow-overwrite", + action="store_true", + help="overwrite an existing record (it is rewritten wholesale; keys " + "this run does not produce are lost and daily_engine_compared cannot " + "be rebuilt)", + ) args = parser.parse_args(argv) assert_no_engine_imports() from qt.hand_anchor_rows import run_hand_anchors # heavy driver, engine-free - return run_hand_anchors(Path(args.selection)) + # Catch the CANONICAL class, not this module's. Under `python -m + # qt.hand_anchors_d2` this file is executed as ``__main__`` AND imported + # again as ``qt.hand_anchors_d2`` (hand_anchor_rows imports it), so there are + # two distinct class objects with the same name and a bare + # ``except RecordOverwriteRefused`` here catches the wrong one -- the guard + # then surfaced as a raw traceback despite the handler being right there. + from qt.hand_anchors_d2 import RecordOverwriteRefused as _Refused + + try: + return run_hand_anchors( + Path(args.selection), allow_overwrite=args.allow_overwrite + ) + except _Refused as exc: # CLI-003: readable, not a traceback + print(f"ERROR: {exc}", file=sys.stderr) + return 1 if __name__ == "__main__": # pragma: no cover - thin CLI shim diff --git a/qt/panel_freeze.py b/qt/panel_freeze.py index 88ed60f..9ec71c4 100644 --- a/qt/panel_freeze.py +++ b/qt/panel_freeze.py @@ -636,6 +636,68 @@ def _verify_one_panel( ) +class FrozenPanelMismatch(RuntimeError): + """A frozen panel on disk is not the one the git-tracked manifest describes.""" + + +def doc_for_frozen_panel(path: Path | str, repo_root: Path | str | None = None) -> Path: + """Which git-tracked manifest holds the expectation for THIS panel file. + + Two frozen trees carry a panel named ``jump_amount_corr_20.parquet`` and they + are DIFFERENT factor definitions: the D1 baseline holds v1.0 (no 14:50 + truncation) and ``pr_c_cutoff_fix/`` holds v1.1 (truncated). The reconciler's + ``frozen_panel_path()`` routes jump to the PR-C tree, so verification has to + follow the same fork or it would check a panel against the other panel's + hash and fail for the wrong reason. The routing decision is read off the + PATH rather than re-derived from the factor id, so the two cannot drift. + """ + path = Path(path) + root = Path(repo_root) if repo_root is not None else repo_root_for(path.parent) + in_pr_c = PR_C_SUBDIR in path.parts + return root / (PR_C_MANIFEST_DOC if in_pr_c else D1_MANIFEST_DOC) + + +def verify_frozen_panel_file( + path: Path | str, + factor_id: str, + *, + repo_root: Path | str | None = None, + doc_path: Path | str | None = None, +) -> str: + """Check ONE frozen panel against the hash authored in git; return that hash. + + Raises :class:`FrozenPanelMismatch` on any disagreement — a caller cannot + accidentally ignore the result the way it could ignore a returned list. + + ``doc_path`` overrides which manifest supplies the expectation; it exists so + the tests can build a small frozen tree with its own document instead of + being forced through the fourteen real factors. + + Deliberately per-panel rather than a whole-tree sweep. The reconciliation + harness runs per factor, and making it verify fifteen panels to read one + turns the check into something people switch off; a check that is disabled + because it is slow protects nothing. + """ + path = Path(path) + if not path.exists(): + raise FrozenPanelMismatch(f"frozen panel missing from disk: {path}") + doc = Path(doc_path) if doc_path is not None else doc_for_frozen_panel(path, repo_root) + expected = parse_doc_manifest(doc).get(factor_id) + if expected is None: + raise FrozenPanelMismatch( + f"{doc.name} carries no expectation for {factor_id!r}; refusing to " + "reconcile against an unverified panel." + ) + actual = canonical_content_hash(read_frozen_panel(path, factor_id)) + if actual != expected.canonical_sha256: + raise FrozenPanelMismatch( + f"{path.name}: canonical content hash {actual} != the git-tracked " + f"{expected.canonical_sha256} (from {doc.name}). These are NOT the " + "frozen bytes; a reconciliation against them would be meaningless." + ) + return actual + + def verify_all( baseline_root: Path | str = DEFAULT_OUTPUT_ROOT, *, diff --git a/tests/test_eval_amp_marginal_anomaly_vol_runner.py b/tests/test_eval_amp_marginal_anomaly_vol_runner.py deleted file mode 100644 index 63acabe..0000000 --- a/tests/test_eval_amp_marginal_anomaly_vol_runner.py +++ /dev/null @@ -1,244 +0,0 @@ -"""PR-E runner: cache-only minute loader + the two-run evaluation core (no network).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import AmpMarginalAnomalyVolFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_amp_marginal_anomaly_vol import ( - _load_amp_anomaly_vol_panel, - evaluate_two_runs, - extract_metrics, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, closes, amps): - """Build STORED_COLUMNS-shaped 1min rows for one session, ON the 5min grid. - - Bar i sits at ``09:35 + 5*i`` minutes (an on-grid time), so the runner's internal - 1min -> 5min resample gives one clean 5min bar per input bar with - ``high=100*(1+amp_i)``, ``low=100`` (amplitude ``amp_i`` exactly) and - ``close=close_i`` (the within-day return). All bars sit inside the 14:50 PIT window. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:35:00") - rows = [] - for i, (c, a) in enumerate(zip(closes, amps)): - be = base + pd.Timedelta(minutes=5 * i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": c, - "high": 100.0 * (1.0 + a), - "low": 100.0, - "close": c, - "volume": 1.0, - "amount": 1.0, - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -# closes/amps designed so each day has 5 within-day pairs and exactly 2 anomalies -# (the two 0.20 |Δamp| bars) -> a finite value with small gates. -_CLOSES = [100.0, 100.0, 100.0, 110.0, 100.0, 105.0] -_AMPS = [0.02, 0.03, 0.04, 0.24, 0.04, 0.05] - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - # Two symbols with cached minute bars across two trading days; a third symbol has - # NO cached minute at all (must be disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - for day in ("2021-07-01", "2021-07-02"): - store.upsert(INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, day, _CLOSES, _AMPS), KEY_COLS) - - cfg = _min_config(root, "2021-07-01", "2021-07-02") - spec = AmpMarginalAnomalyVolFactor().spec - logger = logging.getLogger("test.amav.loader") - load = _load_amp_anomaly_vol_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, - lookback_days=20, min_pool=4, min_selected=2, sigma_k=1.0, - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # values live only on the covered symbols - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-02") - spec = AmpMarginalAnomalyVolFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_amp_anomaly_vol_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.amav.block"), - lookback_days=20, min_pool=4, min_selected=2, sigma_k=1.0, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="amp_marginal_anomaly_vol_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = AmpMarginalAnomalyVolFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - # the axis is now assessed (not NOT_ASSESSED) and is a valid axis state - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, AmpMarginalAnomalyVolFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_intraday_amp_cut_runner.py b/tests/test_eval_intraday_amp_cut_runner.py deleted file mode 100644 index 1d724fd..0000000 --- a/tests/test_eval_intraday_amp_cut_runner.py +++ /dev/null @@ -1,258 +0,0 @@ -"""PR-G runner: cache-only per-symbol stats loop + cross-sectional combine + two-run eval. - -Network-free: the minute loader is seeded through a temp ``IntradayParquetStore`` (no -fetch closure, so ``stk_mins`` live calls are provably 0), and the two-run evaluation core -uses synthetic processed panels. The distinguishing PR-G behaviour is that the loader -assembles the FULL-UNIVERSE ``(V_mean, V_std)`` panel across the per-symbol loop and then -applies the cross-sectional z-score combine ONCE. -""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import IntradayAmpCutFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_intraday_amp_cut import ( - _load_amp_cut_panel, - evaluate_two_runs, - extract_metrics, -) - -# Within-day close chain for a 6-bar day: bar 0 has no return, bars 1..5 have returns -# [+0.05, -0.02, +0.03, -0.04, +0.01]; TOP-return bar is index 1, BOTTOM is index 4. With -# lam=0.20 and 5 valid bars (k=1), V_day = amp[1] - amp[4]. -_CLOSE_CHAIN = [100.0, 105.0, 102.9, 105.987, 101.74752, 102.7649952] - - -def _stored_amp_rows(sym, day, amps): - """STORED_COLUMNS-shaped 1min rows: low=100, high=100*(1+amp), close on _CLOSE_CHAIN.""" - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, (a, c) in enumerate(zip(amps, _CLOSE_CHAIN)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": c, - "high": 100.0 * (1.0 + a), - "low": 100.0, - "close": c, - "volume": 1.0, - "amount": 1.0, - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -_DAY1 = "2021-07-01" -_DAY2 = "2021-07-02" - -# Each symbol has two valid days with DISTINCT V_day (bottom-bar amp differs day1->day2), -# so V_std > 0, and DISTINCT (V_mean, V_std) across symbols so the day-2 cross-section is -# non-degenerate. V_day = 0.02 - amps[4]. -_SYM_AMPS = { - "AAA.SZ": ([0.10, 0.02, 0.03, 0.04, 0.05, 0.06], [0.10, 0.02, 0.03, 0.04, 0.07, 0.06]), - "BBB.SZ": ([0.10, 0.02, 0.03, 0.04, 0.06, 0.06], [0.10, 0.02, 0.03, 0.04, 0.10, 0.06]), - "CCC.SZ": ([0.10, 0.02, 0.03, 0.04, 0.04, 0.06], [0.10, 0.02, 0.03, 0.04, 0.05, 0.06]), -} - -# Small gates so a 2-day cache produces a finite (V_mean, V_std) pair on day 2 and a -# 3-symbol cross-section is enough to z-score. -_LOAD_KW = dict( - lookback_days=10, lam=0.20, min_day_minutes=5, min_valid_days=2, min_cross_section=3 -) - - -def _seed_symbol(store, sym): - a1, a2 = _SYM_AMPS[sym] - store.upsert(INTRADAY_ENDPOINT, sym, "1min", _stored_amp_rows(sym, _DAY1, a1), KEY_COLS) - store.upsert(INTRADAY_ENDPOINT, sym, "1min", _stored_amp_rows(sym, _DAY2, a2), KEY_COLS) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader (per-symbol stats -> cross-sectional combine) -# --------------------------------------------------------------------------- # -def test_amp_cut_loader_is_cache_only_and_combines_cross_section(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - for sym in ("AAA.SZ", "BBB.SZ", "CCC.SZ"): - _seed_symbol(store, sym) - - cfg = _min_config(root, _DAY1, _DAY2) - spec = IntradayAmpCutFactor().spec - logger = logging.getLogger("test.amp_cut.loader") - load = _load_amp_cut_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ", "DDD.SZ"], spec, logger, **_LOAD_KW - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ", "CCC.SZ"} - assert load.empty_symbols == ("DDD.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - # only day 2 carries a finite (V_mean, V_std) pair (min_valid_days=2) - d2 = pd.Timestamp(_DAY2) - assert set(load.factor.index.get_level_values("date")) == {d2} - assert load.factor.notna().all() - # the cross-sectional z-scores centre at 0 on the date - assert load.factor.loc[d2].sum() == pytest.approx(0.0, abs=1e-9) - # the assembled stats panel had day1 (NaN) + day2 rows for all three symbols - assert load.stats_rows == 6 - - -def test_amp_cut_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", _DAY1, _DAY2) - spec = IntradayAmpCutFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_amp_cut_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.amp_cut.block"), **_LOAD_KW - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="intraday_amp_cut_10" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_amp_cut_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = IntradayAmpCutFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None - - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_amp_cut_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee: with no book + no execution facts, at most Watch. - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, IntradayAmpCutFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_jump_amount_corr_runner.py b/tests/test_eval_jump_amount_corr_runner.py deleted file mode 100644 index 270f75c..0000000 --- a/tests/test_eval_jump_amount_corr_runner.py +++ /dev/null @@ -1,232 +0,0 @@ -"""PR-C runner: cache-only minute loader + the two-run evaluation core (no network).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import JumpAmountCorrFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_jump_amount_corr import ( - _load_jump_factor_panel, - evaluate_two_runs, - extract_metrics, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, amps, amounts): - """Build STORED_COLUMNS-shaped 1min rows (open=100, amplitude a) for one session.""" - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, (a, amt) in enumerate(zip(amps, amounts)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": 100.0, - "high": 100.0 + a * 50.0, - "low": 100.0 - a * 50.0, - "close": 100.0, - "volume": 1.0, - "amount": amt, - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - amps = [0.001, 0.02, 0.001, 0.001, 0.02, 0.001] - amts = [10.0, 20.0, 30.0, 40.0, 50.0, 60.0] - # Two symbols with cached minute bars across two trading days; a third symbol - # has NO cached minute at all (must be disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - for day in ("2021-07-01", "2021-07-02"): - store.upsert(INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, day, amps, amts), KEY_COLS) - - cfg = _min_config(root, "2021-07-01", "2021-07-02") - spec = JumpAmountCorrFactor().spec - logger = logging.getLogger("test.jac.loader") - load = _load_jump_factor_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, - lookback_days=20, min_pairs=2, - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # values live only on the covered symbols - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-02") - spec = JumpAmountCorrFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_jump_factor_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.jac.block"), - lookback_days=20, min_pairs=2, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series(rng.standard_normal(len(idx)), index=idx, name="jump_amount_corr_20") - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = JumpAmountCorrFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - # the axis is now assessed (not NOT_ASSESSED) and is a valid axis state - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, JumpAmountCorrFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_minute_ideal_amplitude_runner.py b/tests/test_eval_minute_ideal_amplitude_runner.py deleted file mode 100644 index 81bae7e..0000000 --- a/tests/test_eval_minute_ideal_amplitude_runner.py +++ /dev/null @@ -1,239 +0,0 @@ -"""PR-D runner: cache-only minute loader + the two-run evaluation core (no network).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import MinuteIdealAmplitudeFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_minute_ideal_amplitude import ( - _load_minute_ideal_amp_panel, - evaluate_two_runs, - extract_metrics, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, closes, amps): - """Build STORED_COLUMNS-shaped 1min rows for one session. - - Minute i: ``low=100``, ``high=100*(1+amp_i)`` (amplitude ``amp_i`` exactly), - ``close=close_i`` (controls the price rank). Bars start at 09:31 so all sit - inside the 14:50 PIT window. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, (c, a) in enumerate(zip(closes, amps)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": c, - "high": 100.0 * (1.0 + a), - "low": 100.0, - "close": c, - "volume": 1.0, - "amount": 1.0, - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - closes = [100.0, 101.0, 102.0, 103.0, 104.0, 105.0] - amps = [0.01, 0.02, 0.03, 0.04, 0.05, 0.06] - # Two symbols with cached minute bars across two trading days; a third symbol has - # NO cached minute at all (must be disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - for day in ("2021-07-01", "2021-07-02"): - store.upsert(INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, day, closes, amps), KEY_COLS) - - cfg = _min_config(root, "2021-07-01", "2021-07-02") - spec = MinuteIdealAmplitudeFactor().spec - logger = logging.getLogger("test.mia.loader") - load = _load_minute_ideal_amp_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, - lookback_days=20, lam=0.25, min_minutes=4, - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # values live only on the covered symbols - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-02") - spec = MinuteIdealAmplitudeFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_minute_ideal_amp_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.mia.block"), - lookback_days=20, lam=0.25, min_minutes=4, - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="minute_ideal_amp_10" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = MinuteIdealAmplitudeFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - # the axis is now assessed (not NOT_ASSESSED) and is a valid axis state - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, MinuteIdealAmplitudeFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_peak_interval_kurtosis_runner.py b/tests/test_eval_peak_interval_kurtosis_runner.py deleted file mode 100644 index db220d6..0000000 --- a/tests/test_eval_peak_interval_kurtosis_runner.py +++ /dev/null @@ -1,268 +0,0 @@ -"""PR-H runner: cache-only minute loader + the two-run evaluation core (no network).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import PeakIntervalKurtosisFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_peak_interval_kurtosis import ( - _load_peak_interval_kurtosis_panel, - evaluate_two_runs, - extract_metrics, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, vols): - """Build STORED_COLUMNS-shaped 1min rows for one session of CONSECUTIVE minutes. - - Bar i sits at ``09:31 + i`` minutes (all inside the 14:50 PIT window); consecutive - minutes are 60s apart so interior bars have both 1-minute neighbours. OHLC are dummy - constants (the factor reads only ``volume``); ``amount`` mirrors ``volume``. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, v in enumerate(vols): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": 100.0, - "high": 100.0, - "low": 100.0, - "close": 100.0, - "volume": float(v), - "amount": float(v), - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -# Three constant background days (baseline), then a test day whose five peaks (the -# 200-volume minutes at positions 1, 3, 5, 7, 15) give intervals [2, 2, 2, 8] -> the -# hand-computed excess kurtosis 4.0. -_DAYS = ("2021-07-01", "2021-07-02", "2021-07-03") -_TEST_DAY = "2021-07-04" -_N_SLOTS = 17 -_BG_VOLS = [100.0] * _N_SLOTS - - -def _peak_vols(): - vols = [100.0] * _N_SLOTS - for p in (1, 3, 5, 7, 15): - vols[p] = 200.0 - return vols - - -# Small gates so a 4-day cache produces a value (baseline needs >= 2 prior obs; the pool -# holds the 4 intervals of the single test day). -_LOAD_KW = dict( - lookback_days=20, baseline_days=20, baseline_min_obs=2, - sigma_k=1.0, min_valid_days=1, min_classifiable=1, min_intervals=4, -) - - -def _seed_symbol(store, sym): - for day in _DAYS: - store.upsert(INTRADAY_ENDPOINT, sym, "1min", _stored_rows(sym, day, _BG_VOLS), KEY_COLS) - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", _stored_rows(sym, _TEST_DAY, _peak_vols()), KEY_COLS - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - # Two symbols with cached minute bars; a third has NO cached minute (must be - # disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - _seed_symbol(store, sym) - - cfg = _min_config(root, "2021-07-01", "2021-07-04") - spec = PeakIntervalKurtosisFactor().spec - logger = logging.getLogger("test.pik.loader") - load = _load_peak_interval_kurtosis_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, **_LOAD_KW - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # the test day carries the hand-computed kurtosis of intervals [2, 2, 2, 8] - d = pd.Timestamp(_TEST_DAY) - assert load.factor.loc[(d, "AAA.SZ")] == pytest.approx(4.0) - assert load.factor.loc[(d, "BBB.SZ")] == pytest.approx(4.0) - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-04") - spec = PeakIntervalKurtosisFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_peak_interval_kurtosis_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.pik.block"), **_LOAD_KW - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="peak_interval_kurtosis_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = PeakIntervalKurtosisFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - # the axis is now assessed (not NOT_ASSESSED) and is a valid axis state - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, PeakIntervalKurtosisFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_ridge_minute_return_runner.py b/tests/test_eval_ridge_minute_return_runner.py deleted file mode 100644 index e68a30c..0000000 --- a/tests/test_eval_ridge_minute_return_runner.py +++ /dev/null @@ -1,435 +0,0 @@ -"""PR-K runner: cache-only minute loader, ridge coverage, two-run eval (no net).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from data.clean.intraday_ridge_return import RIDGE_RETURN_MIN_RIDGE_BARS -from factors.compute.intraday_derived import RidgeMinuteReturnFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_ridge_minute_return import ( - _load_ridge_minute_return_panel, - evaluate_two_runs, - extract_metrics, - summarize_ridge_return_coverage, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, closes, vols): - """Build STORED_COLUMNS-shaped 1min rows for one session of CONSECUTIVE minutes. - - Bar i sits at ``09:31 + i`` minutes (all inside the 14:50 PIT window). ``volume`` - drives the classification and ``close`` -- set INDEPENDENTLY -- drives the returns. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, (c, v) in enumerate(zip(closes, vols)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": float(c), - "high": float(c), - "low": float(c), - "close": float(c), - "volume": float(v), - "amount": float(c) * float(v), - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -# Three constant background days (baseline mu=100, sigma=0 -> eruptive threshold 100), then -# the hand-computed CASE A test day: 16 slots, ridge runs at (3,4) and (8,9) with volume -# 200, an ISOLATED peak at slot 12 (volume 300, return +9.0, excluded), and closes chosen so -# the four ridge returns are +0.5 / -0.5 / +3.0 / -0.5 -> daily SUM = +2.5 (a compounded -# reading would give -0.5 instead). -_DAYS = ("2021-07-01", "2021-07-02", "2021-07-03") -_TEST_DAY = "2021-07-04" -_N_SLOTS = 16 -_RIDGES = (3, 4, 8, 9) -_PEAK = 12 -_CASE_A_SUM = 2.5 -_CASE_A_CLOSES = [ - 100.0, 100.0, 100.0, 150.0, 75.0, 75.0, 75.0, 75.0, - 300.0, 150.0, 150.0, 150.0, 1500.0, 150.0, 150.0, 150.0, -] -_BG_CLOSES = [100.0] * _N_SLOTS -_BG_VOLS = [100.0] * _N_SLOTS - - -def _case_a(): - vols = [100.0] * _N_SLOTS - for s in _RIDGES: - vols[s] = 200.0 - vols[_PEAK] = 300.0 - return list(_CASE_A_CLOSES), vols - - -# Small gates so a 4-day cache produces a value (baseline needs >= 2 prior obs; the test day -# carries 4 return-bearing ridge bars). lookback_days=1 so the test day's value IS its own -# daily sum -- the background days have no eruption at all, hence no ridge, hence no valid -# day, so they cannot accumulate in. -_LOAD_KW = dict( - lookback_days=1, baseline_days=20, baseline_min_obs=2, - sigma_k=1.0, min_valid_days=1, min_classifiable=1, min_ridge_bars=1, -) - - -def _seed_symbol(store, sym): - for day in _DAYS: - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, day, _BG_CLOSES, _BG_VOLS), KEY_COLS, - ) - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, _TEST_DAY, *_case_a()), KEY_COLS, - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - # Two symbols with cached minute bars; a third has NO cached minute (must be disclosed - # as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - _seed_symbol(store, sym) - - cfg = _min_config(root, "2021-07-01", "2021-07-04") - spec = RidgeMinuteReturnFactor().spec - logger = logging.getLogger("test.rmr.loader") - load = _load_ridge_minute_return_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, **_LOAD_KW - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # the test day carries the hand-computed summed ridge-minute return - d = pd.Timestamp(_TEST_DAY) - assert load.factor.loc[(d, "AAA.SZ")] == pytest.approx(_CASE_A_SUM) - assert load.factor.loc[(d, "BBB.SZ")] == pytest.approx(_CASE_A_SUM) - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - # a day where NOTHING erupts has no ridge at all -> invalid -> no value emitted - bg = pd.Timestamp(_DAYS[-1]) - assert (bg, "AAA.SZ") not in load.factor.index - - -def test_minute_loader_reports_the_ridge_scarcity_distribution(tmp_path): - """The ridge distribution + validity rate must be MEASURED on the real days.""" - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - _seed_symbol(store, "AAA.SZ") - cfg = _min_config(root, "2021-07-01", "2021-07-04") - load = _load_ridge_minute_return_panel( - cfg, ["AAA.SZ"], RidgeMinuteReturnFactor().spec, - logging.getLogger("test.rmr.cov"), **_LOAD_KW, - ) - cov = load.ridge_coverage - # four symbol-days seen, but the first two have no same-slot baseline yet (it needs 2 - # prior observations) -> only two are CLASSIFIABLE, and of those only the test day has a - # ridge at all -> validity rate 1/2 over the days that had a fair chance. - assert cov.symbol_days == 4 - assert cov.classifiable_days == 2 - assert cov.days_below_classifiable_gate == 2 # the warm-up days, counted separately - assert cov.valid_days == 1 - assert cov.validity_rate == pytest.approx(0.5) - # the flat background day has zero ridge bars, the test day has four; warm-up days are - # excluded from the distribution so they cannot drag it towards zero - assert dict(cov.ridge_return_percentiles)[100] == pytest.approx(4.0) - assert dict(cov.ridge_return_percentiles)[0] == pytest.approx(0.0) - assert cov.ridge_return_mean == pytest.approx(2.0) - # every ridge here sits past the day's first bar, so nothing is lost to the guard - assert cov.ridge_bars_mean == pytest.approx(2.0) - assert cov.return_guard_attrition == pytest.approx(0.0) - # the floors reported are the ones this RUN applied (_LOAD_KW uses 1, not the module - # defaults) -- otherwise the disclosure would describe gates nobody enforced - assert cov.min_ridge_bars == 1 - assert cov.days_below_ridge_gate == 1 # only the flat day (0 ridges) is below 1 - line = cov.render() - assert "ridge scarcity" in line and "valid_days=1" in line - assert "below_ridge_gate(1)" in line - - -def test_coverage_reports_the_return_guard_attrition(): - """Ridge bars lost to the within-day lag must be VISIBLE, not silently absorbed.""" - diag = pd.DataFrame( - { - "classifiable_bars": [240, 240], - "ridge_bars": [10, 30], - # one ridge on each day is the day's first visible bar -> no return - "ridge_return_bars": [9, 29], - "valid": [False, True], - }, - index=pd.DatetimeIndex(pd.bdate_range("2022-01-03", periods=2), name="trade_date"), - ) - cov = summarize_ridge_return_coverage([diag]) - assert cov.ridge_bars_mean == pytest.approx(20.0) - assert cov.ridge_return_mean == pytest.approx(19.0) - assert cov.return_guard_attrition == pytest.approx(1.0 - 19.0 / 20.0) - # the gate is applied to the RETURN-carrying count, so the 9-ridge day falls below 10 - assert cov.days_below_ridge_gate == 1 - assert "return_guard_attrition" in cov.render() - - -def test_summarize_ridge_coverage_counterfactual_at_the_comparison_floor(): - """The disclosure quantifies exactly what the scarcity floor buys (vs PR-J's 20).""" - diag = pd.DataFrame( - { - "classifiable_bars": [240, 240, 240, 240], - "ridge_bars": [4, 12, 25, 30], - "ridge_return_bars": [4, 12, 25, 30], - # as the factor would mark them under the default floor of 10 - "valid": [False, True, True, True], - }, - index=pd.DatetimeIndex(pd.bdate_range("2022-01-03", periods=4), name="trade_date"), - ) - cov = summarize_ridge_return_coverage([diag]) - assert cov.symbol_days == 4 - assert cov.classifiable_days == 4 # every day clears PR-F's classifiable floor - assert cov.valid_days == 3 - # raising the ridge floor to PR-J's 20 would keep only the 25 / 30 days - assert cov.valid_days_at_comparison_floor == 2 - assert cov.days_below_ridge_gate == 1 - assert cov.ridge_return_mean == pytest.approx((4 + 12 + 25 + 30) / 4) - # defaults are the PINNED production floors when the caller does not override - assert cov.min_ridge_bars == RIDGE_RETURN_MIN_RIDGE_BARS == 10 - assert f"below_ridge_gate({RIDGE_RETURN_MIN_RIDGE_BARS})" in cov.render() - - -def test_summarize_ridge_coverage_handles_no_frames(): - cov = summarize_ridge_return_coverage([]) - assert cov.symbol_days == 0 - assert cov.classifiable_days == 0 - assert cov.valid_days == 0 - assert np.isnan(cov.validity_rate) - assert np.isnan(cov.return_guard_attrition) - assert cov.render() # renders without dividing by zero - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-04") - spec = RidgeMinuteReturnFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_ridge_minute_return_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.rmr.block"), **_LOAD_KW - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="ridge_minute_return_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = RidgeMinuteReturnFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report artifact - # alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_extract_metrics_surfaces_the_pr_k_comparison_quantities(tmp_path): - """PR-K's analysis needs turnover / net-by-cost / autocorr / cross-section size. - - These are the quantities compared head-on with PR-I's and PR-J's for the same cell, so - the runner must extract them rather than leaving them buried in the JSON. - """ - factor, book, price = _synthetic_panels() - reports = evaluate_two_runs( - factor, RidgeMinuteReturnFactor().spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - m = extract_metrics(reports.no_book) - for key in ( - "long_short_turnover", "rank_autocorr_lag1", "half_life_periods", - "cross_section_size_mean", "cross_section_size_median", - "monotonicity_spearman", "gross_long_short_mean", - ): - assert key in m, key - assert m[key] is not None, key - # the cost gradient is a per-scenario mapping covering the declared scenarios - net = m["net_long_short_by_cost"] - assert isinstance(net, dict) and net - assert {1.0, 2.0, 4.0} <= {float(k) for k in net} - # higher costs can only reduce a gross-positive spread; the scenarios must not be equal - assert len({round(float(v), 12) for v in net.values()}) > 1 - - -def test_absent_stability_metrics_render_as_na_not_a_crash(): - """A Skipped ``stability_cost`` section must not blow up the CLI summary line. - - The section is Skipped when no period carries a quantile label, which the - scarcity-gated ridge family can plausibly reach on a thin universe. The metrics then - come back as None, and the summary is printed AFTER the reports are already on disk - and OUTSIDE the command's error handling -- so an unguarded ``:.4f`` would surface as - a raw TypeError on an otherwise successful run. - """ - from qt.cli import _fmt_metric - - assert _fmt_metric(None) == "n/a" - assert _fmt_metric(None, ".2f") == "n/a" - assert _fmt_metric(2.108441) == "2.1084" - assert _fmt_metric(8.061448, ".2f") == "8.06" - # the exact expression the summary builds, with every stability field absent - absent = {"long_short_turnover": None, "rank_autocorr_lag1": None, - "half_life_periods": None} - line = ( - f"turnover={_fmt_metric(absent['long_short_turnover'])} " - f"rank_autocorr_lag1={_fmt_metric(absent['rank_autocorr_lag1'])} " - f"half_life={_fmt_metric(absent['half_life_periods'], '.2f')}" - ) - assert line == "turnover=n/a rank_autocorr_lag1=n/a half_life=n/a" - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most Watch — - # regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, RidgeMinuteReturnFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_valley_price_quantile_runner.py b/tests/test_eval_valley_price_quantile_runner.py deleted file mode 100644 index fe25272..0000000 --- a/tests/test_eval_valley_price_quantile_runner.py +++ /dev/null @@ -1,509 +0,0 @@ -"""PR-L runner: cache-only minute loader + reversal neutralization + the two evaluations. - -Network-free throughout: the minute "cache" is a real ``IntradayParquetStore`` seeded in a -tmp_path, and the two-run evaluation seam is exercised on synthetic processed panels. - -The structural novelty this file has to pin down, beyond PR-I..PR-K's loader contract, is -that the loader RESIDUALIZES: it streams per-symbol raw quantiles, then applies ONE -cross-sectional reversal neutralization against the T-1 reversal built from the daily -panel's FRONT-ADJUSTED closes. Tests cover that the shipped factor is the residual (not -the raw quantile), that the daily closes actually reach it, and that the neutralization -coverage is measured rather than assumed. -""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import MANDATORY_SECTIONS, EvalConfig, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from data.clean.intraday_valley_quantile import ( - VALLEY_QUANTILE_MIN_VALLEY_BARS, - reversal_20, -) -from factors.compute.intraday_derived import ValleyPriceQuantileFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_valley_price_quantile import ( - _load_valley_price_quantile_panel, - evaluate_two_runs, - extract_metrics, - summarize_neutralization, -) - - -# --------------------------------------------------------------------------- # -# Minute cache fixtures -# --------------------------------------------------------------------------- # -_N_SLOTS = 16 -_ERUPT = (3, 7) -_HIGH_SLOT, _LOW_SLOT = 5, 9 -_BG_DAYS = ("2021-07-01", "2021-07-02", "2021-07-05") -_TEST_DAY = "2021-07-06" -# Range on the test day: intraday [80, 120], prev_close 100 (inside) -> hi=120, lo=80. -_HI, _LO = 120.0, 80.0 -_PREV_CLOSE = 100.0 -# Per-symbol valley VWAP -> q = (vwap - 80) / 40. -_SYM_VWAP = {"AAA.SZ": 90.0, "BBB.SZ": 100.0, "CCC.SZ": 110.0} -_SYM_Q = {s: (v - _LO) / (_HI - _LO) for s, v in _SYM_VWAP.items()} - - -def _stored_rows(sym, day, *, vols, amts, highs, lows, closes): - """STORED_COLUMNS-shaped 1min rows for one session of CONSECUTIVE minutes. - - Bar i sits at ``09:31 + i`` minutes (all inside the 14:50 PIT window). ``amount`` is - set INDEPENDENTLY of ``volume`` (the per-bar price is amount/volume) and high / low / - close are explicit, because this factor reads the day's range and the previous day's - last visible close alongside the valley VWAP. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i in range(len(vols)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": float(closes[i]), - "high": float(highs[i]), - "low": float(lows[i]), - "close": float(closes[i]), - "volume": float(vols[i]), - "amount": float(amts[i]), - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _background_day(sym, day): - """Flat volume-100 session at price 100 -> baseline mu=100, sigma=0; hi == lo.""" - n = _N_SLOTS - return _stored_rows( - sym, day, - vols=[100.0] * n, amts=[100.0 * _PREV_CLOSE] * n, - highs=[_PREV_CLOSE] * n, lows=[_PREV_CLOSE] * n, closes=[_PREV_CLOSE] * n, - ) - - -def _test_day(sym): - """Engineered day whose valley VWAP is ``_SYM_VWAP[sym]`` inside the range [80, 120].""" - n = _N_SLOTS - vwap = _SYM_VWAP[sym] - vols = [100.0] * n - amts = [100.0 * vwap] * n - for s in _ERUPT: - vols[s] = 200.0 - amts[s] = 200.0 * 500.0 # eruptive bars traded elsewhere; excluded from the VWAP - highs = [110.0] * n - lows = [95.0] * n - highs[_HIGH_SLOT] = _HI - lows[_LOW_SLOT] = _LO - return _stored_rows( - sym, _TEST_DAY, - vols=vols, amts=amts, highs=highs, lows=lows, closes=[100.0] * n, - ) - - -def _seed_symbol(store, sym): - for day in _BG_DAYS: - store.upsert(INTRADAY_ENDPOINT, sym, "1min", _background_day(sym, day), KEY_COLS) - store.upsert(INTRADAY_ENDPOINT, sym, "1min", _test_day(sym), KEY_COLS) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -def _daily_panel(symbols, *, n_days=25, end=_TEST_DAY, seed=5): - """Daily FRONT-ADJUSTED close panel spanning enough history for a T-1 20-day reversal. - - The panel is INDEPENDENT of the minute cache (it is the runner's own daily panel) and - must reach back >= 22 business days before the signal day so ``rev20`` is finite there. - """ - dates = pd.bdate_range(end=pd.Timestamp(end), periods=n_days) - rng = np.random.default_rng(seed) - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - close = 100.0 * np.exp( - np.cumsum(rng.normal(0, 0.02, size=(len(dates), len(symbols))), axis=0) - ) - return pd.DataFrame({"close": close.reshape(-1)}, index=idx) - - -# Small gates so a 4-day cache produces a value. lookback_days=1 so the test day's value IS -# its own daily quantile; the flat background days have hi == lo, hence no valid day, so -# they cannot accumulate in. -_LOAD_KW = dict( - lookback_days=1, baseline_days=20, baseline_min_obs=2, - sigma_k=1.0, min_valid_days=1, min_classifiable=1, min_valley_bars=1, - min_cross_section=3, reversal_days=20, -) - - -# --------------------------------------------------------------------------- # -# Cache-only loader -# --------------------------------------------------------------------------- # -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - for sym in _SYM_VWAP: - _seed_symbol(store, sym) - - syms = sorted(_SYM_VWAP) - cfg = _min_config(root, "2021-07-01", _TEST_DAY) - spec = ValleyPriceQuantileFactor().spec - load = _load_valley_price_quantile_panel( - cfg, syms + ["ZZZ.SZ"], spec, _daily_panel(syms + ["ZZZ.SZ"]), - logging.getLogger("test.vpq.loader"), **_LOAD_KW, - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == set(syms) # all three produced a raw value - assert load.empty_symbols == ("ZZZ.SZ",) # uncovered disclosed, never fetched - assert load.factor.name == spec.factor_id - - # The RAW panel carries the hand-computed daily quantiles. - d = pd.Timestamp(_TEST_DAY) - for sym, q in _SYM_Q.items(): - assert load.raw.loc[(d, sym)] == pytest.approx(q) - - # A flat background day (hi == lo) is invalid -> no value emitted. - assert (pd.Timestamp(_BG_DAYS[-1]), "AAA.SZ") not in load.raw.index - - -def test_loader_ships_the_residual_not_the_raw_quantile(tmp_path): - """The factor the loader returns is the REVERSAL-NEUTRALIZED panel. - - Guards the wiring bug where the neutralization is computed and then discarded. The - residuals of a 3-name cross-section also sum to ~0 by construction of an intercept - OLS fit, which the raw quantiles (0.25 / 0.50 / 0.75) do not. - """ - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - for sym in _SYM_VWAP: - _seed_symbol(store, sym) - syms = sorted(_SYM_VWAP) - - load = _load_valley_price_quantile_panel( - _min_config(root, "2021-07-01", _TEST_DAY), syms, - ValleyPriceQuantileFactor().spec, _daily_panel(syms), - logging.getLogger("test.vpq.resid"), **_LOAD_KW, - ) - d = pd.Timestamp(_TEST_DAY) - shipped = np.array([float(load.factor.loc[(d, s)]) for s in syms]) - raw = np.array([float(load.raw.loc[(d, s)]) for s in syms]) - assert np.isfinite(shipped).all() - assert not np.allclose(shipped, raw) - assert shipped.sum() == pytest.approx(0.0, abs=1e-12) # intercept OLS residual - - -def test_loader_actually_consumes_the_daily_closes(tmp_path): - """A different daily close panel must produce a different factor. - - Without this, the reversal could be silently absent and every test above would still - pass on the raw quantiles. - """ - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - for sym in _SYM_VWAP: - _seed_symbol(store, sym) - syms = sorted(_SYM_VWAP) - cfg = _min_config(root, "2021-07-01", _TEST_DAY) - spec = ValleyPriceQuantileFactor().spec - - a = _load_valley_price_quantile_panel( - cfg, syms, spec, _daily_panel(syms, seed=5), - logging.getLogger("test.vpq.px.a"), **_LOAD_KW, - ) - b = _load_valley_price_quantile_panel( - cfg, syms, spec, _daily_panel(syms, seed=99), - logging.getLogger("test.vpq.px.b"), **_LOAD_KW, - ) - d = pd.Timestamp(_TEST_DAY) - va = [float(a.factor.loc[(d, s)]) for s in syms] - vb = [float(b.factor.loc[(d, s)]) for s in syms] - assert not np.allclose(va, vb) - # ... while the RAW quantiles, which never see a daily close, are identical. - assert [float(a.raw.loc[(d, s)]) for s in syms] == [ - float(b.raw.loc[(d, s)]) for s in syms - ] - - -def test_loader_reports_the_neutralization_coverage(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - for sym in _SYM_VWAP: - _seed_symbol(store, sym) - syms = sorted(_SYM_VWAP) - - load = _load_valley_price_quantile_panel( - _min_config(root, "2021-07-01", _TEST_DAY), syms, - ValleyPriceQuantileFactor().spec, _daily_panel(syms), - logging.getLogger("test.vpq.cov"), **_LOAD_KW, - ) - cov = load.coverage - assert cov.raw_rows == 3 # one valid day x three symbols - assert cov.rev_rows == 3 # all three had a finite T-1 reversal - assert cov.residual_rows == 3 # ... and all three were residualized - assert cov.dates_total == 1 - assert cov.dates_residualized == 1 - assert cov.cross_section_min == 3 and cov.cross_section_max == 3 - assert np.isfinite(cov.raw_rev_spearman_mean) - - -def test_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", _TEST_DAY) - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_valley_price_quantile_panel( - cfg, ["ZZZ.SZ"], ValleyPriceQuantileFactor().spec, - _daily_panel(["ZZZ.SZ"]), logging.getLogger("test.vpq.block"), **_LOAD_KW, - ) - - -def test_thin_cross_section_leaves_the_factor_nan(tmp_path): - """Below ``min_cross_section`` the date cannot be residualized -> NaN, never raw.""" - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - for sym in _SYM_VWAP: - _seed_symbol(store, sym) - syms = sorted(_SYM_VWAP) - - load = _load_valley_price_quantile_panel( - _min_config(root, "2021-07-01", _TEST_DAY), syms, - ValleyPriceQuantileFactor().spec, _daily_panel(syms), - logging.getLogger("test.vpq.thin"), - **dict(_LOAD_KW, min_cross_section=4), - ) - assert not np.isfinite(load.factor.to_numpy(dtype=float)).any() - assert load.raw.notna().any() # the raw panel is unaffected - assert load.coverage.residual_rows == 0 - - -# --------------------------------------------------------------------------- # -# summarize_neutralization -# --------------------------------------------------------------------------- # -def _panel(dates, syms, values, name): - idx = pd.MultiIndex.from_product([dates, syms], names=["date", "symbol"]) - return pd.Series(np.asarray(values, dtype=float).reshape(-1), index=idx, name=name) - - -def test_summarize_neutralization_counts_missing_reversal_rows(): - dates = pd.bdate_range("2023-01-02", periods=2) - syms = ["A", "B", "C"] - raw = _panel(dates, syms, [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]], "q") - rev = _panel(dates, syms, [[1.0, 2.0, 3.0], [1.0, np.nan, np.nan]], "r") - resid = _panel(dates, syms, [[0.0, 0.0, 0.0], [np.nan] * 3], "f") - cov = summarize_neutralization(raw, rev, resid, min_cross_section=3) - assert cov.raw_rows == 6 - assert cov.rev_rows == 4 # only four rows had BOTH - assert cov.residual_rows == 3 - assert cov.dates_total == 2 - assert cov.dates_residualized == 1 - assert cov.cross_section_min == 1 and cov.cross_section_max == 3 - - -def test_summarize_neutralization_handles_an_all_missing_reversal(): - dates = pd.bdate_range("2023-01-02", periods=1) - syms = ["A", "B"] - raw = _panel(dates, syms, [[0.1, 0.2]], "q") - rev = _panel(dates, syms, [[np.nan, np.nan]], "r") - resid = _panel(dates, syms, [[np.nan, np.nan]], "f") - cov = summarize_neutralization(raw, rev, resid, min_cross_section=3) - assert cov.rev_rows == 0 - assert cov.residual_rows == 0 - assert cov.dates_residualized == 0 - assert not np.isfinite(cov.raw_rev_spearman_mean) # no exposure is measurable - - -def test_reversal_from_a_daily_panel_is_the_t_minus_1_ratio(): - """The runner's reversal input is the panel's close column, read at T-1.""" - dates = pd.bdate_range("2023-01-02", periods=25) - panel = pd.DataFrame( - {"close": np.tile(np.arange(100.0, 125.0), 1)}, - index=pd.MultiIndex.from_product([dates, ["A"]], names=["date", "symbol"]), - ) - rev = reversal_20(panel[["close"]], days=20) - d = dates[22] - assert float(rev.loc[(d, "A")]) == pytest.approx(-((121.0 / 101.0) - 1.0)) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="valley_price_quantile_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = ValleyPriceQuantileFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None - - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_extract_metrics_surfaces_the_pr_l_comparison_quantities(tmp_path): - """PR-L is compared head-on with PR-I / PR-J / PR-K on the same cell. - - Beyond the shared tradability quantities, this run needs ``ic_pearson_mean`` next to - the rank ``ic_mean``: the PR-K review found their divergence predicted the - monotonicity gate failing, and PR-L is the first test of that on a sign=+1 factor. - """ - factor, book, price = _synthetic_panels() - reports = evaluate_two_runs( - factor, ValleyPriceQuantileFactor().spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - m = extract_metrics(reports.no_book) - for key in ( - "long_short_turnover", "rank_autocorr_lag1", "half_life_periods", - "cross_section_size_mean", "cross_section_size_median", - "monotonicity_spearman", "gross_long_short_mean", - "ic_mean", "ic_pearson_mean", - ): - assert key in m, key - assert m[key] is not None, key - net = m["net_long_short_by_cost"] - assert isinstance(net, dict) and net - assert {1.0, 2.0, 4.0} <= {float(k) for k in net} - assert len({round(float(v), 12) for v in net.values()}) > 1 - # sign=+1 -> the aligned spreads are NOT mis-signed for this factor and are surfaced. - assert isinstance(m["aligned_spread_by_cost"], dict) - - -def test_spec_sign_is_positive_so_aligned_spread_is_not_mis_signed(): - """The frozen layer's cost-sign defect only bites at sign=-1; PR-L is +1.""" - assert ValleyPriceQuantileFactor().spec.expected_ic_sign == 1 - - -def test_no_book_run_never_reaches_adopt(tmp_path): - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, ValleyPriceQuantileFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED - - -def test_min_valley_bars_default_matches_the_pr_i_floor(): - """PR-L reuses PR-I's valley floor so the two runs' coverage is comparable.""" - from data.clean.intraday_valley_vwap import VALLEY_VWAP_MIN_VALLEY_BARS - - assert VALLEY_QUANTILE_MIN_VALLEY_BARS == VALLEY_VWAP_MIN_VALLEY_BARS diff --git a/tests/test_eval_valley_relative_vwap_runner.py b/tests/test_eval_valley_relative_vwap_runner.py deleted file mode 100644 index d4b0958..0000000 --- a/tests/test_eval_valley_relative_vwap_runner.py +++ /dev/null @@ -1,288 +0,0 @@ -"""PR-I runner: cache-only minute loader + the two-run evaluation core (no network).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import ValleyRelativeVwapFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_valley_relative_vwap import ( - _load_valley_relative_vwap_panel, - evaluate_two_runs, - extract_metrics, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, vols, amts): - """Build STORED_COLUMNS-shaped 1min rows for one session of CONSECUTIVE minutes. - - Bar i sits at ``09:31 + i`` minutes (all inside the 14:50 PIT window). OHLC are dummy - constants; ``volume`` drives the classification and ``amount`` -- set INDEPENDENTLY -- - drives the VWAPs. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, (v, a) in enumerate(zip(vols, amts)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": 100.0, - "high": 100.0, - "low": 100.0, - "close": 100.0, - "volume": float(v), - "amount": float(a), - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -# Three constant background days (baseline mu=100, sigma=0 -> eruptive threshold 100), -# then the hand-computed CASE A test day: 12 slots, eruptions at 3 and 7 (volume 200 / -# amount 4000 -> price 20); the ten valley bars carry volume 100 with amount 1000 (x5) -# and 1200 (x5) -> valley VWAP 11.0, day VWAP 19000/1400, ratio 77/95. -_DAYS = ("2021-07-01", "2021-07-02", "2021-07-03") -_TEST_DAY = "2021-07-04" -_N_SLOTS = 12 -_ERUPT = (3, 7) -_CASE_A_RATIO = 77.0 / 95.0 -_BG_VOLS = [100.0] * _N_SLOTS -_BG_AMTS = [1000.0] * _N_SLOTS - - -def _case_a(): - vols = [100.0] * _N_SLOTS - amts = [0.0] * _N_SLOTS - valley_slots = [s for s in range(_N_SLOTS) if s not in _ERUPT] - for i, s in enumerate(valley_slots): - amts[s] = 1000.0 if i < 5 else 1200.0 - for s in _ERUPT: - vols[s] = 200.0 - amts[s] = 4000.0 - return vols, amts - - -# Small gates so a 4-day cache produces a value (baseline needs >= 2 prior obs; the test -# day carries 10 tradable valley bars). lookback_days=1 so the test day's value IS its own -# ratio: with these tiny gates the last background day also becomes valid, and a day where -# NOTHING erupts has ratio exactly 1.0 (every bar is a valley, so the two VWAPs coincide), -# which would otherwise average into the assertion. -_LOAD_KW = dict( - lookback_days=1, baseline_days=20, baseline_min_obs=2, - sigma_k=1.0, min_valid_days=1, min_classifiable=1, min_valley_bars=1, -) - - -def _seed_symbol(store, sym): - for day in _DAYS: - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, day, _BG_VOLS, _BG_AMTS), KEY_COLS, - ) - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, _TEST_DAY, *_case_a()), KEY_COLS, - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - # Two symbols with cached minute bars; a third has NO cached minute (must be - # disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - _seed_symbol(store, sym) - - cfg = _min_config(root, "2021-07-01", "2021-07-04") - spec = ValleyRelativeVwapFactor().spec - logger = logging.getLogger("test.vrv.loader") - load = _load_valley_relative_vwap_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, **_LOAD_KW - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # the test day carries the hand-computed valley/day VWAP ratio - d = pd.Timestamp(_TEST_DAY) - assert load.factor.loc[(d, "AAA.SZ")] == pytest.approx(_CASE_A_RATIO) - assert load.factor.loc[(d, "BBB.SZ")] == pytest.approx(_CASE_A_RATIO) - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - # the last background day (nothing erupts -> every bar is a valley) has ratio - # exactly 1.0: the two VWAPs are sums over the SAME bar set - bg = pd.Timestamp(_DAYS[-1]) - assert load.factor.loc[(bg, "AAA.SZ")] == pytest.approx(1.0) - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-04") - spec = ValleyRelativeVwapFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_valley_relative_vwap_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.vrv.block"), **_LOAD_KW - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="valley_relative_vwap_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = ValleyRelativeVwapFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - # the axis is now assessed (not NOT_ASSESSED) and is a valid axis state - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, ValleyRelativeVwapFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_valley_ridge_vwap_ratio_runner.py b/tests/test_eval_valley_ridge_vwap_ratio_runner.py deleted file mode 100644 index 79097ad..0000000 --- a/tests/test_eval_valley_ridge_vwap_ratio_runner.py +++ /dev/null @@ -1,365 +0,0 @@ -"""PR-J runner: cache-only minute loader, ridge-scarcity coverage, two-run eval (no net).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from data.clean.intraday_valley_ridge_vwap import ( - VALLEY_RIDGE_MIN_RIDGE_BARS, - VALLEY_RIDGE_MIN_VALLEY_BARS, -) -from factors.compute.intraday_derived import ValleyRidgeVwapRatioFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_valley_ridge_vwap_ratio import ( - _load_valley_ridge_vwap_ratio_panel, - evaluate_two_runs, - extract_metrics, - summarize_ridge_coverage, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, vols, amts): - """Build STORED_COLUMNS-shaped 1min rows for one session of CONSECUTIVE minutes. - - Bar i sits at ``09:31 + i`` minutes (all inside the 14:50 PIT window). OHLC are dummy - constants; ``volume`` drives the classification and ``amount`` -- set INDEPENDENTLY -- - drives the VWAPs. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, (v, a) in enumerate(zip(vols, amts)): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": 100.0, - "high": 100.0, - "low": 100.0, - "close": 100.0, - "volume": float(v), - "amount": float(a), - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -# Three constant background days (baseline mu=100, sigma=0 -> eruptive threshold 100), -# then the hand-computed CASE A test day: 16 slots, ridge runs at (3,4) and (8,9) with -# volume 200 / amount 4000 (price 20), an ISOLATED peak at slot 12 (volume 300 / amount -# 9000 -> price 30, excluded from BOTH legs), and 11 valley bars at volume 100 carrying -# amount 1000 (x6) and 1200 (x5) -> valley VWAP 12000/1100, ridge VWAP 20, ratio 6/11. -_DAYS = ("2021-07-01", "2021-07-02", "2021-07-03") -_TEST_DAY = "2021-07-04" -_N_SLOTS = 16 -_RIDGES = (3, 4, 8, 9) -_PEAK = 12 -_CASE_A_RATIO = 6.0 / 11.0 -_BG_VOLS = [100.0] * _N_SLOTS -_BG_AMTS = [1000.0] * _N_SLOTS - - -def _case_a(): - vols = [100.0] * _N_SLOTS - amts = [0.0] * _N_SLOTS - for s in _RIDGES: - vols[s] = 200.0 - amts[s] = 4000.0 - vols[_PEAK] = 300.0 - amts[_PEAK] = 9000.0 - valley_slots = [s for s in range(_N_SLOTS) if s not in _RIDGES and s != _PEAK] - for i, s in enumerate(valley_slots): - amts[s] = 1000.0 if i < 6 else 1200.0 - return vols, amts - - -# Small gates so a 4-day cache produces a value (baseline needs >= 2 prior obs; the test -# day carries 4 tradable ridge bars and 11 tradable valley bars). lookback_days=1 so the -# test day's value IS its own ratio -- the background days have no eruption at all, hence -# no ridge, hence no valid day, so they cannot average in. -_LOAD_KW = dict( - lookback_days=1, baseline_days=20, baseline_min_obs=2, - sigma_k=1.0, min_valid_days=1, min_classifiable=1, - min_valley_bars=1, min_ridge_bars=1, -) - - -def _seed_symbol(store, sym): - for day in _DAYS: - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, day, _BG_VOLS, _BG_AMTS), KEY_COLS, - ) - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", - _stored_rows(sym, _TEST_DAY, *_case_a()), KEY_COLS, - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - # Two symbols with cached minute bars; a third has NO cached minute (must be - # disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - _seed_symbol(store, sym) - - cfg = _min_config(root, "2021-07-01", "2021-07-04") - spec = ValleyRidgeVwapRatioFactor().spec - logger = logging.getLogger("test.vrr.loader") - load = _load_valley_ridge_vwap_ratio_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, **_LOAD_KW - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # the test day carries the hand-computed valley/ridge VWAP ratio - d = pd.Timestamp(_TEST_DAY) - assert load.factor.loc[(d, "AAA.SZ")] == pytest.approx(_CASE_A_RATIO) - assert load.factor.loc[(d, "BBB.SZ")] == pytest.approx(_CASE_A_RATIO) - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - # a day where NOTHING erupts has no ridge leg at all -> invalid -> no value emitted - bg = pd.Timestamp(_DAYS[-1]) - assert (bg, "AAA.SZ") not in load.factor.index - - -def test_minute_loader_reports_the_ridge_scarcity_distribution(tmp_path): - """The ridge-bar distribution + validity rate must be MEASURED on the real days.""" - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - _seed_symbol(store, "AAA.SZ") - cfg = _min_config(root, "2021-07-01", "2021-07-04") - load = _load_valley_ridge_vwap_ratio_panel( - cfg, ["AAA.SZ"], ValleyRidgeVwapRatioFactor().spec, - logging.getLogger("test.vrr.cov"), **_LOAD_KW, - ) - cov = load.ridge_coverage - # four symbol-days seen, but the first two have no same-slot baseline yet (it needs - # 2 prior observations) -> only two are CLASSIFIABLE, and of those only the test day - # has a ridge at all -> validity rate 1/2 over the days that had a fair chance. - assert cov.symbol_days == 4 - assert cov.classifiable_days == 2 - assert cov.days_below_classifiable_gate == 2 # the warm-up days, counted separately - assert cov.valid_days == 1 - assert cov.validity_rate == pytest.approx(0.5) - # the flat background day has zero ridge bars, the test day has four; warm-up days - # are excluded from the distribution so they cannot drag it towards zero - assert dict(cov.ridge_percentiles)[100] == pytest.approx(4.0) - assert dict(cov.ridge_percentiles)[0] == pytest.approx(0.0) - assert cov.ridge_mean == pytest.approx(2.0) - # the floors reported are the ones this RUN applied (_LOAD_KW uses 1, not the module - # defaults) -- otherwise the disclosure would describe gates nobody enforced - assert cov.min_ridge_bars == 1 - assert cov.min_valley_bars == 1 - assert cov.days_below_ridge_gate == 1 # only the flat day (0 ridges) is below 1 - line = cov.render() - assert "ridge scarcity" in line and "valid_days=1" in line - assert "below_ridge_gate(1)" in line and "below_valley_gate(1)" in line - - -def test_summarize_ridge_coverage_counterfactual_at_the_valley_floor(): - """The disclosure quantifies exactly what the LOWERED ridge floor buys.""" - diag = pd.DataFrame( - { - "classifiable_bars": [240, 240, 240, 240], - "valley_bars": [200, 200, 200, 200], - "ridge_bars": [4, 12, 25, 30], - # as the factor would mark them under the default floor of 10 - "valid": [False, True, True, True], - }, - index=pd.DatetimeIndex(pd.bdate_range("2022-01-03", periods=4), name="trade_date"), - ) - cov = summarize_ridge_coverage([diag]) - assert cov.symbol_days == 4 - assert cov.classifiable_days == 4 # every day clears PR-F's classifiable floor - assert cov.valid_days == 3 - # holding the ridge leg to the VALLEY floor (20) would keep only the 25 / 30 days - assert cov.valid_days_at_valley_floor == 2 - assert cov.days_below_ridge_gate == 1 - assert cov.days_below_valley_gate == 0 - assert cov.ridge_mean == pytest.approx((4 + 12 + 25 + 30) / 4) - # defaults are the PINNED production floors when the caller does not override - assert cov.min_ridge_bars == VALLEY_RIDGE_MIN_RIDGE_BARS == 10 - assert cov.min_valley_bars == VALLEY_RIDGE_MIN_VALLEY_BARS == 20 - assert f"below_ridge_gate({VALLEY_RIDGE_MIN_RIDGE_BARS})" in cov.render() - - -def test_summarize_ridge_coverage_handles_no_frames(): - cov = summarize_ridge_coverage([]) - assert cov.symbol_days == 0 - assert cov.classifiable_days == 0 - assert cov.valid_days == 0 - assert np.isnan(cov.validity_rate) - assert cov.render() # renders without dividing by zero - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-04") - spec = ValleyRidgeVwapRatioFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_valley_ridge_vwap_ratio_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.vrr.block"), **_LOAD_KW - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="valley_ridge_vwap_ratio_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = ValleyRidgeVwapRatioFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, ValleyRidgeVwapRatioFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_eval_volume_peak_count_runner.py b/tests/test_eval_volume_peak_count_runner.py deleted file mode 100644 index 7a7e8de..0000000 --- a/tests/test_eval_volume_peak_count_runner.py +++ /dev/null @@ -1,258 +0,0 @@ -"""PR-F runner: cache-only minute loader + the two-run evaluation core (no network).""" - -from __future__ import annotations - -import logging - -import numpy as np -import pandas as pd -import pytest - -from analytics.eval import EvalConfig, MANDATORY_SECTIONS, Section, Skipped -from analytics.eval.verdict import AXIS_NOT_ASSESSED, AXIS_VERDICTS -from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT -from data.cache.intraday_parquet_store import KEY_COLS, IntradayParquetStore -from factors.compute.intraday_derived import VolumePeakCountFactor -from qt.config import ( - AlphaCfg, - BacktestCfg, - CacheCfg, - CostCfg, - DataCfg, - FactorCfg, - OutputCfg, - PortfolioCfg, - RootConfig, - UniverseCfg, -) -from qt.eval_volume_peak_count import ( - _load_volume_peak_count_panel, - evaluate_two_runs, - extract_metrics, -) - - -# --------------------------------------------------------------------------- # -# Minute cache-only loader -# --------------------------------------------------------------------------- # -def _stored_rows(sym, day, vols): - """Build STORED_COLUMNS-shaped 1min rows for one session of CONSECUTIVE minutes. - - Bar i sits at ``09:31 + i`` minutes (all inside the 14:50 PIT window); consecutive - minutes are 60s apart so interior bars have both 1-minute neighbours. OHLC are dummy - constants (the factor reads only ``volume``); ``amount`` mirrors ``volume``. - """ - base = pd.Timestamp(day) + pd.Timedelta("09:31:00") - rows = [] - for i, v in enumerate(vols): - be = base + pd.Timedelta(minutes=i) - rows.append( - { - "symbol": sym, - "bar_end": be, - "source_trade_time": be, - "open": 100.0, - "high": 100.0, - "low": 100.0, - "close": 100.0, - "volume": float(v), - "amount": float(v), - "freq": "1min", - } - ) - return pd.DataFrame(rows) - - -def _min_config(root, start, end): - return RootConfig( - data=DataCfg( - source="tushare", - start=start, - end=end, - external_secret_file="/nonexistent.json", - cache=CacheCfg(enabled=True, root_dir=str(root)), - ), - universe=UniverseCfg(type="static", symbols=["A.SZ"]), - factors=[FactorCfg(name="momentum_20")], - alpha=AlphaCfg(), - portfolio=PortfolioCfg(top_n=1), - backtest=BacktestCfg(), - cost=CostCfg(), - output=OutputCfg(), - ) - - -# Three constant background days (baseline) then a test day with two clean peaks (the -# 200-volume minutes at 09:32 and 09:35, both with mild neighbours). -_DAYS = ("2021-07-01", "2021-07-02", "2021-07-03") -_TEST_DAY = "2021-07-04" -_BG_VOLS = [100.0] * 6 -_PEAK_VOLS = [100.0, 200.0, 100.0, 100.0, 200.0, 100.0] - -# Small gates so a 4-day cache produces a value (baseline needs >= 2 prior obs). -_LOAD_KW = dict( - lookback_days=20, baseline_days=20, baseline_min_obs=2, - sigma_k=1.0, min_valid_days=1, min_classifiable=1, -) - - -def _seed_symbol(store, sym): - for day in _DAYS: - store.upsert(INTRADAY_ENDPOINT, sym, "1min", _stored_rows(sym, day, _BG_VOLS), KEY_COLS) - store.upsert( - INTRADAY_ENDPOINT, sym, "1min", _stored_rows(sym, _TEST_DAY, _PEAK_VOLS), KEY_COLS - ) - - -def test_minute_loader_is_cache_only_and_discloses_empty(tmp_path): - root = tmp_path / "cache" - store = IntradayParquetStore(str(root)) - # Two symbols with cached minute bars; a third has NO cached minute (must be - # disclosed as empty, never fetched). - for sym in ("AAA.SZ", "BBB.SZ"): - _seed_symbol(store, sym) - - cfg = _min_config(root, "2021-07-01", "2021-07-04") - spec = VolumePeakCountFactor().spec - logger = logging.getLogger("test.vpc.loader") - load = _load_volume_peak_count_panel( - cfg, ["AAA.SZ", "BBB.SZ", "CCC.SZ"], spec, logger, **_LOAD_KW - ) - assert load.live_calls == 0 # store read has no fetch closure - assert set(load.covered) == {"AAA.SZ", "BBB.SZ"} # both produced a value - assert load.empty_symbols == ("CCC.SZ",) # uncovered disclosed, not fetched - assert load.factor.name == spec.factor_id - assert load.factor.notna().any() - # the test day carries the two-peak count (=2) on both covered symbols - d = pd.Timestamp(_TEST_DAY) - assert load.factor.loc[(d, "AAA.SZ")] == pytest.approx(2.0) - assert load.factor.loc[(d, "BBB.SZ")] == pytest.approx(2.0) - assert set(load.factor.index.get_level_values("symbol")) == {"AAA.SZ", "BBB.SZ"} - - -def test_minute_loader_blocks_when_nothing_cached(tmp_path): - cfg = _min_config(tmp_path / "empty", "2021-07-01", "2021-07-04") - spec = VolumePeakCountFactor().spec - with pytest.raises(ValueError, match="no requested symbol produced"): - _load_volume_peak_count_panel( - cfg, ["ZZZ.SZ"], spec, logging.getLogger("test.vpc.block"), **_LOAD_KW - ) - - -# --------------------------------------------------------------------------- # -# Evaluation core (two runs) — synthetic processed panels, no network -# --------------------------------------------------------------------------- # -def _synthetic_panels(n_days=90, n_symbols=15, seed=7): - rng = np.random.default_rng(seed) - dates = pd.bdate_range("2022-01-03", periods=n_days) - symbols = [f"{i:06d}.SZ" for i in range(n_symbols)] - idx = pd.MultiIndex.from_product([dates, symbols], names=["date", "symbol"]) - - # processed (z-scored-ish) subject factor + a 3-column book on the same grid - factor = pd.Series( - rng.standard_normal(len(idx)), index=idx, name="volume_peak_count_20" - ) - book = pd.DataFrame( - { - "value_ep": rng.standard_normal(len(idx)), - "value_bp": rng.standard_normal(len(idx)), - "volatility_20": rng.standard_normal(len(idx)), - }, - index=idx, - ) - # a qfq-style price panel with the CORE_COLUMNS the forward-return boundary needs - close = pd.Series( - 100.0 * np.exp(np.cumsum(0.001 * rng.standard_normal(len(idx)))), index=idx - ) - price = pd.DataFrame( - { - "open": close.to_numpy(), - "high": close.to_numpy() * 1.01, - "low": close.to_numpy() * 0.99, - "close": close.to_numpy(), - "volume": 1_000.0, - "amount": 100_000.0, - "adj_factor": 1.0, - }, - index=idx, - ) - return factor, book, price - - -def _eval_cfg(): - return EvalConfig( - universe="000905.SH", - universe_is_pit=True, - start="2022-01-03", - end="2022-05-10", - is_exploratory=True, - post_hoc_selected=False, - rebalance="daily", - n_quantiles=5, - oos_split="2022-03-15", - winsorize=None, - standardize="zscore", - neutralization=("industry", "size"), - industry_level="L1", - ) - - -def test_evaluate_two_runs_produces_full_reports_and_incremental_axis(tmp_path): - factor, book, price = _synthetic_panels() - spec = VolumePeakCountFactor().spec - reports = evaluate_two_runs( - factor, spec, _eval_cfg(), price, book, - universe_symbols=tuple(sorted(set(book.index.get_level_values("symbol")))), - fee_rate=0.001, - report_dir=tmp_path, - ) - - for report in (reports.no_book, reports.with_book): - by = report.by_name() - assert set(by) == set(MANDATORY_SECTIONS) # all 8 present - assert all(isinstance(s, (Section, Skipped)) for s in by.values()) - assert report.verdict is not None # a verdict was produced - - # no-book: purity Skipped (no book) -> Incremental NOT_ASSESSED - assert isinstance(reports.no_book.by_name()["purity"], Skipped) - assert reports.no_book.verdict.incremental.verdict == AXIS_NOT_ASSESSED - - # with-book: purity is a real Section that populated the Incremental facts - purity = reports.with_book.by_name()["purity"] - assert isinstance(purity, Section) - assert purity.payload["known_factors_supplied"] is True - assert "incremental_ic_ir" in purity.payload - # the axis is now assessed (not NOT_ASSESSED) and is a valid axis state - incr = reports.with_book.verdict.incremental.verdict - assert incr in AXIS_VERDICTS and incr != AXIS_NOT_ASSESSED - - # reports were written to disk (md + json), both runs - for p in (reports.no_book_md, reports.no_book_json, - reports.with_book_md, reports.with_book_json): - assert p.exists() and p.stat().st_size > 0 - - # the research-style dashboard PNG is emitted for both runs (mandatory report - # artifact alongside md/json) - for p in (reports.no_book_dashboard, reports.with_book_dashboard): - assert p.exists() and p.stat().st_size > 20_000 - with open(p, "rb") as fh: - assert fh.read(8) == b"\x89PNG\r\n\x1a\n" - - # metrics extraction surfaces the gated fields - m = extract_metrics(reports.no_book) - assert m["deployment"] in {"Adopt", "Watch", "Reject", "INSUFFICIENT-DATA"} - assert "effective_samples" in m and "ic_ir" in m - - -def test_no_book_run_never_reaches_adopt(tmp_path): - # Structural guarantee (design §6): with no book + no execution facts, at most - # Watch — regardless of the signal (exploratory cap + NOT_ASSESSED axes). - factor, book, price = _synthetic_panels(seed=3) - reports = evaluate_two_runs( - factor, VolumePeakCountFactor().spec, _eval_cfg(), price, book, - universe_symbols=(), - fee_rate=0.001, - report_dir=tmp_path, - ) - assert reports.no_book.verdict.verdict != "Adopt" - assert reports.no_book.verdict.tradable.verdict == AXIS_NOT_ASSESSED diff --git a/tests/test_frozen_input_verification.py b/tests/test_frozen_input_verification.py new file mode 100644 index 0000000..996a6e2 --- /dev/null +++ b/tests/test_frozen_input_verification.py @@ -0,0 +1,559 @@ +"""The C5 harness's frozen inputs are now checked before they are believed. + +Three read sites used to open frozen artifacts with no integrity check at all, +while the directory holding them is demonstrably writable — one file in it was +overwritten on 2026-07-25. So "the harness reconciled against whatever was on +disk" was never hypothetical, and these tests pin the checks that close it: + +* ``verify_frozen_panel_file`` — one panel against the hash authored in git, + following the same jump -> PR-C fork the reconciler's path helper takes; +* ``verify_anchor_record`` — the hand-anchor record's bytes AND its inventory; +* ``check_overwrite_allowed`` — the write side, refusing to clobber that record + without intent. + +Every conviction test starts from a state that verifies GREEN, so a red result +is attributable to the tampering rather than to a check that is red on +everything. +""" + +from __future__ import annotations + +import json +from pathlib import Path + +import pytest + +from qt.hand_anchor_manifest import ( + MANIFEST_SCHEMA, + AnchorRecordMismatch, + record_inventory, + verify_anchor_record, +) +from qt.hand_anchors_d2 import ( + PRODUCED_RECORD_KEYS, + RecordOverwriteRefused, + check_overwrite_allowed, +) +from qt.panel_freeze import ( + D1_MANIFEST_DOC, + PR_C_MANIFEST_DOC, + PR_C_SUBDIR, + FrozenPanelMismatch, + canonical_content_hash, + doc_for_frozen_panel, + verify_frozen_panel_file, +) +from tests.fixtures.frozen_baseline import build_frozen_tree, make_panel, rewrite_panel + +REPO = Path(__file__).resolve().parents[1] + + +# --------------------------------------------------------------------------- # +# A/B — per-panel verification at the harness read sites +# --------------------------------------------------------------------------- # +def test_a_frozen_panel_that_matches_git_verifies(tmp_path: Path): + doc = build_frozen_tree(tmp_path) + panel_path = tmp_path / "panels" / "alpha_20.parquet" + expected = canonical_content_hash(make_panel("alpha_20")) + assert verify_frozen_panel_file(panel_path, "alpha_20", doc_path=doc) == expected + + +def test_a_drifted_panel_is_refused_at_the_read_site(tmp_path: Path): + doc = build_frozen_tree(tmp_path) + panel_path = tmp_path / "panels" / "alpha_20.parquet" + assert verify_frozen_panel_file(panel_path, "alpha_20", doc_path=doc) # green + + panel = make_panel("alpha_20") + panel.iloc[0] = float(panel.iloc[0]) + 1.0 + rewrite_panel(tmp_path, "alpha_20", panel) + with pytest.raises(FrozenPanelMismatch, match="canonical content hash"): + verify_frozen_panel_file(panel_path, "alpha_20", doc_path=doc) + + +def test_a_missing_panel_is_refused_rather_than_skipped(tmp_path: Path): + doc = build_frozen_tree(tmp_path) + (tmp_path / "panels" / "alpha_20.parquet").unlink() + with pytest.raises(FrozenPanelMismatch, match="missing from disk"): + verify_frozen_panel_file( + tmp_path / "panels" / "alpha_20.parquet", "alpha_20", doc_path=doc + ) + + +def test_a_factor_absent_from_the_document_is_refused(tmp_path: Path): + """No expectation in git means nothing to verify against — which must be a + refusal, not a pass.""" + doc = build_frozen_tree(tmp_path) + from qt.panel_freeze import atomic_write_parquet + + atomic_write_parquet(make_panel("stranger_20"), tmp_path / "panels" / "stranger_20.parquet") + with pytest.raises(FrozenPanelMismatch, match="no expectation"): + verify_frozen_panel_file( + tmp_path / "panels" / "stranger_20.parquet", "stranger_20", doc_path=doc + ) + + +def test_the_jump_fork_is_read_off_the_path_not_the_factor_id(): + """Both trees hold a ``jump_amount_corr_20.parquet`` and they are DIFFERENT + factor definitions (v1.0 untruncated vs v1.1 truncated). Verification has to + follow the same fork the reconciler's path helper takes, or it checks a panel + against the other panel's hash.""" + d1 = doc_for_frozen_panel(REPO / "artifacts/refactor_baseline/panels/x.parquet", REPO) + prc = doc_for_frozen_panel( + REPO / f"artifacts/refactor_baseline/{PR_C_SUBDIR}/panels/x.parquet", REPO + ) + assert d1 == REPO / D1_MANIFEST_DOC + assert prc == REPO / PR_C_MANIFEST_DOC + assert d1 != prc + + +def test_the_reconciler_routes_jump_the_same_way_verification_does(): + """Coupling: if frozen_panel_path ever stops routing jump to the PR-C tree, + or verification stops following it, the pair must not drift silently.""" + from qt.factor_eval_reconcile import frozen_panel_path + + jump = frozen_panel_path("jump_amount_corr_20", REPO) + other = frozen_panel_path("volume_peak_count_20", REPO) + assert doc_for_frozen_panel(jump, REPO) == REPO / PR_C_MANIFEST_DOC + assert doc_for_frozen_panel(other, REPO) == REPO / D1_MANIFEST_DOC + + +# --------------------------------------------------------------------------- # +# C-1 — the hand-anchor record +# --------------------------------------------------------------------------- # +def _record(tmp_path: Path, **overrides) -> tuple[Path, Path]: + payload = { + "seed": 1, + "tolerance": 1e-12, + "frozen14": [{"factor_id": "a", "ok": True}], + "daily_pending_engine": [{"factor_id": "momentum_20"}], + "all_ok_frozen14": True, + } + payload.update(overrides) + record = tmp_path / "hand_anchors_d2.json" + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + import hashlib + + manifest = tmp_path / "manifest.json" + manifest.write_text( + json.dumps( + { + "schema": MANIFEST_SCHEMA, + "sha256": hashlib.sha256(record.read_bytes()).hexdigest(), + "contents": record_inventory(payload), + } + ), + encoding="utf-8", + ) + return record, manifest + + +def test_an_untouched_record_verifies(tmp_path: Path): + record, manifest = _record(tmp_path) + check = verify_anchor_record(record, manifest) + assert check.inventory["frozen14_rows"] == 1 + assert check.inventory["daily_engine_compared_rows"] == 0 + + +def test_an_edited_record_is_refused(tmp_path: Path): + record, manifest = _record(tmp_path) + assert verify_anchor_record(record, manifest) # green control + payload = json.loads(record.read_text()) + payload["frozen14"][0]["ok"] = False # flip a verdict + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + with pytest.raises(AnchorRecordMismatch, match="sha256"): + verify_anchor_record(record, manifest) + + +def test_a_manifest_updated_without_noticing_the_contents_change_is_refused(tmp_path: Path): + """The sha alone cannot see this: someone reruns the tool, updates the sha to + match, and the record silently loses rows. The inventory is what catches it, + which is why it is asserted rather than merely written down.""" + record, manifest = _record(tmp_path, daily_engine_compared=[{"ok": True}]) + assert verify_anchor_record(record, manifest) # green, 1 compared row + + payload = json.loads(record.read_text()) + payload["daily_engine_compared"] = [] # the 2026-07-25 shape + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + import hashlib + + stale = json.loads(manifest.read_text()) + stale["sha256"] = hashlib.sha256(record.read_bytes()).hexdigest() # sha "fixed" + manifest.write_text(json.dumps(stale), encoding="utf-8") + + with pytest.raises(AnchorRecordMismatch, match="daily_engine_compared_rows"): + verify_anchor_record(record, manifest) + + +def test_a_missing_manifest_is_refused_not_skipped(tmp_path: Path): + record, _ = _record(tmp_path) + with pytest.raises(AnchorRecordMismatch, match="cannot be verified"): + verify_anchor_record(record, tmp_path / "nope.json") + + +def test_the_real_record_matches_its_committed_manifest(): + """Coupling to reality: the pinned sha must be the bytes actually on disk, or + every anchors run fails.""" + record = REPO / "artifacts/refactor_baseline/hand_anchors_d2.json" + manifest = REPO / "docs/factors/d5_hand_anchor_record_manifest.json" + if not record.exists(): # gitignored bulk tree + pytest.skip("frozen baseline not present in this checkout") + check = verify_anchor_record(record, manifest) + assert check.inventory["frozen14_rows"] == 70 + assert check.inventory["daily_pending_engine_rows"] == 20 + + +def test_the_committed_manifest_states_the_record_is_incomplete(): + """Pinning an incomplete record is fine; letting a later reader take "pinned" + for "complete" is not. The shortfall must be stated in the manifest.""" + manifest = json.loads( + (REPO / "docs/factors/d5_hand_anchor_record_manifest.json").read_text(encoding="utf-8") + ) + assert manifest["contents"]["daily_engine_compared_rows"] == 0 + prose = manifest["what_this_record_does_NOT_contain"] + assert "daily_engine_compared" in prose and "EMPTY" in prose + assert "cannot be completed" in prose + # and it must say the D2 conclusion itself survived + assert "88 hand anchors" in prose + + +# --------------------------------------------------------------------------- # +# D-1 — the write side +# --------------------------------------------------------------------------- # +def test_writing_a_fresh_record_is_not_refused(tmp_path: Path): + """The tool must still be able to do its job.""" + check_overwrite_allowed( + tmp_path / "absent.json", PRODUCED_RECORD_KEYS, allow_overwrite=False + ) + + +def test_overwriting_an_existing_record_is_refused_by_default(tmp_path: Path): + target = tmp_path / "rec.json" + target.write_text(json.dumps({k: 1 for k in PRODUCED_RECORD_KEYS}), encoding="utf-8") + with pytest.raises(RecordOverwriteRefused, match="refusing to overwrite"): + check_overwrite_allowed(target, PRODUCED_RECORD_KEYS, allow_overwrite=False) + + +def test_the_refusal_names_the_keys_this_run_would_drop(tmp_path: Path): + """Computed by DIFFERENCE against what the run produces, so a future key is + protected without anyone remembering to add it to a list.""" + target = tmp_path / "rec.json" + payload = {k: 1 for k in PRODUCED_RECORD_KEYS} + payload["daily_engine_compared"] = [{"ok": True}] + payload["some_future_key"] = 1 + target.write_text(json.dumps(payload), encoding="utf-8") + with pytest.raises(RecordOverwriteRefused) as caught: + check_overwrite_allowed(target, PRODUCED_RECORD_KEYS, allow_overwrite=False) + message = str(caught.value) + assert "daily_engine_compared" in message and "some_future_key" in message + assert "CANNOT be rebuilt" in message + assert "d5_hand_anchor_record_manifest.json" in message # what to do after + + +def test_explicit_intent_is_honoured(tmp_path: Path): + target = tmp_path / "rec.json" + target.write_text(json.dumps({"daily_engine_compared": []}), encoding="utf-8") + check_overwrite_allowed(target, PRODUCED_RECORD_KEYS, allow_overwrite=True) + + +def test_an_unreadable_existing_record_still_refuses(tmp_path: Path): + """Corrupt JSON must not be read as "nothing at risk, go ahead".""" + target = tmp_path / "rec.json" + target.write_text("{not json", encoding="utf-8") + with pytest.raises(RecordOverwriteRefused): + check_overwrite_allowed(target, PRODUCED_RECORD_KEYS, allow_overwrite=False) + + +def test_the_declared_key_set_is_what_the_writer_actually_produces(): + """The guard reasons about PRODUCED_RECORD_KEYS before the work runs, so the + writer asserts its payload equals that set. Pin both halves here.""" + source = (REPO / "qt" / "hand_anchor_rows.py").read_text(encoding="utf-8") + assert "PRODUCED_RECORD_KEYS" in source + assert "payload keys drifted" in source + assert PRODUCED_RECORD_KEYS == frozenset( + {"seed", "tolerance", "elapsed_seconds", "frozen14", + "daily_pending_engine", "all_ok_frozen14"} + ) + + +def test_the_verified_path_is_the_only_gateway_to_a_frozen_panel(): + """AST, not grep: every call to ``frozen_panel_path`` in the reconciler must + sit inside ``verified_frozen_panel_path``. + + This is the property that makes A/B real. Checking "the read site calls + something verified" per site turns "did we verify this read?" into a + per-site question, and the site that forgets is precisely the one that will + not announce itself. With one gateway there is nothing to forget. + + Range: it proves no OTHER site resolves a frozen panel path, not that the + verifier's own body is correct — the tests above cover that. + """ + import ast + + source = (REPO / "qt" / "factor_eval_reconcile.py").read_text(encoding="utf-8") + tree = ast.parse(source) + gateway = "verified_frozen_panel_path" + + def enclosing(node: ast.AST) -> str | None: + for func in ast.walk(tree): + if isinstance(func, ast.FunctionDef) and any( + inner is node for inner in ast.walk(func) + ): + return func.name + return None + + offenders = [ + enclosing(call) + for call in ast.walk(tree) + if isinstance(call, ast.Call) + and isinstance(call.func, ast.Name) + and call.func.id == "frozen_panel_path" + and enclosing(call) != gateway + ] + assert not offenders, ( + "these resolve a frozen panel path without the verifying gateway: " + f"{offenders}" + ) + # anti-vacuity: the gateway itself must actually make the call + assert f"def {gateway}" in source and "frozen_panel_path(factor_id, repo_root)" in source + + +def test_both_frozen_panel_reads_go_through_the_gateway(): + """Companion to the above: the two read sites must still READ something, so + the gateway is not merely unused.""" + import ast + + source = (REPO / "qt" / "factor_eval_reconcile.py").read_text(encoding="utf-8") + tree = ast.parse(source) + for name in ("run_panels_mode", "run_anchors_mode"): + func = next( + n for n in ast.walk(tree) if isinstance(n, ast.FunctionDef) and n.name == name + ) + body = ast.get_source_segment(source, func) or "" + assert "read_parquet" in body, f"{name} no longer reads a panel" + assert "verified_frozen_panel_path" in body, f"{name} bypasses the gateway" + + +# --------------------------------------------------------------------------- # +# The anchors read site: that the check is WIRED, not merely that it works +# --------------------------------------------------------------------------- # +def _anchor_repo(tmp_path: Path, *, tamper: bool) -> Path: + """A miniature repo_root holding the record and its git-tracked manifest. + + Built through the production inventory helper, so the manifest states what + the record actually contains — then optionally tampered AFTER the manifest + is written, which is the real-world shape (bytes drift, expectation does + not). + """ + import hashlib + + from qt.factor_eval_reconcile import ANCHORS_JSON + from qt.hand_anchor_manifest import DEFAULT_MANIFEST + + payload = { + "seed": 1, + "tolerance": 1e-12, + "frozen14": [ + {"factor_id": "alpha_20", "date": "2024-01-02", "symbol": "000001.SZ", + "hand": 1.0, "engine": 1.0, "rel_diff": 0.0, "ok": True}, + ], + "daily_pending_engine": [], + "all_ok_frozen14": True, + } + record = tmp_path / ANCHORS_JSON + record.parent.mkdir(parents=True, exist_ok=True) + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + + manifest = tmp_path / DEFAULT_MANIFEST + manifest.parent.mkdir(parents=True, exist_ok=True) + manifest.write_text( + json.dumps( + { + "schema": MANIFEST_SCHEMA, + "sha256": hashlib.sha256(record.read_bytes()).hexdigest(), + "contents": record_inventory(payload), + } + ), + encoding="utf-8", + ) + if tamper: + payload["frozen14"][0]["ok"] = False # flip a verdict after pinning + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + return tmp_path + + +def test_the_anchors_read_site_returns_rows_when_the_record_is_intact(tmp_path: Path): + """GREEN CONTROL. Without it, the refusal test below could pass simply + because load_anchor_rows raises on everything.""" + from qt.factor_eval_reconcile import load_anchor_rows + + rows = load_anchor_rows("alpha_20", _anchor_repo(tmp_path, tamper=False)) + assert len(rows) == 1 and rows[0]["symbol"] == "000001.SZ" + + +def test_the_anchors_read_site_refuses_a_record_that_disagrees_with_its_manifest( + tmp_path: Path, +): + """BEHAVIOURAL, on purpose — this pins the WIRING, which is what was missing. + + The verifier itself was thoroughly tested while nothing asserted that + ``load_anchor_rows`` calls it: replacing that call with ``pass`` left the + whole suite green. That is the identical shape as the first A/B mutation + (function pinned, wiring not), surviving in the third read site after the + other two were fixed. + + An AST assertion would only show the call is written down. A call with the + wrong argument, or one whose exception is swallowed, passes AST and fails + here — so the load-bearing test is the one that actually feeds the function + a bad record and demands a refusal. + """ + from qt.factor_eval_reconcile import ReconciliationError, load_anchor_rows + + with pytest.raises(ReconciliationError, match="sha256"): + load_anchor_rows("alpha_20", _anchor_repo(tmp_path, tamper=True)) + + +def test_the_anchors_read_site_refuses_before_it_parses(tmp_path: Path): + """Order matters: a corrupt record must be refused for the RIGHT reason. + If parsing came first, a tampered-and-unparseable record would surface as a + JSON error and the integrity failure would never be named.""" + from qt.factor_eval_reconcile import ANCHORS_JSON, ReconciliationError, load_anchor_rows + + repo = _anchor_repo(tmp_path, tamper=False) + (repo / ANCHORS_JSON).write_text("{ not json at all", encoding="utf-8") + with pytest.raises(ReconciliationError, match="sha256"): # not a JSONDecodeError + load_anchor_rows("alpha_20", repo) + + +def test_a_new_key_cannot_appear_without_the_manifest_noticing(tmp_path: Path): + """LOW-1: counts alone cannot see a key APPEAR. + + The reviewer added ``all_ok_daily: true`` to the record, refreshed the sha, + and it was accepted — while the manifest's most prominent prose says exactly + that key is absent. The key SET is now compared, so "pinned is not complete" + no longer rests on prose. + """ + import hashlib + + record, manifest = _record(tmp_path) + assert verify_anchor_record(record, manifest) # green control + + payload = json.loads(record.read_text()) + payload["all_ok_daily"] = True # the key the manifest says is ABSENT + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + refreshed = json.loads(manifest.read_text()) + refreshed["sha256"] = hashlib.sha256(record.read_bytes()).hexdigest() + manifest.write_text(json.dumps(refreshed), encoding="utf-8") + + with pytest.raises(AnchorRecordMismatch, match="top_level_keys"): + verify_anchor_record(record, manifest) + + +def test_the_inventory_adds_nothing_on_the_rerun_path(tmp_path: Path): + """The documented RANGE, measured rather than asserted. + + A plain `hand_anchor_rows` rerun keeps every count and the key set identical + (same seed, same stratified selection) while the values inside move. It is + refused with a stale sha and ACCEPTED once the sha is honestly refreshed -- + so on that path the guard is the reviewed git diff, not this check. + + This test therefore asserts a LIMITATION. If someone later strengthens the + check so this case is refused, this test fails -- and that is the intended + signal to update the module's RANGE section in the same commit, so the + docstring cannot drift wider than what the code does. + """ + import hashlib + + record, manifest = _record(tmp_path) + assert verify_anchor_record(record, manifest) # green control + + payload = json.loads(record.read_text()) + before = record_inventory(payload) + payload["frozen14"][0]["engine"] = 999.0 # a value moves... + payload["frozen14"][0]["ok"] = False + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + assert record_inventory(payload) == before, "premise broken: a count changed" + + with pytest.raises(AnchorRecordMismatch, match="sha256"): + verify_anchor_record(record, manifest) # ...caught only by the sha + + refreshed = json.loads(manifest.read_text()) + refreshed["sha256"] = hashlib.sha256(record.read_bytes()).hexdigest() + manifest.write_text(json.dumps(refreshed), encoding="utf-8") + verify_anchor_record(record, manifest) # ACCEPTED — the documented limit + + +def test_the_inventory_still_bites_where_the_shape_moves(tmp_path: Path): + """The other half of the range: it is limited, not useless. + + Stating only the limitation would be its own overclaim in the opposite + direction. + """ + import hashlib + + record, manifest = _record(tmp_path) + payload = json.loads(record.read_text()) + payload["daily_engine_compared"] = [{"ok": True}, {"ok": True}] # rows gained + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + refreshed = json.loads(manifest.read_text()) + refreshed["sha256"] = hashlib.sha256(record.read_bytes()).hexdigest() + manifest.write_text(json.dumps(refreshed), encoding="utf-8") + + with pytest.raises(AnchorRecordMismatch, match="daily_engine_compared_rows"): + verify_anchor_record(record, manifest) + + +def test_the_range_section_states_where_the_protection_actually_is(): + """The claim and the code move together: the docstring must name the rerun + path as uncovered, and must not still promise the unreachable case.""" + import re + + import qt.hand_anchor_manifest as module + + # Whitespace-normalised: these sentences WRAP in the source, so a raw + # substring match tests line breaks rather than the claim. (Same trap as the + # earlier wrapped-string scan in this suite.) + def flat(text: str) -> str: + return re.sub(r"\s+", " ", text or "") + + doc = flat(module.__doc__) + assert "RANGE OF THE INVENTORY CHECK" in doc + assert "the protection is the human reading the git diff" in doc + assert "no longer reachable FROM HERE" in doc + # NOT NARROWER than reality either: the flag is neither a count nor a key, + # and it IS compared. Saying "counts or keys" would understate the guard and + # invite someone to add a redundant one. + assert "the ``all_ok_frozen14`` verdict flag" in doc + assert "more than row counts" in doc + # the old overclaim must not survive in the function docstring either + assert "silently went to zero" not in flat(module.verify_anchor_record.__doc__) + + +def test_the_inventory_also_bites_when_the_verdict_flag_flips(tmp_path: Path): + """The direction that showed the first RANGE wording was too NARROW. + + ``all_ok_frozen14`` is a bool: neither a row count nor a key. Describing the + guard as catching "counts or keys" understated it — and an understated range + is a defect in its own right, because it invites someone to add a redundant + guard for something already covered. + """ + import hashlib + + record, manifest = _record(tmp_path) + payload = json.loads(record.read_text()) + before = record_inventory(payload) + payload["all_ok_frozen14"] = not payload["all_ok_frozen14"] + record.write_text(json.dumps(payload, indent=2), encoding="utf-8") + + after = record_inventory(payload) + assert after["top_level_keys"] == before["top_level_keys"], "premise: no key moved" + assert all( + after[k] == before[k] + for k in ("frozen14_rows", "daily_pending_engine_rows", "daily_engine_compared_rows") + ), "premise: no count moved" + + refreshed = json.loads(manifest.read_text()) + refreshed["sha256"] = hashlib.sha256(record.read_bytes()).hexdigest() + manifest.write_text(json.dumps(refreshed), encoding="utf-8") + + with pytest.raises(AnchorRecordMismatch, match="all_ok_frozen14"): + verify_anchor_record(record, manifest)