diff --git a/docs/factors/d5_exec_baseline_freeze.md b/docs/factors/d5_exec_baseline_freeze.md new file mode 100644 index 0000000..dfa3cff --- /dev/null +++ b/docs/factors/d5_exec_baseline_freeze.md @@ -0,0 +1,154 @@ +# 因子层重构 D5 —— exec 基准评估 artifact 冻结 + +> **状态**：D5 第一个交付物（C1），**先于任何 D5 代码改动执行**。 +> 设计出处：v3.2 §五第 3/4 腿（语义对账的比较对象）、§九 D5 行。 +> 本文是 provenance 叙述；**机器可读权威 manifest 是入 git 的 +> `docs/factors/d5_exec_baseline_manifest.json`**（77 条 sha256）。 +> 冻结的字节在 `artifacts/refactor_baseline/exec_baseline/`（gitignored，不入 git）。 +> +> ⚠️ **前置条件**：本文所有命令都需要 gitignored 的 `artifacts/` 树。**在没有它的 worktree 里 +> 开箱跑不起来**——需先对主 checkout 的 `artifacts` 建符号链接，或用 `--repo-root` 指向 +> 一个有该树的 checkout（先例：`qt/factor_hotpath_smoke.py` docstring 的同类说明）。 +> 建符号链接时注意：若 worktree 里已存在真实 `artifacts/` 目录，`ln -s` 会嵌套成 +> `artifacts/artifacts`（本步实际踩到过）。 + +--- + +## 一、为什么必须先冻结（读这段再动任何东西） + +D5 要在删掉 11 个旧 runner 之前证明新引擎没有悄悄改变任何东西。唯一能证明的办法是拿新 +runner 的输出和**旧 runner 的输出**比。而旧输出： + +1. **只有一份**，在 `artifacts/reports/`，**gitignored、只存在于本机**； +2. **不可从当前代码再生**——需要 pre-D5 的树 + 约 1.5 小时真实 run； +3. 设计 §五 提到的耐久副本 `scratchpad/baseline_reports/` **已不存在**（旧会话 scratchpad，已清）； +4. **新 runner 沿用同名输出路径**。 + +即：新 runner 跑一次就会覆盖唯一的基线，随后的"对账"会比对刚被自己覆盖的文件——正是 +CLAUDE.md 记录在案的 `compare_postmerge.py` **空对账**失效模式（拿新结果和自己比， +构造性恒真，`ic=1.0` vs `ic=999.0` 也会打印"精确复现"）。 + +**所以顺序不可交换：先冻结，再改代码。** 本 commit 不含任何 D5 引擎代码。 + +## 二、冻结了什么（77 个文件，按因子列表派生而非按目录扫描） + +| 类别 | 数量 | 说明 | +|---|---|---| +| `eval_{factor}_exec_{book}.json` | 22 | **对账的权威机器可读源**（IC / ICIR / 分位价差 / verdict 全在此） | +| `eval_{factor}_exec_{book}.md` | 22 | 渲染报告 | +| `eval_{factor}_exec_basis_sanity.md` | 11 | #79 的覆盖率/偏差披露（含 `coverage_bias_bad_vwap` 原话） | +| `eval_{factor}_exec_{book}_dashboard.png` | 22 | 单图 dashboard（16MB，同样不可再生） | + +11 因子 × 2 book。清单**由因子列表派生**（`expected_artifact_names()`），不是目录扫描—— +少一个文件是**可读报错**，不是一个静静变小的基线。多一个未知成员同样报错。 + +> 注：`_exec_basis_sanity.md` 与 dashboard 不在 lead 交办的「22 json + 22 md」里，是本步 +> 清点目录时发现的同族产物。它们同样单份、同样不可再生，故一并冻结——**冻结的成本是一次 +> 文件拷贝，漏冻的成本是永久丢失**。 + +**未冻结**：close 基准的 **66** 个 artifact（22 json + 22 md + **22 png**）。它们不是 D5 的对账 +参照（统一 runner 是 exec-only，决策 4），且未被新 runner 的输出路径覆盖，原地不动。 +（本数曾误写 44——**漏的正是 dashboard**，而上一段刚说明 exec 侧发现 dashboard 才一并冻结， +close 侧没回头改数。同一次清点里，同一类东西数了两遍、两次结果不同。） + +## 三、Provenance（观察到的事实 vs 归因） + +**观察到的**（可复验）： + +- 77 个 exec 文件的 mtime **全部落在同一秒 `2026-07-21 15:33:20`**（实测跨度 **9.46 ms**）。 +- 对照：66 个 close 基准 artifact 里，**22 个 JSON 的 mtime 呈 11 个递进秒值** + `09:24:33 → 09:32:45 → … → 10:43:54`——每因子一个（同因子的 no_book/with_book 同秒）， + 正是一次真实 11 因子 run 的写入签名，且与 CLAUDE.md 记录的 #74 合并后全量重跑窗口 + （09:24:33→10:43:54）**逐值吻合**。 + ⚠️ **「11」是按 JSON 计的**；全部 66 个文件（含 md/png）共 **27 个不同秒值**，因为每因子 + 的多个产物写在相邻但不同的秒里。引用本条时必须带口径，否则 11 与 27 会看起来矛盾。 + +**推论**：exec 家族**不是**在 `artifacts/reports/` 里跑出来的，而是**被整体拷贝进来的** +（一次 1.5h 的 run 不可能让 77 个文件落在同一个 9.46ms 窗口里；对照组恰恰不是这样）。拷贝时刻在 PR #79 合并 +（`b04c966`，15:25）之后 8 分钟——与「#79 在独立 worktree 里跑完、合并后把产物拷进主 checkout」 +的形态一致。 + +**归因（非观察）**：内容归属于 PR #79 的 exec 基准 run。这是**从 PR 记录推断的**，不是从 +文件系统观察到的。本文如此区分，是因为 #79 的 code review 未返回、靠自查合并（CLAUDE.md +已记录），其 provenance 本就该按证据强度分级陈述，而不是写成"由 SHA X 产生"。 + +**冻结时刻**：`frozen_at_git_head = 45c14aa`（D0–D4 已合并的 `main`）。这是**冻结动作**发生 +的 SHA，**不是产生 artifact 的 SHA**——manifest 字段名即为 `frozen_at_git_head`，不叫 +`producing_sha`，避免 D1 manifest 那种「producing SHA」措辞被误读。 + +## 四、结构性隔离：对账工具为什么读不到 live 路径 + +lead 的要求是「用路径断言或结构性隔离保证，不靠纪律」。实现为三层，**第 2 层承重**： + +| # | 机制 | 拦得住什么 | +|---|---|---| +| 1 | 读取器要求一份 git-tracked manifest 描述该目录 | 随手指向 `artifacts/reports/`（那里没有 manifest） | +| 2 | **每次读取都对该文件 sha256 校验 manifest** | **把 live 产物拷进冻结目录**——它问的不是"路径看着对不对"，而是"字节是不是冻结的那些字节" | +| 3 | 目录若解析为 `artifacts/reports` 直接拒绝 | 显式误用，早失败早报错 | + +**为什么 (2) 有牙**：字节在 gitignored 的 `artifacts/` 下，哈希在 git 里。覆盖冻结树的人 +无法同时悄悄移动哈希——哈希只能经一次会被 review 的 `git diff` 变动。 + +**mutation 证据（实跑，非声称）**： + +| mutation | 结果 | +|---|---| +| 读取器指向 `artifacts/reports/` | **RAISED** `refusing to read the LIVE artifacts directory` | +| 冻结文件末字节 XOR 0x01 | **RAISED** `on-disk sha256 … != manifest …`；`verify_all` **76 ok / 1 problem**；同目录未改的兄弟文件仍正常读出（守卫是定向的，不是一刀切） | +| 冻结后改 **live** 源文件再 freeze | **RAISED** `differs from the live artifact`，且冻结副本**逐字节未动** | +| 少一个 / 多一个清单成员 | **RAISED**（分别 `absent` / `unexpected`） | +| **清空 manifest 的 `source_note` + `frozen_at_git_head`** | **rc=1 FAILED** `test_committed_manifest_describes_the_real_frozen_baseline`（新增的恒跑 provenance 断言；**修复前 16 个测试对此全绿**） | +| **把一个冻结文件塞进 git 索引**（`update-index --cacheinfo`，绕开 symlink） | **rc=1 FAILED** `test_frozen_bytes_are_untracked_…`；索引内 artifacts 路径 **0→1→0** | +| 在未改动的树上跑 `freeze`（真实执行） | `copied: 0 / already ok: 77 / manifest: **unchanged**`，manifest porcelain **0 行** | + +⚠️ **本步第二次空 mutation（照实记录）**：上表倒数第二行的**第一版**用 `git add -f`， +但 git 对符号链接下的路径报 `beyond a symbolic link` 而**拒绝执行**——mutation 什么也没改， +测试于是"通过"，且是**为错误的理由通过**。改用 `git update-index --cacheinfo` 直接写索引 +（不走文件系统、不受 symlink 影响）才真正把它变红。**同一份工作里两次写出不会失败的 +mutation**，都靠"预期与观察不符"而非靠测试本身抓住——见 §四末与本行。 + +⚠️ **记一次本步自己的失误（照实记录，与项目既有先例同格式）**：tamper 测试的第一版把 +`b'"ic"'` 替换成 `b'"IC"'`，而该字节串**在文件里根本不存在** → `.replace()` 是 no-op → +"mutation" 什么也没改，测试对着一个从未被触发的守卫打印通过。它之所以被抓住，只因为同一次 +输出里 `verify_all` 报了 `ok=77 problems=0`，与"我刚破坏了一个文件"的预期矛盾。 +**这正是 §六.10「不变性测试没有 mutation 证据」要防的形态，而本步亲手造了一个。** +现版本对 mutation 本身加了断言（`assert after != before` 且 sha256 必须不同）， +使"mutation 是空的"变成测试失败而不是测试通过。 + +## 五、复验命令 + +``` +# 校验冻结树与 manifest 一致（不拷贝任何东西） +python -m qt.exec_baseline_freeze --verify + +# 再次冻结：**字节幂等**——树没变则一个字节都不写（含 manifest）；内容不同则拒绝并保留原副本 +python -m qt.exec_baseline_freeze +``` + +⚠️ **「字节幂等」这个限定词是修出来的，不是一开始就对**（评审 HIGH，照实记录）：初版 +`freeze()` 在末尾**无条件重写 manifest**，与本轮有没有拷贝文件无关，且 `source_note` 来自 +CLI 参数、默认空、**不从既有 manifest 继承**。于是在一棵完全没变的树上跑上面这条本文档 +**主动教人跑**的命令，会打印 `copied: 0 / already ok: 77`（纯成功），而 git diff 里 +`frozen_at_utc` / `frozen_at_git_head` 已改指向重跑者、**§三整段 provenance 叙述被清空**—— +**坏掉的恰是本 PR 最下功夫保护的东西**，而且 16 个测试在它被清空后全部通过。 + +重写后的 `frozen_at_git_head` 还是一个**主动的假声明**（它会宣称基线冻结于重跑者的 HEAD）， +正是本文 §三末尾专门解释过要避免的那种误读——同 #76/#78/#82 形态。 + +**修法**：provenance 三字段从既有 manifest **继承**（显式 `--source-note` 才覆盖）； +内容不变则**完全不写**。现在 `--verify` 与 `freeze` 都不会改动 provenance。 + +冻结/校验工具：`qt/exec_baseline_freeze.py`；测试：`tests/test_exec_baseline_freeze.py`。 +读取器 `FrozenExecBaseline` 是 D5 对账**唯一**支持的入口——对账代码不得自己 `open()` +`artifacts/reports/` 下的任何文件。 + +## 六、清单摘要 + +完整 77 条 `(name, sha256, size, source_mtime_utc)` 见 +`docs/factors/d5_exec_baseline_manifest.json`。抽样： + +| 文件 | sha256（前 12） | bytes | +|---|---|---| +| `eval_amp_marginal_anomaly_vol_exec_no_book.json` | `6755304f05c7` | 19487 | +| `eval_amp_marginal_anomaly_vol_exec_no_book.md` | `2300300c7d26` | 16286 | +| `eval_amp_marginal_anomaly_vol_exec_basis_sanity.md` | `4856188e2674` | 3892 | diff --git a/docs/factors/d5_exec_baseline_manifest.json b/docs/factors/d5_exec_baseline_manifest.json new file mode 100644 index 0000000..09e8e0c --- /dev/null +++ b/docs/factors/d5_exec_baseline_manifest.json @@ -0,0 +1,490 @@ +{ + "schema": "d5-exec-baseline-freeze/1", + "frozen_at_utc": "2026-07-25T13:44:24.553842+00:00", + "frozen_at_git_head": "45c14aa036c37ddafc490bc16f961ec5212b5025", + "source_dir": "artifacts/reports", + "frozen_root": "artifacts/refactor_baseline/exec_baseline", + "file_count": 77, + "factors": [ + "jump_amount_corr", + "minute_ideal_amplitude", + "amp_marginal_anomaly_vol", + "volume_peak_count", + "intraday_amp_cut", + "peak_interval_kurtosis", + "valley_relative_vwap", + "valley_ridge_vwap_ratio", + "ridge_minute_return", + "valley_price_quantile", + "peak_ridge_amount_ratio" + ], + "books": [ + "no_book", + "with_book" + ], + "source_note": "PR #79 exec-to-exec rebasis artifacts. OBSERVED: all 77 files share mtime 2026-07-21 15:33:20 UTC-7 (bulk copy; ~2ms spread), 8 minutes after PR #79 merged as b04c966 at 15:25. Content is ATTRIBUTED to the #79 exec-basis run per the PR record; the copy instant is observed, the run is not.", + "files": [ + { + "name": "eval_amp_marginal_anomaly_vol_exec_basis_sanity.md", + "sha256": "4856188e26740f9cdf3629aae6e12d1932670c4be3ccba9779a54c74bdd63a2f", + "size_bytes": 3892, + "source_mtime_utc": "2026-07-21T22:33:20.985336+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_no_book.json", + "sha256": "6755304f05c705f5ef0551bf3e3f20a4deec9b9d45899798c1ea4a00f1ac80c3", + "size_bytes": 19487, + "source_mtime_utc": "2026-07-21T22:33:20.977414+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_no_book.md", + "sha256": "2300300c7d26f8b4d8f00d58e7a459e3b4e3664c36f0dbea70e4fa2c35918923", + "size_bytes": 16286, + "source_mtime_utc": "2026-07-21T22:33:20.978123+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_no_book_dashboard.png", + "sha256": "d39ba8fb4590c94dc747b82357dd694dd48303076b13a45d20577c0409139669", + "size_bytes": 655165, + "source_mtime_utc": "2026-07-21T22:33:20.979059+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_with_book.json", + "sha256": "e80b48c6aeec464791a210e2a89fdf14e064d9ec8fae9daa0711576ece4dd2a5", + "size_bytes": 21286, + "source_mtime_utc": "2026-07-21T22:33:20.977462+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_with_book.md", + "sha256": "f1df356c1db62e5730f261d678fd90e67252e8d36f96a451f28b7b0846da9980", + "size_bytes": 17717, + "source_mtime_utc": "2026-07-21T22:33:20.978161+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_with_book_dashboard.png", + "sha256": "a585fcc7492c5d0138d5e13bac1764272d2e0fe77dfbe6b29f411db79fad2b2e", + "size_bytes": 665637, + "source_mtime_utc": "2026-07-21T22:33:20.979250+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_basis_sanity.md", + "sha256": "6fd77ccf8f094d862a6c49bece2c5247bdcb120ad9ee9a59c6c722e13e7e8158", + "size_bytes": 3900, + "source_mtime_utc": "2026-07-21T22:33:20.985399+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_no_book.json", + "sha256": "80cf44b25c4be0c9091e3cc43cc56fdcd6d61b5cb3b378e36b842fb2265ffad8", + "size_bytes": 21131, + "source_mtime_utc": "2026-07-21T22:33:20.977496+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_no_book.md", + "sha256": "eb80ca1dc57267f2470c393047b0acf7eea4dcbdb6e8b682545571d955035a1e", + "size_bytes": 17771, + "source_mtime_utc": "2026-07-21T22:33:20.978218+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_no_book_dashboard.png", + "sha256": "a29121a103c87b2d310ca32ffe18aa603c3b0316b2a5bc09da9488e1c8329ad0", + "size_bytes": 668176, + "source_mtime_utc": "2026-07-21T22:33:20.979444+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_with_book.json", + "sha256": "dc63b7e620d80b37482cc5b280c482de2f70e7c1b31da0b64f1047e431f7fe56", + "size_bytes": 22715, + "source_mtime_utc": "2026-07-21T22:33:20.977527+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_with_book.md", + "sha256": "61ad89dac238e11b0e5e74c24b20c15cb9b033e1d590be0080eec2865dff9936", + "size_bytes": 18994, + "source_mtime_utc": "2026-07-21T22:33:20.978252+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_with_book_dashboard.png", + "sha256": "987f7c14b86401823e747fd0c32ee6bb02ecd36973db4d23c0920e6b4a042bc5", + "size_bytes": 678388, + "source_mtime_utc": "2026-07-21T22:33:20.979649+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_basis_sanity.md", + "sha256": "a67c1e297293ae3a08904005bbd7ab18155394e9d977b7b3a290251116189db4", + "size_bytes": 3885, + "source_mtime_utc": "2026-07-21T22:33:20.985427+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_no_book.json", + "sha256": "126bdc8febb82dbf8e43c2e162e88044f2135924013518b4f03d1a89c3147b0f", + "size_bytes": 20546, + "source_mtime_utc": "2026-07-21T22:33:20.977555+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_no_book.md", + "sha256": "7fce0749339a4753c7fd508c0f86ecd4330b0cb523a93088f98425b18396bb35", + "size_bytes": 16625, + "source_mtime_utc": "2026-07-21T22:33:20.978304+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_no_book_dashboard.png", + "sha256": "8a83f2ecd73c40a85027914595bf1ac6deb23fcb9a5d9019bd16d808c24428c4", + "size_bytes": 620927, + "source_mtime_utc": "2026-07-21T22:33:20.979841+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_with_book.json", + "sha256": "08d791ad46e6793191662eb185083aa1e2a92f8fce82ad11889b6b39b464cfc6", + "size_bytes": 22093, + "source_mtime_utc": "2026-07-21T22:33:20.977582+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_with_book.md", + "sha256": "c9aea4a639c433aa2f6d4691c59025efb20f5c6a4cd79ee35eea104539feb529", + "size_bytes": 17811, + "source_mtime_utc": "2026-07-21T22:33:20.978333+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_with_book_dashboard.png", + "sha256": "7a7714627f10305b1d165f68285f38f4c3c4afb1f31495bdd93b93b1cdb68cdc", + "size_bytes": 630809, + "source_mtime_utc": "2026-07-21T22:33:20.980035+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_basis_sanity.md", + "sha256": "12fa8a81a07451f960ef56ba9770ed03da00bb1a196a02f1c9a0f600575dd239", + "size_bytes": 3894, + "source_mtime_utc": "2026-07-21T22:33:20.985453+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_no_book.json", + "sha256": "495ff3189631c4ee435c3be69db9ad7bf670fa03fdf90d2015e774515f109c3e", + "size_bytes": 20409, + "source_mtime_utc": "2026-07-21T22:33:20.977612+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_no_book.md", + "sha256": "482cc7a219628fac2eb11b5b65708ace0880c4b8d4fabbb6cd3e4e61823d3e6d", + "size_bytes": 16611, + "source_mtime_utc": "2026-07-21T22:33:20.978386+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_no_book_dashboard.png", + "sha256": "878767ccae74ed0e55315d1fb064dea66e155e07f002134a25fa7acacf16933b", + "size_bytes": 572962, + "source_mtime_utc": "2026-07-21T22:33:20.980259+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_with_book.json", + "sha256": "85293e8d63edb08e4ebadcfb3fddc683d08ebb7fd7e3a28e4954d3bf27d7eae8", + "size_bytes": 21955, + "source_mtime_utc": "2026-07-21T22:33:20.977637+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_with_book.md", + "sha256": "84176a922ca7d8a83aa58aee1eff81972223ff3d3e60bcf17ea1659fe173f480", + "size_bytes": 17796, + "source_mtime_utc": "2026-07-21T22:33:20.978414+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_with_book_dashboard.png", + "sha256": "7241548bb091d481494fb917c9e781763acee4b8ec94c10322f17546e895b59d", + "size_bytes": 582827, + "source_mtime_utc": "2026-07-21T22:33:20.980434+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985477+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_no_book.json", + "sha256": "db9527bf4ad190e633fb0dbbf0057cdbfaee28160df8c674fdfa09e5b6f8103d", + "size_bytes": 20351, + "source_mtime_utc": "2026-07-21T22:33:20.977666+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_no_book.md", + "sha256": "622b6f17e0e83226204242096e43199b6f6592ca2dac4ee9a2ffed7957183203", + "size_bytes": 17288, + "source_mtime_utc": "2026-07-21T22:33:20.978471+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_no_book_dashboard.png", + "sha256": "291039b28536bc8084c0b64bc643910bdbb36541c774ba8e99560dd76d276542", + "size_bytes": 758107, + "source_mtime_utc": "2026-07-21T22:33:20.980605+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_with_book.json", + "sha256": "af92d2974d1dc69c13f6226cd97eae5454c15f9a240b364d3fc380995222865a", + "size_bytes": 22110, + "source_mtime_utc": "2026-07-21T22:33:20.977691+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_with_book.md", + "sha256": "d7346960f74ff95e925c191c1fdf1f506dbae5c574569c5c702fd0872b541e83", + "size_bytes": 18679, + "source_mtime_utc": "2026-07-21T22:33:20.978499+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_with_book_dashboard.png", + "sha256": "18b6ace5286822803b38e7630ddaff37680d052fc7f448ef2e7aa7f16c820024", + "size_bytes": 768042, + "source_mtime_utc": "2026-07-21T22:33:20.980828+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985501+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_no_book.json", + "sha256": "bc963b3007d363483a25fc10f38b4902cbb796966ee6a5d325c89b6e8a4538df", + "size_bytes": 20489, + "source_mtime_utc": "2026-07-21T22:33:20.977716+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_no_book.md", + "sha256": "2ba2f0e2dfb7b48ebbacd87d966ec75b57b79e882ef6b93e794c7fb25fb3b84f", + "size_bytes": 19109, + "source_mtime_utc": "2026-07-21T22:33:20.978554+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_no_book_dashboard.png", + "sha256": "62d8d74166e3d48d2807027aea8dc847d6a6bd36117123bc361dac1243ac7558", + "size_bytes": 767069, + "source_mtime_utc": "2026-07-21T22:33:20.981082+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_with_book.json", + "sha256": "bdb4795630a71a5097b16a4f9657da8baae61e8e5a3ddf550efb9f3b1e731e8e", + "size_bytes": 22023, + "source_mtime_utc": "2026-07-21T22:33:20.977742+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_with_book.md", + "sha256": "f0739f5ab6f6c0bb0648a3dcf218a5e0e3142d5aeb5f53a9d4920ece06a4aaf4", + "size_bytes": 20282, + "source_mtime_utc": "2026-07-21T22:33:20.978582+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_with_book_dashboard.png", + "sha256": "7da1593f9adfaf1499ac39f6948b18916450097e1f956747e473f639f55b8d90", + "size_bytes": 777536, + "source_mtime_utc": "2026-07-21T22:33:20.981301+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985526+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_no_book.json", + "sha256": "f4e1778cc22e3758d4b5ce0ce792dc61edfb704fc5e4141e830a0376c6c46f9f", + "size_bytes": 21107, + "source_mtime_utc": "2026-07-21T22:33:20.977768+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_no_book.md", + "sha256": "ed2fe47c2851f0880360985b66de0be89fe23d61ef4fb7df9352899cd7443b20", + "size_bytes": 19551, + "source_mtime_utc": "2026-07-21T22:33:20.978633+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_no_book_dashboard.png", + "sha256": "0be2a1c2a6210c7b47074af7fbd904853d6daeb65b3f7a88301205aa3c0b8d5d", + "size_bytes": 845707, + "source_mtime_utc": "2026-07-21T22:33:20.981516+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_with_book.json", + "sha256": "8fd9e6595c6a21f050b671cac0b8ca96b0bb0bd72d71912d7b5688756c41aade", + "size_bytes": 22906, + "source_mtime_utc": "2026-07-21T22:33:20.977796+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_with_book.md", + "sha256": "c7fe175a5d162e3a533c164dcc5015b0ebe39ebf260b2ebd73f866126593571c", + "size_bytes": 20989, + "source_mtime_utc": "2026-07-21T22:33:20.978660+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_with_book_dashboard.png", + "sha256": "fb527aaf7134910131236dc2d5336efd65b1abcb0496f04172302349dd99e016", + "size_bytes": 854968, + "source_mtime_utc": "2026-07-21T22:33:20.981749+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_basis_sanity.md", + "sha256": "9fc709f253b8771a878b08d0035ba638fcc793d8b5d1b804215a710e90aaf05c", + "size_bytes": 3887, + "source_mtime_utc": "2026-07-21T22:33:20.985550+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_no_book.json", + "sha256": "8f29016285efb8ebf7119256955ab1e8cb54ab37a94d12b508a2ec8d749376a4", + "size_bytes": 20577, + "source_mtime_utc": "2026-07-21T22:33:20.977822+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_no_book.md", + "sha256": "a4ab52edd64c78ab3827dbda71d29a4f68eae475730df12162f9695ef1e0bf5e", + "size_bytes": 19922, + "source_mtime_utc": "2026-07-21T22:33:20.978712+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_no_book_dashboard.png", + "sha256": "6d5ac41bb26109fcd0d265bb6655eaf33472d0e196ad2ad6d869c8cabec6ed83", + "size_bytes": 777643, + "source_mtime_utc": "2026-07-21T22:33:20.982016+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_with_book.json", + "sha256": "b17311ac1ce7566c441f3d62ab373ddf07b95ad9ca9f08ae57f344197b4ee3ec", + "size_bytes": 22113, + "source_mtime_utc": "2026-07-21T22:33:20.977850+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_with_book.md", + "sha256": "c9ea08b66cd0c8ed7fba7e490041f59c8f3c5b437dc44c4a9f36d6ae49240505", + "size_bytes": 21097, + "source_mtime_utc": "2026-07-21T22:33:20.978740+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_with_book_dashboard.png", + "sha256": "aa3c34db623f5efc8267aabc78be5903961a49450fe0b909d5f31501cb0be746", + "size_bytes": 787357, + "source_mtime_utc": "2026-07-21T22:33:20.982236+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985574+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_no_book.json", + "sha256": "237af1b8ebce42b795c7d37e0c2491cee53420c2dd86637882a93a41cc85d4ab", + "size_bytes": 20492, + "source_mtime_utc": "2026-07-21T22:33:20.977883+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_no_book.md", + "sha256": "fd4be587f4344cb2b0209a9baa427333968cce280f9ba6763a864f90a5e8a37c", + "size_bytes": 17746, + "source_mtime_utc": "2026-07-21T22:33:20.978795+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_no_book_dashboard.png", + "sha256": "d92867037ebc470e3f1133a91cb93ba965e2c29157482593ed5bb08bec163f9a", + "size_bytes": 734803, + "source_mtime_utc": "2026-07-21T22:33:20.982455+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_with_book.json", + "sha256": "32e74de07d6fbdb953b38c8095589806cbda57b98620df32304a5c57d351517c", + "size_bytes": 22027, + "source_mtime_utc": "2026-07-21T22:33:20.977919+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_with_book.md", + "sha256": "6d63e704cc770d9bc9d462da60271abe02fcaa267a0e0bc85e321ec245dcf821", + "size_bytes": 18920, + "source_mtime_utc": "2026-07-21T22:33:20.978822+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_with_book_dashboard.png", + "sha256": "43d63d2db041d653839ac2c82d67962eedcbe80dede7f8e14cdb5d52dda4e25d", + "size_bytes": 745381, + "source_mtime_utc": "2026-07-21T22:33:20.982663+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.986839+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_no_book.json", + "sha256": "d9e6694f0d0a00eb36fad38131e919c0bc77f17ecd6a72798d93aa269e879eda", + "size_bytes": 21109, + "source_mtime_utc": "2026-07-21T22:33:20.977958+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_no_book.md", + "sha256": "6aa5456f2a40773920b8eb54bb5915452380cf012ddc03d9acda9908d07a58fe", + "size_bytes": 19026, + "source_mtime_utc": "2026-07-21T22:33:20.978875+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_no_book_dashboard.png", + "sha256": "aa106bd3a08a164aaa9b781d97e4f5592e8c9e015cfb104d771df4703d549a20", + "size_bytes": 805954, + "source_mtime_utc": "2026-07-21T22:33:20.982870+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_with_book.json", + "sha256": "f5b752d68097dc2d13577349188c3f2b01d89a23344f9cbd10582d6c62d07f1e", + "size_bytes": 22682, + "source_mtime_utc": "2026-07-21T22:33:20.977999+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_with_book.md", + "sha256": "59391763e075461d98bcbb5d5432e7b42a5ebec151da24a03d7df32b003e8335", + "size_bytes": 20238, + "source_mtime_utc": "2026-07-21T22:33:20.978908+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_with_book_dashboard.png", + "sha256": "e3157782b76ad035a49cd7b9e301641905463c453aebaf98d1e42914b25becdf", + "size_bytes": 816590, + "source_mtime_utc": "2026-07-21T22:33:20.983125+00:00" + }, + { + "name": "eval_volume_peak_count_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.986876+00:00" + }, + { + "name": "eval_volume_peak_count_exec_no_book.json", + "sha256": "acb4b510ec4f620115e4b5de1f7cbaaa9a076444a372691acb1ba41ac20b8ecd", + "size_bytes": 19773, + "source_mtime_utc": "2026-07-21T22:33:20.978030+00:00" + }, + { + "name": "eval_volume_peak_count_exec_no_book.md", + "sha256": "79c1389d26badf0c5e2dfb25fee367eae394ee454e736052961ce241736b48e5", + "size_bytes": 16506, + "source_mtime_utc": "2026-07-21T22:33:20.978963+00:00" + }, + { + "name": "eval_volume_peak_count_exec_no_book_dashboard.png", + "sha256": "5d0a6085e376d8ab5f3b82b78d0f4aa5f61c602012079e67841b14d4274de814", + "size_bytes": 666669, + "source_mtime_utc": "2026-07-21T22:33:20.983360+00:00" + }, + { + "name": "eval_volume_peak_count_exec_with_book.json", + "sha256": "f4603792251374f575d25c04dc27632e990cfda44c35fceb9c8527d33cb2383a", + "size_bytes": 21535, + "source_mtime_utc": "2026-07-21T22:33:20.978060+00:00" + }, + { + "name": "eval_volume_peak_count_exec_with_book.md", + "sha256": "4d09a3b744b0f8559ba54321e4b852dd65abad1ef487a206f8d2ef839f638e78", + "size_bytes": 17900, + "source_mtime_utc": "2026-07-21T22:33:20.979005+00:00" + }, + { + "name": "eval_volume_peak_count_exec_with_book_dashboard.png", + "sha256": "a5993d0637a158d21e1746545a91846100ed102c23af91272862276d2f3b5476", + "size_bytes": 676724, + "source_mtime_utc": "2026-07-21T22:33:20.983557+00:00" + } + ] +} diff --git a/docs/factors/d5_runner_difference_catalogue.md b/docs/factors/d5_runner_difference_catalogue.md new file mode 100644 index 0000000..83dcdd9 --- /dev/null +++ b/docs/factors/d5_runner_difference_catalogue.md @@ -0,0 +1,267 @@ +# D5 C2 —— 11 个 eval runner 的差异编目（统一前置） + +> **状态**：D5 交付物 C2。设计出处 v3.2 §六.5：11 个 runner 的差异**先编目再合并**， +> 合并后按 §五语义对账逐项归因——**「未编目的差异」算失败**。 +> 本表是 **C5 对账的差异白名单唯一来源**：对账中出现表外差异 = 验收不通过。 +> 编目方法：全 11 份 AST body 比对 + 11 份 config 展平 YAML 全量比对 + 逐个分歧区精读。 +> 路径均相对仓库根；行号为编目时实测。 + +**总量**：11 个 runner 共 6550 行（496–749 行/个）。 + +--- + +## 一、结论：只需要四个扩展点 + +| # | 扩展点 | 为什么不能做成固定 body | +|---|---|---| +| 1 | **因子构造 + per-symbol reducer，kwargs 不透明（2–9 个）** | 各因子预注册参数个数与名字都不同；且 reducer 返回**三种形态** | +| 2 | **可选的 post-loop 截面 finalizer** | 2 个因子的值**不是**逐 symbol 产出的；其中 1 个还要日频面板 | +| 3 | **可选的 add-Section 覆盖披露** | 4 个 runner 各有一份**形状不同**的稀缺性披露，且这是**故意的** | +| 4 | **metric 键集** | 三档 19/27/29，但**与因子无关**（见下），可收敛 | + +其余全部可归一（§四 C1–C13）。 + +## 二、(a) 真参数 —— 必须作为 runner 参数存活 + +### 2.1 per-symbol reduction 形态（**最承重的一条**） + +统一 runner **不能假设 per-symbol reducer 返回因子值**： + +| 形态 | runner 数 | 成员 | +|---|---|---| +| 纯 series | 6 | jump_amount_corr / minute_ideal_amplitude / amp_marginal_anomaly_vol / volume_peak_count / peak_interval_kurtosis / valley_relative_vwap | +| series + diagnostics sink | 3 | valley_ridge_vwap_ratio / ridge_minute_return / peak_ridge_amount_ratio | +| **stats frame + 强制截面 finalizer** | 2 | **intraday_amp_cut / valley_price_quantile** —— 循环**根本不产出因子** | + +### 2.2 post-loop 截面 finalizer（2 个） + +- `qt/eval_intraday_amp_cut.py:172-177` → `combine_amp_cut_cross_section(stats, min_cross_section=…)`（研报第 4 步截面 z-score）。其覆盖率判定测的是 `V_MEAN_COL` **与** `V_STD_COL` 双 finite（`:155`），不是"series 非 NaN"。 +- `qt/eval_valley_price_quantile.py:281-284` → `reversal_20(panel[["close"]])` + `residualize_on_reversal(raw, rev, min_cross_section=…)`。 + +> ⚠️ 与 D5 饱和探测（`docs/factors/d5_saturation_feasibility.md`）**直接耦合**： +> `intraday_amp_cut` 的截面步正是 per-symbol 流式化切口必须落在其**之前**的原因。 + +### 2.3 需要日频面板的 loader（1 个） + +`_load_valley_price_quantile_panel(cfg, symbols, spec, panel, logger, *, …)` +（`qt/eval_valley_price_quantile.py:202-207`，调用点 `:598`）——**唯一** 5 位置参数 loader， +其余 10 个都是 4 个。reversal 取自前复权面板 `panel['close']` 的 T-1。 +→ **reducer 签名必须能接住日频面板。** + +### 2.4 因子构造参数（全部为模块常量，无一 config 驱动；kwargs 个数 2–9，不能做固定列表） + +| runner | Factor 类 | lookback | 额外 loader kwargs | +|---|---|---|---| +| jump_amount_corr | JumpAmountCorrFactor | 20 | `min_pairs=10` | +| minute_ideal_amplitude | MinuteIdealAmplitudeFactor | 10 | `lam=0.25, min_minutes=1150` | +| amp_marginal_anomaly_vol | AmpMarginalAnomalyVolFactor | 20 | `min_pool=460, min_selected=20, sigma_k=1.0` | +| volume_peak_count | VolumePeakCountFactor | 20 | PRV-5 | +| intraday_amp_cut | IntradayAmpCutFactor | 10 | `lam=0.20, min_day_minutes=100, min_valid_days=6, min_cross_section=10` | +| peak_interval_kurtosis | PeakIntervalKurtosisFactor | 20 | PRV-5 + `min_intervals=20` | +| valley_relative_vwap | ValleyRelativeVwapFactor | 20 | PRV-5 + `min_valley_bars=20` | +| valley_ridge_vwap_ratio | ValleyRidgeVwapRatioFactor | 20 | PRV-5 + `min_valley_bars=20, min_ridge_bars=10` | +| ridge_minute_return | RidgeMinuteReturnFactor | 20 | PRV-5 + `min_ridge_bars=10` | +| valley_price_quantile | ValleyPriceQuantileFactor | 20 | PRV-5 + `min_valley_bars=20, min_cross_section=10, reversal_days=20` | +| peak_ridge_amount_ratio | PeakRidgeAmountRatioFactor | 20 | PRV-5 + `min_peak_bars=5, min_ridge_bars=10` | + +PRV-5 = `baseline_days=20, baseline_min_obs=10, sigma_k=1.0, min_valid_days=10, min_classifiable=100` +（`factors/compute/minute/primitives.py:55-59`，8 个 runner 共享）。 +**这些是预注册定义，参数化保留、不统一**（v3.2 §〇 总原则）。 + +## 三、(b) 真异构披露 —— 走 add-Section 扩展点 + +4 个 runner 各有一份稀缺性/中性化披露，**无共享基类**，4 个独立 frozen dataclass 各带 `render()`。 +两种机制： + +| runner | 类 | 定义行 | 机制 | +|---|---|---|---| +| valley_ridge_vwap_ratio | `RidgeCoverage` | `:110-233` | A：循环中 diagnostics sink（`:310`） | +| ridge_minute_return | `RidgeReturnCoverage` | `:120-253` | A：sink（`:328`） | +| peak_ridge_amount_ratio | `PeakCoverage` | `:122-243` | A：sink（`:320`） | +| valley_price_quantile | `NeutralizationCoverage` | `:121-182` | B：循环后从已装配面板算（`:285`），无 sink | + +**它们不是同一对象换字段名**——共有 `symbol_days / classifiable_days / valid_days / +days_below_classifiable_gate / min_classifiable`，其余各不相同： + +- `RidgeCoverage`：`ridge_percentiles / ridge_mean / valley_median`；两道 gate；反事实 `valid_days_at_valley_floor` +- `RidgeReturnCoverage`：另**追踪两个计数** `ridge_bars_mean/median`；额外属性 `return_guard_attrition`；反事实 floor `_COMPARISON_FLOOR = 20`（**硬编码**对齐 PR-J） +- `PeakCoverage`：反事实 floor `_COUNTERFACTUAL_PEAK_FLOOR = PEAK_RIDGE_MIN_RIDGE_BARS`（**派生**，非硬编码） +- `NeutralizationCoverage`：形状完全不同——`raw_rows/rev_rows/residual_rows/dates_total/dates_residualized/cross_section_min,median,max/raw_rev_spearman_mean` + +**披露措辞**（4 条 format 模板，同时进 run log 与 stdout）： +`eval_valley_ridge_vwap_ratio.py:151-168` / `eval_ridge_minute_return.py:173-189` / +`eval_peak_ridge_amount_ratio.py:164-177` / `eval_valley_price_quantile.py:293-301`。 +各类 docstring 均强调：**报出的 floor 必须是本次 run 实际施加的那个**，不是模块默认值。 + +**判定 (b) 而非 (c)**：这些是各因子**自己 gate 的实测事实**，量纲与语义都不同， +强行统一 = 丢信息。这正是 §3.6 的 add-Section 扩展点（"may ADD sections but never +drop a mandatory one"），**不是第二套 runner**（kill 档案 A8-F03）。 + +**一处归一**：PR-L 在 `qt/cli.py:525-530` 用**内联 f-string** 重渲染，而其三个兄弟走 +`.render()`（`:436` / `:479` / `:570`）→ 统一走 `.render()`。 + +## 四、全 11 份完全一致（可做固定 body，无参数） + +| 轴 | 值 | 证据 | +|---|---|---| +| **horizon** | `h = 1`，且**根本不是 runner 关切** | 来自 `FactorSpec.forward_return_horizon`，仅在共享模块 `qt/exec_basis_eval.py:213` 消费。**不要加 horizon 旋钮** | +| **`execution_capacity` 透传** | **零个 runner 设置**；仅共享模块 `qt/exec_basis_eval.py:305`(no_book) / `:315`(with_book)，同一 `disclosure` dict（`:97-151`） | 该 dict **故意省略** `tradable`/`capacity_sufficient` 两个 Tradable 轴会读的键 → 该轴恒 NOT_ASSESSED，模块 docstring `:21-27` 明写 | +| **因子簿** | `ValueFactor("value_ep"), ValueFactor("value_bp"), VolatilityFactor(window=20)` | `eval_jump_amount_corr.py:480-488`，**11 份 AST body 逐字节相同** | +| **费率/成本** | `fee_rate = cfg.cost.fee_rate`（配置 0.001）；`cost_scenarios = (1.0, 2.0, 4.0)` 11 份硬编码相同 | `qt/exec_basis_eval.py:304, 313` | +| **分钟读取窗口** | 11 份相同：`[data.start, data.end 23:59:59]` | **无 runner 向前扩窗做 warm-up** → 前 lookback_days 结构性 NaN（与 D4 饱和工作直接相关） | +| **process/中性化** | 因子一次 + 因子簿一次 `_process_factors`；`neutralization=("industry","size"), standardize="zscore", winsorize=None` | 唯一例外是 PR-L 的 reversal 残差化，发生在 loader **内部**、`_process_factors` **之前** | +| **报告/图输出** | `{stem}_no_book|_with_book` .md+.json + 2 PNG；exec 侧同形 | `eval_jump_amount_corr.py:221-228`；`qt/exec_basis_eval.py:321-330` | +| **`_exec_basis_sanity.md`** | **仅**共享模块 `qt/exec_basis_eval.py:271-283` 产出，11 份无差异，无 runner 代码触碰 | 渲染器 `qt/exec_basis_sanity.py:355+` | +| **config** | **D1 manifest 的声称经复核成立**：11 份展平 YAML 共 56 个 union 键，**恰有两个不同**：`project.name` 与 `data.output_name` | 其余逐值相同（CSI500 / 2021-07-01..2026-06-30 / split 2024-01-01 / fee 0.001 / quantiles 5 / top_n 50 / L1 / cache root / factors 列表） | + +**`execution_capacity` 附注**：CLAUDE.md 把「`execution_capacity` 透传是否在作者自带测试之外 +仍保持 Tradable 轴 `NOT_ASSESSED`」列为 **#79 未独立核**三项之一。本次编目**关闭了这一项的 +结构面**：透传是**单一共享代码路径、无 runner 变体、且不含任何能动 verdict 的键**， +故该性质**按构造**不可能逐因子变化。（另两项仍未核。） + +### metric 键集：三档，但**与因子无关** + +19 键（8 个）/ 27 键（ridge_minute_return，`:490-519`）/ 29 键（valley_price_quantile `:436-465`、 +peak_ridge_amount_ratio `:488-517`）。**多出的键全部取自每次 run 都产出的 `return_risk` / +`stability_cost` payload**——即「PR-K 之后开始多报几个」的**报告选择**，不是因子性质。 +→ **收敛到 29 键超集 减去 `aligned_spread_by_cost`**（该键不存在，见 BUG 2）。 + +## 五、(c) 无意义漂移 —— 已判定归一，逐条记录 + +| # | 漂移 | +|---|---| +| C1 | `_build_eval_config` 11 份不同 body、语义全同；差别仅 `oos is None` 错误串里的 runner 名。17 个 EvalConfig kwargs 逐字符相同 | +| C2 | `_check_preconditions` 同上，同 4 项检查，差别仅嵌入的 runner 名 | +| C3 | 11 个 result dataclass 名不同，14–16 字段相同 | +| C4 | 11 个私有 load-dataclass 名不同 | +| C5 | 局部变量命名 `jump_raw` / `amp_raw` / `factor_raw` | +| C6 | 11 个 `_load_*_panel` 函数名不同，11 行骨架相同 | +| C7 | 日志措辞（`rows read` 换行位置；`symbols with a value` vs `with a stat`） | +| C8 | `logger.info("eval config: …")` 单行 vs 折行 | +| C9 | `_write_report` 声明单行 vs 四行折行，body 相同 | +| C10 | docstring 88 vs 90 列重排——**任何两份 byte-diff 的主体其实是这个** | +| C11 | 仅 4 个大 runner import numpy（只被覆盖率 summarizer 用） | +| C12 | `qt/cli.py:206-580` 11 份 `_cmd_run_eval_*`，同 try/except + f-string | +| C13 | 两份相同 `_fmt` helper：`qt/cli.py:395-409` 与 `qt/exec_basis_eval.py:355-367` | + +## 六、发现的缺陷（**已标记，未修**） + +> 按 lead 指令「flag separately，do not fix」。以下 BUG 1/2 由本 agent **独立复核确认** +> （非仅采信编目），复核方式记在各条。 + +### BUG 1（CRITICAL，用户可见）—— runner 描述了一个 **PR #74 已经修掉**的缺陷 + +**声称**（`qt/eval_ridge_minute_return.py:477-481`，并在 `:49-56`、`:705-708` 复述）： +> "The frozen layer's `aligned_spread_*` computes `sign * (gross - cost)` … costs are added +> back rather than deducted … **The frozen layer is not patched**" + +**实际**（`analytics/eval/standard.py:419`，本 agent 直接读取确认）： +``` +aligned_base = sign * gross - base_cost +``` +上方注释以**过去时**描述旧缺陷（"The pre-v0.8 form `sign * net` expanded, at sign=-1, to +`-gross + cost`"）——即 PR #74 的 v0.8 修复**已在位**。 + +**且它每次 PR-K run 都打到 stdout**（`qt/cli.py:443` +"net long-short by cost (aligned_spread_* UNRELIABLE at sign=-1)"）并写进 run log +（`eval_ridge_minute_return.py:708`）。另有四处断言同一死声称的否定形式： +`qt/cli.py:484-485`、`:535-536`、`eval_valley_price_quantile.py:57-59` 与 `:422-424`、 +`eval_peak_ridge_amount_ratio.py:475`。 + +**祖先关系已验**：`08829bd`（v0.8 cost-correct aligned spread）**既是** HEAD 祖先、 +**也是** `36c8c85`（最后触碰 ridge runner 的 commit）的祖先 → **该文字在最后一次被编辑时 +就已经是陈旧的**。 + +**这正是 #76/#78/#82 的失效模式**：行为改了、措辞没改。按项目自定规则，修法是 +**author-once（只写一遍、其余组合引用）**，不是再复制一份正确措辞。 +→ **归属**：本条与 D5 统一 runner 天然同批（11 份措辞坍缩成一份），但**它是独立缺陷， +不因 D5 而存在**；若 D5 拆期，本条应单独修。 + +### BUG 2（HIGH）—— `aligned_spread_by_cost` 被提取，而 frozen 层从不产出它 + +`eval_valley_price_quantile.py:461` 与 `eval_peak_ridge_amount_ratio.py:513` 都做 +`dict(ret_risk.get("aligned_spread_by_cost", {}) or {})`，但 `analytics/` 全仓**不存在**该键 +（只有 `aligned_spread_annual_return|sharpe|volatility|max_drawdown|sortino|final_nav` +与 `net_long_short_by_cost`）。 + +**本 agent 独立复核**：直接读**冻结的 exec artifact** 的 `return_risk` payload， +两个因子的 `aligned_spread_by_cost` 均 **不存在**；实际 `aligned_*` 键恰为上述六个。 +→ 两个 runner 报的**永远是 `{}`**。 + +**唯一"守卫"是一个不可能失败的测试**： +`tests/test_eval_valley_price_quantile_runner.py:485` +`assert isinstance(m["aligned_spread_by_cost"], dict)` —— `{}` 是 dict，**无条件通过**。 +正是项目方法论 ① 点名的形态。 + +### BUG 3（HIGH）—— `qt/eval_peak_ridge_amount_ratio.py` **完全没有 runner 测试** + +11 个里 10 个有 `tests/test_eval__runner.py`；PR-M 没有 +（`grep -rn "qt.eval_peak_ridge_amount_ratio" tests/` 无命中）。 +后果：`PeakCoverage` + `summarize_peak_coverage`（`:122-243`，**122 行披露逻辑**） +**零测试覆盖**，而三个兄弟 summarizer 都有反事实 + 空帧测试。 +`tests/test_peak_ridge_amount_ratio_factor.py` 只覆盖因子数学。 + +### BUG 4（MEDIUM）—— PR-M 的 run log 比终端输出更贫 + +它提取全 29 键、CLI 也打印 net-spread/pearson/monotonicity（`qt/cli.py:486-489`）， +但不像 PR-K（`:708`）/ PR-L（`:654,:660`）那样发对应 `logger.info`。 + +### BUG 5（LOW）—— config 声明了一份 runner 根本不读的因子簿 + +每份 config 都有 `factors:[value_ep,value_bp,volatility_20]`，与硬编码 +`_build_book_factors()` 相同，但**无 runner 读 `cfg.factors`**。今天两者一致故无害， +但**改 config 会静默无效**。同形：`backtest.rebalance='monthly'` 而 +`_build_eval_config` 硬编码 `rebalance='daily'`；整个 `backtest:/portfolio:/alpha:` 块对这些 run 惰性。 + +### BUG 6（LOW / 近失）—— 重名测试函数，**目前**两份都被收集 + +`test_summarize_ridge_coverage_handles_no_frames` 同时定义于 +`tests/test_eval_ridge_minute_return_runner.py:246` 与 +`tests/test_eval_valley_ridge_vwap_ratio_runner.py:232`。 +**因为 `tests/__init__.py` 存在**，两份都被收集（实测该两文件收 10+7=17）。 +→ ⚠️ **对 C6 的直接警告**：若统一 runner 的测试整合过程中删掉 `tests/__init__.py`， +这条会立刻变成静默丢测试。 + +### BUG 7（LOW）—— `eval_ridge_minute_return.py:471` 自称"Beyond the fields PR-C..PR-J surfaced"， +但 PR-J（valley_ridge_vwap_ratio）属 19 键基础档，它点名的对照集在自己的边界上就是错的。 + +## 六之二、C6 爆炸半径：谁 import 这 11 个 runner（**删之前必须处置**） + +严格说这不是「runner 之间的差异」，但删除动作的影响面必须落在纸面上——与 BUG 6 +（`tests/__init__.py` 收集陷阱）同一接口。**非测试模块共四个**（实测 +`grep -rln "qt\.eval_" --include=*.py | grep -v ^./tests/`）： + +| 模块 | 用途 | C6 处置 | +|---|---|---| +| `qt/cli.py` | 11 个 `_cmd_run_eval_*` 子命令（§五 C13） | 随统一 runner 收敛为一个子命令 | +| **`qt/panel_freeze.py`** | **生产 D1 冻结基线的工具**——调各 runner 私有 `_load_*_panel` + `_build_book_factors`，「零公式重抄」正是靠 import 它们实现的 | **删 runner 会打断「重新生成 / 重新验证 D1 基线」的能力** | +| **`qt/panel_reconcile.py`** | D2 逐格对账工具（同上依赖） | 同上 | +| **`qt/hand_anchors_engine_values.py`** | D2 手算锚的引擎侧取值 | 同上 | + +⚠️ **后三个全是 D1/D2 的验收工具**，而 **D5 面板腿的比较对象正是 `panel_freeze.py` 的产物**。 +按设计 v3.2 §五第 4 腿的 **provenance 规则**，基线只许从钉住的 pre-D2 SHA 重新生成—— +所以严格说 C6 之后从**当前树**重跑它们本来就不合法。但「工具还在、只是不该从这里跑」与 +「工具已被删、想复核也无从下手」是两回事：**这与 C1 存在的理由是同一种不对称损失** +（复核能力一旦丢失，代价远大于保留一份不再调用的代码）。 + +**处置要求（C6 执行时逐条落实，不得默认删掉了事）**：① 三个工具**要么保留**（连同它们 +import 的 runner loader，即便统一 runner 已上线）、**要么显式记录**「D1/D2 基线自此不可从 +本仓再生，只能依赖已冻结的 artifact + manifest 哈希」并让 lead 知情裁定； +② 无论哪种，`docs/factors/d1_panel_freeze_manifest.md` 的「重跑命令」一节都会变成陈旧描述， +**必须同批更新**——否则就是本项目 #76/#78/#82 那条形态的又一次复发（文档教人跑一条已经 +跑不了的命令）。 + +## 七、与 C5 对账的接口 + +**本表 §二/§三 = 允许出现的差异白名单；§四 = 必须逐值一致的项；§五 = 已判定归一（对账中 +不应产生数值差异）。** + +另有一条**已知且必然**的 artifact 差异，来自 D1/D3 而非 D5，**必须计入白名单** +（本 agent 独立复核确认）： + +> **`spec` 块键数 16 → 20**。冻结 artifact 的 `spec` 有 16 键，当前代码产出 20 键， +> 新增恰为 `requires` / `adjustment` / `overnight_boundary` / `lookback_depth`，**零删除**。 +> 成因：`render.py:156` 的 `sanitize_payload(vars(report.spec))` 会把 FactorSpec 的**任何** +> 新字段自动带进 JSON，而这四个字段是 PR #86（契约 v1.0）与 PR #91（v1.1 `lookback_depth`） +> 加的。`requires` 经 `clean_value` 的 `str()` 兜底渲染成 **repr 字符串列表**。 +> **属预期漂移，非回归**——但若不预先登记，会在对账时被当成 D5 引入的差异去追。 diff --git a/docs/factors/d5_saturation_feasibility.md b/docs/factors/d5_saturation_feasibility.md new file mode 100644 index 0000000..12ee7a1 --- /dev/null +++ b/docs/factors/d5_saturation_feasibility.md @@ -0,0 +1,147 @@ +# D5 前置探测：D4 materializer 能否跑真实评估 universe + +> **状态**：D5 第二个交付物，**先于任何长 run**（lead 指令：「宁可报『这条路走不通、原因如下』， +> 也不要跑一个 2 小时后失败或产出错值的 run」）。 +> 复验：`python -m qt.saturation_probe`（cache-only，实测 `live_calls=0`）。 +> ⚠️ **前置条件**：需要 gitignored 的 `artifacts/`（分钟缓存 + D1 冻结面板）。**在没有它的 +> worktree 里开箱跑不起来**——先对主 checkout 的 `artifacts` 建符号链接，或用 +> `--cache-root` / `--universe-panel` 指向绝对路径（先例：`qt/factor_hotpath_smoke.py`）。 +> 建链接时注意：worktree 里若已存在真实 `artifacts/` 目录，`ln -s` 会嵌套成 +> `artifacts/artifacts`（本步实际踩到过）。 +> 全部数字为**实测**，标注为外推的才是外推。 + +--- + +## 结论先行 + +**D4 的 materializer 跑不了 D5 的评估 universe。** 不是慢，是 **2.19× 于本机可用内存**。 +它在 40 票热路径 smoke 上验收，评估面是 **995 票**，而它把整个 universe 装进**一个** frame。 + +且这不止影响 pooled 因子：**bounded 路径同样整 universe 一次性加载** +（`factors/materialize.py:262`），评估窗口本身就要 **52.7 GB**，已经贴着 56 GB 可用内存。 +所以「per-symbol 流式化」不是 pooled 因子的局部优化，**是 11 个因子全都需要的结构前提**。 + +--- + +## 一、饱和为什么必然走到 floor（不是偶发，是构造性的） + +`_pooled_pool_saturated` 要求**每一个被请求的 symbol** 在锁定子窗口里、在 `emit_start` +当天或之前攒够 `lookback_days` 个有效日，否则否决终止（D4 review HIGH 的修法，正确且 +故意保守）。 + +**实测**：评估 universe 995 票中，**84 票的第一根 1min bar 在 `emit_start` 2021-07-01 +当天或之后**（最晚到 2026 年）。这些票**无论往回加载多深都攒不出 emit_start 之前的有效日** +——因为那时它们还没上市。于是判据永远为 False，循环一路扩到声明 floor `2015-01-05`。 + +| 首根 bar 年份 | 票数 | +|---|---| +| 2015 | 668 | +| 2016–2020 | 223 | +| **2021（含 ≥07-01 的）** | 39 | +| **2022–2026** | **65** | + +这是 CSI500 的**正常构成**（指数持续纳入新上市公司），不是数据缺陷。**换言之：只要 +universe 里有一只在窗口内上市的票，pooled 加载就一定打到 floor。** 对 CSI500 五年窗口， +这是必然而非可能。 + +## 二、代价（实测 + 线性外推，外推处已标注） + +**磁盘**（精确，遍历 month 分区）： + +| 范围 | 分区文件 | 字节 | +|---|---|---| +| 评估窗口 2021-07..2026-06 | 57,542 | **8.18 GB** | +| floor 深度 2015-01..2026-06 | 117,971 | **16.43 GB**（2.01×） | + +**内存/耗时**（20 票**实测**，× 49.75 **外推**——单帧 concat 的线性外推在这里是保守的， +真实 concat 峰值更高）： + +| 加载 | 实测（20 票） | 外推（995 票） | +|---|---|---| +| 评估窗口 | 5.69M 行 / **1.06 GB** frame / 峰值 RSS 2.50 GB / 5.3s | 283M 行 / **52.7 GB** / ~4.4 min | +| floor 深度 | 13.21M 行 / **2.46 GB** frame / 峰值 RSS 5.57 GB / 12.7s | 657M 行 / **122.2 GB** / ~10.5 min | + +本机 `free`：总 62 GB，**可用 56 GB**。 + +- floor 深度 **122.2 GB / 56 GB = 2.19×** → **OOM，不可行**。 +- 评估窗口 **52.7 GB / 56 GB = 0.94×** → 名义装得下，但那是**因子计算开始之前**的裸 bar + 帧，且 8 个 pooled 因子每个都要一次。**实际同样不可行。** + +**为什么旧 runner 没这个问题**：11 个旧 runner 的 `_load_*_panel` **逐 symbol** 读、读完 +立刻聚合到日频，**从不同时持有整个 universe 的分钟 bar**。D4 的 materializer 改成了整批 +加载——这是 D4 引入的新形态，在 40 票上看不出来。 + +## 三、修法：per-symbol 流式化，且切口必须在 cross-section combine **之前** + +**实测的决定性证据**（12 票真实数据，整 universe 算 vs 逐 symbol 算再拼）： + +| 因子 | index 相同 | 可比 cell 数 | max\|diff\| | NaN 集合差异 | 判定 | +|---|---|---|---|---|---| +| `ridge_minute_return_20`（pooled） | True | **728** | **0.000e+00** | **0** | **SAFE** | +| `volume_peak_count_20`（pooled） | True | **1524** | **0.000e+00** | **0** | **SAFE** | +| `intraday_amp_cut_10` | True | **0** | **n/a（无可比 cell）** | **1692（全部行）** | **NOT SAFE** | + +前两个在 728 / 1524 个**真实可比 cell** 上逐位相同——per-symbol 纯性在真实数据上得到确认 +（不是靠既有 isolation 测试推断，也不是在空集合上真空通过）。 + +⚠️ 第三行的 `max|diff|` **必须显示 `n/a` 而不是 `0.000e+00`**（评审 NIT，已改）：per-symbol +一侧全是 NaN，两边**没有任何一个 cell 可比**，而"0.000e+00"紧挨着 "NOT SAFE" 会被读成 +「值其实一致、只是 NaN 标记不同」——恰好相反。`compared=0` 这一列就是为了让"没得比" +无法被误读成"比过了且相等"。 + +`intraday_amp_cut` **全部 1692 行翻成 NaN**：它的第 4 步是**按日截面 z-score**，要求当日 +finite 对数 ≥ `AMP_CUT_MIN_CROSS_SECTION=10`，而**单票截面 n=1 恒 < 10 → 定义上全 NaN**。 +这正是 D4 docstring 警告的耦合，实测复现。 + +**因此切口在哪里是有对错的**： + +- ❌ 把**整个因子**逐 symbol 算再拼 → `intraday_amp_cut` 全毁。 +- ✅ 逐 symbol 算到**该因子的 per-symbol 中间量**，拼成全 universe 面板，**再做那一次 + 截面 combine**。`intraday_amp_cut` 的模块结构本来就是这样组织的 + （`factors/compute/minute/intraday_amp_cut.py` docstring：先出两列 `(V_mean, V_std)` + 面板，**再 `combine_amp_cut_cross_section` 一次**），旧 runner 亦然。 + +### 与 `AMP_CUT_MIN_CROSS_SECTION` 耦合的显式论证（lead 要求单独论证，不许当纯优化） + +D4 的警告原文是「per-symbol 加载深度会改 `intraday_amp_cut` 的截面 z-score 组成」。 +**该警告针对的是 per-symbol *截断*，不是 per-symbol *饱和***，两者必须分开： + +- **per-symbol 截断**（每票只加载「它大概需要」的深度）：某票在日 d 本应 finite 却因加载 + 过浅变 NaN → 当日截面成员变少 → z-score 分母变、`min_cross_section` 门可能翻 → + **确实改值**。这条 D4 警告成立，**不可做**。 +- **per-symbol 饱和**（每票加载到**它自己**的饱和/floor，判据仍是 D4 那条结构性判据， + 只是逐票判而非全 universe 齐步走）：每票的 per-symbol 中间量**等于**它在全 universe + floor 加载下的值（上表前两行逐位实测 + 既有 cross-symbol isolation 性质）→ 日 d 的 + finite 成员集合**不变** → 截面 combine 输入不变 → **值不变**。 + +**即：把「全 universe 一起扩到 floor」换成「每票各自扩到自己的 floor」，对每票的 +saturation *终点* 没有放松**（84 票里任何一票该到 floor 的仍到 floor），只是不再强迫 +另外 911 票陪着一起到 floor。**这不是把判据放宽，是把判据从「universe 齐步」改成 +「逐票各自」，而逐票判据的终点集合与齐步判据完全相同。** + +⚠️ **仍属定义相邻、必须验收**：上述论证的承重前提是「per-symbol 中间量与加载几何无关」。 +该前提对 10 个 per-symbol 纯因子已实测（0.0 逐位）；对 `intraday_amp_cut` 需要**单独** +证明其 `(V_mean, V_std)` per-symbol 部分同样与几何无关，并用**全 universe floor 加载 +vs per-symbol 饱和加载**在小 universe 上做端到端逐格对账。**该对账未完成前，不得把 +per-symbol 饱和当作已验证的等价改写。** + +## 四、对 D5 计划的影响 + +1. **C5 的全量 run 在修好之前不可能执行**——不是「慢」，是 OOM。任何「先跑起来看看」 + 都会在 10 分钟的加载后死掉，或更糟：在评估窗口那档勉强不死而开始 swap。 +2. 修法（per-symbol 流式 + 切口在 combine 之前）**在 D4 已声明为 D5 工作** + （`materialize.py:397` "A per-symbol saturation start is the D5 optimization"）， + 故属 D5 范围，但**它是修改因子取值路径的改动，必须按 §五四腿验收**， + 不能作为「让 run 跑起来」的顺手改动混进去。 +3. `valley_price_quantile` 的 D5 绑定（C4 交付物）**继承同一路径**——lead 已明令 + 不许用固定深度 trim 绕过，per-symbol 饱和是它唯一合法的加载形态。 + +## 五、未做 / 留给后续 + +- **per-symbol 饱和的实现与四腿验收**：本探测只证明了「必须这么做」与「这么做在两个 + pooled 因子上逐位等价」，**没有实现它**，也没有对 `intraday_amp_cut` 做端到端对账。 +- 声明 floor 目前仍是硬编码常量 `qt/factor_hotpath_smoke.py::CACHE_MINUTE_DATA_START + = "2015-01-05"`，对所有 symbol 同值。**探测显示按票派生 floor 是有意义的**（668 票 + 确实从 2015 开始，但 327 票晚得多），但**改 floor 派生方式本身**同样是定义相邻变更， + 与 per-symbol 饱和是两件事，不要合并处理。 +- 本探测未测 `concat` 峰值与 GC 行为，线性外推对峰值是**乐观**的。 diff --git a/qt/exec_baseline_freeze.py b/qt/exec_baseline_freeze.py new file mode 100644 index 0000000..cee3041 --- /dev/null +++ b/qt/exec_baseline_freeze.py @@ -0,0 +1,450 @@ +"""D5 exec-basis evaluation baseline freeze — the ONLY reconciliation reference. + +The 22 exec-basis evaluation artifacts (11 factors x {no_book, with_book}), their +22 rendered Markdown reports, the 11 ``_exec_basis_sanity.md`` coverage +disclosures and the 22 dashboards live in ``artifacts/reports/`` and are +**gitignored, single-copy, and not reproducible from current code**: regenerating +them needs a checkout of the pre-D5 tree plus a ~1.5h real run. The unified D5 +runner writes to those same filenames. Running it once would therefore destroy +the only thing D5 can be judged against, and any "reconciliation" performed +afterwards would compare the new output with a copy of itself — the +``compare_postmerge.py`` empty-reconciliation failure mode this repo has already +committed once and recorded in CLAUDE.md. + +So: freeze first, reconcile against the frozen copy, never against the live path. + +Structural isolation (NOT discipline) +------------------------------------- +``FrozenExecBaseline`` is the only supported reader, and it cannot be aimed at +the live artifacts directory even by accident: + +1. it requires a git-tracked manifest describing the tree it is given; +2. **every file it returns is sha256-verified against that manifest at read + time** — so a live artifact copied over a frozen one is a loud failure, not a + silent substitution. This is the load-bearing guard: it does not ask whether + the *path* looks right, it asks whether the *bytes* are the frozen bytes; +3. it refuses outright to resolve to the live reports directory. + +The manifest lives in git (``docs/factors/d5_exec_baseline_manifest.json``) while +the bytes live under gitignored ``artifacts/``. That split is deliberate and is +the reason (2) has teeth: an attacker-in-the-form-of-a-tired-engineer who +overwrites the frozen tree cannot also silently move the hashes, because the +hashes are under version control and show up in ``git diff``. + +Both entry points are BYTE-idempotent on an unchanged tree: re-running them +writes nothing, including the manifest, so ``git diff`` stays clean. Provenance +fields (``frozen_at_utc`` / ``frozen_at_git_head`` / ``source_note``) are +inherited from an existing manifest rather than restamped — a verification run +must not be able to re-date the history it is verifying. + +Usage (needs the gitignored ``artifacts/`` tree; in a worktree without one, pass +``--repo-root`` pointing at a checkout that has it):: + + python -m qt.exec_baseline_freeze # freeze; refuses to clobber + python -m qt.exec_baseline_freeze --verify # re-verify the frozen tree, copy nothing + +A FIRST freeze (no manifest to inherit from) must pass ``--source-note``: the +committed manifest is asserted to carry a non-empty provenance note, so a +freeze without one is fail-closed at test time rather than at review time:: + + python -m qt.exec_baseline_freeze --source-note "" + +Re-freezing an unchanged tree inherits the existing provenance and writes no +bytes. Deleting the frozen tree and re-freezing from a DIFFERENT live source +would carry the old note onto new bytes — the 77 hash lines in the manifest +diff make that loud, but pass ``--source-note`` when the source really changed. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import subprocess +import sys +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path + +#: Repository-relative default locations. +DEFAULT_REPORTS_DIR = "artifacts/reports" +DEFAULT_FROZEN_ROOT = "artifacts/refactor_baseline/exec_baseline" +DEFAULT_MANIFEST = "docs/factors/d5_exec_baseline_manifest.json" + +#: Manifest schema tag. Bumping it invalidates every reader — intentional. +MANIFEST_SCHEMA = "d5-exec-baseline-freeze/1" + +#: Only the exec-basis family is frozen. The close-basis artifacts are NOT the +#: D5 reference (the unified runner is exec-only, design v3.2 decision 4) and +#: are left alone. +EXEC_MARKER = "_exec_" +FILENAME_PREFIX = "eval_" +FROZEN_SUFFIXES = (".json", ".md", ".png") + +#: The eleven closing minute factors, in the canonical order used by reports. +FACTORS = ( + "jump_amount_corr", + "minute_ideal_amplitude", + "amp_marginal_anomaly_vol", + "volume_peak_count", + "intraday_amp_cut", + "peak_interval_kurtosis", + "valley_relative_vwap", + "valley_ridge_vwap_ratio", + "ridge_minute_return", + "valley_price_quantile", + "peak_ridge_amount_ratio", +) +BOOKS = ("no_book", "with_book") + +#: Expected inventory: 22 JSON + 22 MD + 11 sanity MD + 22 PNG. +EXPECTED_FILE_COUNT = 77 + + +class BaselineIntegrityError(RuntimeError): + """Raised when frozen bytes do not match the git-tracked manifest.""" + + +def sha256_file(path: Path) -> str: + """Return the hex sha256 of a file's bytes. + + For opaque artifacts (JSON/Markdown/PNG) the byte hash IS canonical — unlike + parquet, there is no writer metadata to normalise away. + """ + digest = hashlib.sha256() + with open(path, "rb") as handle: + for chunk in iter(lambda: handle.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def expected_artifact_names() -> tuple[str, ...]: + """The 77 exec-basis artifact names this freeze is defined over. + + Derived from the factor list rather than from a directory listing, so a + missing file is a loud absence instead of a quietly smaller baseline. + """ + names: list[str] = [] + for factor in FACTORS: + names.append(f"eval_{factor}_exec_basis_sanity.md") + for book in BOOKS: + stem = f"eval_{factor}_exec_{book}" + names.extend((f"{stem}.json", f"{stem}.md", f"{stem}_dashboard.png")) + return tuple(sorted(names)) + + +def discover_exec_artifacts(reports_dir: Path) -> list[Path]: + """List the exec-basis artifacts actually present in ``reports_dir``.""" + found = [ + path + for path in sorted(reports_dir.iterdir()) + if path.is_file() + and path.name.startswith(FILENAME_PREFIX) + and EXEC_MARKER in path.name + and path.suffix in FROZEN_SUFFIXES + ] + return found + + +def _git_head(repo_root: Path) -> str: + try: + out = subprocess.run( + ["git", "-C", str(repo_root), "rev-parse", "HEAD"], + capture_output=True, + text=True, + check=True, + ) + return out.stdout.strip() + except (subprocess.CalledProcessError, OSError): # pragma: no cover - env dependent + return "unknown" + + +def _atomic_write_bytes(path: Path, payload: bytes) -> None: + """tmp + os.replace, the repository's durable-write pattern.""" + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(f".{path.name}.tmp{os.getpid()}") + try: + with open(tmp, "wb") as handle: + handle.write(payload) + handle.flush() + os.fsync(handle.fileno()) + os.replace(tmp, path) + finally: + if tmp.exists(): # pragma: no cover - only on write failure + tmp.unlink() + + +@dataclass(frozen=True) +class FreezeResult: + copied: tuple[str, ...] + already_frozen: tuple[str, ...] + manifest_path: Path + frozen_root: Path + #: False when an existing manifest was already correct and was left alone. + manifest_written: bool = True + + +def _load_manifest_if_present(manifest_path: Path) -> dict | None: + if not manifest_path.exists(): + return None + try: + return json.loads(manifest_path.read_text()) + except json.JSONDecodeError: + return None + + +def freeze( + reports_dir: Path, + frozen_root: Path, + manifest_path: Path, + *, + repo_root: Path, + source_note: str = "", +) -> FreezeResult: + """Copy the exec-basis artifacts into ``frozen_root`` and write the manifest. + + Idempotent and non-destructive: a destination that already exists is + verified against the freshly hashed source and left alone; a MISMATCH is an + error, never an overwrite. Freezing twice is safe; freezing over a different + baseline is impossible. + """ + reports_dir = reports_dir.resolve() + frozen_root = frozen_root.resolve() + if frozen_root == reports_dir or frozen_root.is_relative_to(reports_dir): + raise ValueError( + f"refusing to freeze into the live reports directory: {frozen_root}" + ) + + expected = set(expected_artifact_names()) + present = {path.name: path for path in discover_exec_artifacts(reports_dir)} + missing = sorted(expected - set(present)) + if missing: + raise FileNotFoundError( + f"{len(missing)} expected exec-basis artifact(s) absent from " + f"{reports_dir}: {missing[:5]}{' ...' if len(missing) > 5 else ''}" + ) + unexpected = sorted(set(present) - expected) + if unexpected: + raise RuntimeError( + "unexpected exec-basis artifact(s) present — the freeze inventory is " + f"defined by {len(expected)} names and refuses to guess: {unexpected}" + ) + + frozen_root.mkdir(parents=True, exist_ok=True) + entries: list[dict[str, object]] = [] + copied: list[str] = [] + already: list[str] = [] + + for name in sorted(expected): + src = present[name] + payload = src.read_bytes() + digest = hashlib.sha256(payload).hexdigest() + dst = frozen_root / name + if dst.exists(): + existing = sha256_file(dst) + if existing != digest: + raise BaselineIntegrityError( + f"{name}: frozen copy ({existing[:16]}) differs from the live " + f"artifact ({digest[:16]}). The frozen baseline is NOT " + "overwritten. Investigate before proceeding — one of the two " + "is not the PR #79 baseline." + ) + already.append(name) + else: + _atomic_write_bytes(dst, payload) + copied.append(name) + + stat = src.stat() + entries.append( + { + "name": name, + "sha256": digest, + "size_bytes": stat.st_size, + "source_mtime_utc": datetime.fromtimestamp( + stat.st_mtime, tz=timezone.utc + ).isoformat(), + } + ) + + # PROVENANCE IS INHERITED, NEVER SILENTLY RESTAMPED. + # + # An earlier version rebuilt these three fields on every invocation, so the + # documented "re-run me to check the freeze" command quietly repointed + # frozen_at_git_head at whoever re-ran it and blanked source_note — while + # reporting `copied: 0` and passing every test. That turned the freeze tool + # into a generator of the exact false provenance claim it exists to prevent + # (the #76/#78/#82 shape: the behaviour changed, the wording did not). + # + # So: an existing manifest supplies these unless the caller explicitly + # overrides them. Re-freezing verifies bytes; it does not re-date history. + # Inherit only what is actually there: a manifest that never carried + # provenance has nothing to preserve, and inheriting "" from it would put an + # empty field where the always-run manifest assertions expect a real one — + # trading a restamp for a silent blank. + previous = _load_manifest_if_present(manifest_path) or {} + frozen_at_utc = str(previous.get("frozen_at_utc") or datetime.now(timezone.utc).isoformat()) + frozen_at_head = str(previous.get("frozen_at_git_head") or _git_head(repo_root)) + note = source_note or str(previous.get("source_note") or "") + + manifest = { + "schema": MANIFEST_SCHEMA, + "frozen_at_utc": frozen_at_utc, + "frozen_at_git_head": frozen_at_head, + # Logical, repo-relative locations. The on-disk paths resolve through a + # gitignored symlink on some checkouts; recording the resolved absolute + # path would make this git-tracked manifest machine-specific. + "source_dir": DEFAULT_REPORTS_DIR, + "frozen_root": DEFAULT_FROZEN_ROOT, + "file_count": len(entries), + "factors": list(FACTORS), + "books": list(BOOKS), + "source_note": note, + "files": entries, + } + # BYTE-idempotent: an unchanged tree rewrites nothing at all, so re-running + # the documented verification command leaves a clean `git diff`. + manifest_written = manifest != previous + if manifest_written: + _atomic_write_bytes( + manifest_path, + (json.dumps(manifest, indent=2, sort_keys=False) + "\n").encode(), + ) + return FreezeResult( + copied=tuple(copied), + already_frozen=tuple(already), + manifest_path=manifest_path, + frozen_root=frozen_root, + manifest_written=manifest_written, + ) + + +class FrozenExecBaseline: + """Verified read-only accessor for the frozen exec-basis baseline. + + Every read is checked against the git-tracked manifest, so this object + cannot serve bytes that are not the frozen bytes — including the case where + the caller aims it at a directory of freshly generated artifacts. + """ + + def __init__(self, frozen_root: Path, manifest_path: Path) -> None: + manifest_path = Path(manifest_path) + if not manifest_path.exists(): + raise FileNotFoundError( + f"no frozen-baseline manifest at {manifest_path}; the baseline " + "must be frozen (and its manifest committed) before any " + "reconciliation can read it" + ) + manifest = json.loads(manifest_path.read_text()) + if manifest.get("schema") != MANIFEST_SCHEMA: + raise BaselineIntegrityError( + f"manifest schema {manifest.get('schema')!r} != {MANIFEST_SCHEMA!r}" + ) + + root = Path(frozen_root).resolve() + # Refuse the live path outright. This is the cheap guard; the hash check + # below is the one that actually cannot be talked around. + if root.name == "reports" and root.parent.name == "artifacts": + raise BaselineIntegrityError( + f"refusing to read the LIVE artifacts directory as a frozen " + f"baseline: {root}. Reconciliation reads frozen copies only." + ) + if not root.is_dir(): + raise FileNotFoundError(f"frozen baseline root not found: {root}") + + self.root = root + self.manifest_path = manifest_path + self._expected: dict[str, str] = { + str(entry["name"]): str(entry["sha256"]) for entry in manifest["files"] + } + self.frozen_at_git_head = manifest.get("frozen_at_git_head", "unknown") + self.file_count = int(manifest.get("file_count", len(self._expected))) + + @property + def names(self) -> tuple[str, ...]: + return tuple(sorted(self._expected)) + + def read_bytes(self, name: str) -> bytes: + """Return the frozen bytes of ``name``, or raise.""" + if name not in self._expected: + raise KeyError(f"{name!r} is not part of the frozen exec baseline") + path = self.root / name + if not path.exists(): + raise FileNotFoundError(f"frozen artifact missing from disk: {path}") + payload = path.read_bytes() + digest = hashlib.sha256(payload).hexdigest() + if digest != self._expected[name]: + raise BaselineIntegrityError( + f"{name}: on-disk sha256 {digest[:16]} != manifest " + f"{self._expected[name][:16]}. The frozen baseline has been " + "modified; reconciliation against it would be meaningless." + ) + return payload + + def read_text(self, name: str) -> str: + return self.read_bytes(name).decode("utf-8") + + def read_json(self, name: str) -> dict: + return json.loads(self.read_text(name)) + + def report_json(self, factor: str, book: str) -> dict: + return self.read_json(f"eval_{factor}_exec_{book}.json") + + def verify_all(self) -> tuple[int, list[str]]: + """Verify every manifest entry. Returns (ok_count, problems).""" + problems: list[str] = [] + ok = 0 + for name in self.names: + try: + self.read_bytes(name) + ok += 1 + except (BaselineIntegrityError, FileNotFoundError) as exc: + problems.append(f"{name}: {exc}") + return ok, problems + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--repo-root", default=".") + parser.add_argument("--reports-dir", default=None) + parser.add_argument("--frozen-root", default=None) + parser.add_argument("--manifest", default=None) + parser.add_argument( + "--verify", + action="store_true", + help="verify the frozen tree against the manifest; copy nothing", + ) + parser.add_argument("--source-note", default="") + args = parser.parse_args(argv) + + repo_root = Path(args.repo_root).resolve() + reports_dir = Path(args.reports_dir or repo_root / DEFAULT_REPORTS_DIR) + frozen_root = Path(args.frozen_root or repo_root / DEFAULT_FROZEN_ROOT) + manifest_path = Path(args.manifest or repo_root / DEFAULT_MANIFEST) + + if args.verify: + baseline = FrozenExecBaseline(frozen_root, manifest_path) + ok, problems = baseline.verify_all() + print(f"verified {ok}/{baseline.file_count} frozen artifacts") + print(f"frozen at git head: {baseline.frozen_at_git_head}") + for problem in problems: + print(f" PROBLEM {problem}") + return 1 if problems else 0 + + result = freeze( + reports_dir, + frozen_root, + manifest_path, + repo_root=repo_root, + source_note=args.source_note, + ) + print(f"frozen root : {result.frozen_root}") + print(f"manifest : {result.manifest_path}") + print(f"copied : {len(result.copied)}") + print(f"already ok : {len(result.already_frozen)}") + print(f"manifest : {'REWRITTEN' if result.manifest_written else 'unchanged'}") + return 0 + + +if __name__ == "__main__": # pragma: no cover + raise SystemExit(main(sys.argv[1:])) diff --git a/qt/saturation_probe.py b/qt/saturation_probe.py new file mode 100644 index 0000000..9034102 --- /dev/null +++ b/qt/saturation_probe.py @@ -0,0 +1,228 @@ +"""D5 feasibility probe: can the D4 materializer run the real evaluation universe? + +D4's materializer was accepted against a 40-symbol hot-path smoke. D5 needs it on +the eleven-factor evaluation plane (CSI500, 995 symbols, 2021-07-01..2026-06-30), +so before committing to a multi-hour run this probe MEASURES, on the real cache, +the four quantities that decide whether that run is possible at all: + +1. **listing depth** — per-symbol first 1min month, read from the parquet store's + directory layout (no data is read). A symbol whose bars begin at or after + ``emit_start`` can never accumulate ``lookback_days`` valid days before it, so + the pooled saturation criterion can never terminate early and the load expands + to the declared floor; +2. **on-disk volume** at the evaluation window vs at the floor; +3. **in-memory cost** of a whole-universe single-frame load, measured on a sample + and extrapolated linearly (the materializer loads every symbol into ONE frame: + ``sources.minute.minute_bars(list(symbols), ...)``); +4. **per-symbol equivalence** — whether materializing symbol-by-symbol and + concatenating reproduces the whole-universe values exactly. This is the + decisive question for the fix, and it is answered per factor rather than + assumed: ten factors are per-symbol pure, ``intraday_amp_cut`` is not (it + z-scores across the loaded cross-section, ``AMP_CUT_MIN_CROSS_SECTION=10``). + +Run: ``python -m qt.saturation_probe`` (cache-only; zero live calls). +""" + +from __future__ import annotations + +import argparse +import resource +import time +from pathlib import Path + +import pandas as pd + +from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT +from data.clean.intraday_schema import RAW_INTRADAY_FREQ +from factors import registry as factor_registry +from factors.compute.minute.binding import minute_raw_from_bars +from qt.factor_hotpath_smoke import CACHE_MINUTE_DATA_START, CacheMinuteProvider + +DEFAULT_CACHE_ROOT = "artifacts/cache/tushare/v1" +#: The frozen D1 panel is on the exact evaluation data plane, so its symbol list +#: IS the evaluation universe — no need to re-resolve PIT membership here. +DEFAULT_UNIVERSE_PANEL = "artifacts/refactor_baseline/panels/ridge_minute_return_20.parquet" +EVAL_START = pd.Timestamp("2021-07-01") +EVAL_END = pd.Timestamp("2026-06-30") +GB = 1024**3 + + +def universe_from_frozen_panel(path: str) -> list[str]: + return sorted(pd.read_parquet(path)["symbol"].unique().tolist()) + + +def first_minute_month(root: str, symbols: set[str]) -> pd.Series: + """Per-symbol earliest 1min month, from DIRECTORY STRUCTURE only.""" + base = Path(root) / INTRADAY_ENDPOINT / f"freq={RAW_INTRADAY_FREQ}" + first: dict[str, pd.Timestamp] = {} + for prefix in base.glob("symbol_prefix=*"): + for sym_dir in prefix.glob("symbol=*"): + sym = sym_dir.name.split("=", 1)[1] + if sym not in symbols: + continue + months = [ + pd.Timestamp( + year=int(yd.name.split("=", 1)[1]), + month=int(mf.name.split("=", 1)[1].replace(".parquet", "")), + day=1, + ) + for yd in sym_dir.glob("year=*") + for mf in yd.glob("month=*") + ] + if months: + first[sym] = min(months) + return pd.Series(first).sort_values() + + +def partition_bytes(root: str, symbols: set[str]) -> tuple[int, int]: + """(evaluation-window bytes, floor-depth bytes) of 1min month partitions.""" + base = Path(root) / INTRADAY_ENDPOINT / f"freq={RAW_INTRADAY_FREQ}" + ev = full = 0 + lo = (EVAL_START.year, EVAL_START.month) + hi = (EVAL_END.year, EVAL_END.month) + for prefix in base.glob("symbol_prefix=*"): + for sym_dir in prefix.glob("symbol=*"): + if sym_dir.name.split("=", 1)[1] not in symbols: + continue + for yd in sym_dir.glob("year=*"): + year = int(yd.name.split("=", 1)[1]) + for mf in yd.glob("month=*"): + month = int(mf.name.split("=", 1)[1].replace(".parquet", "")) + size = mf.stat().st_size + full += size + if lo <= (year, month) <= hi: + ev += size + return ev, full + + +def measure_load(provider, symbols: list[str], start, end) -> dict[str, float]: + t0 = time.time() + bars = provider.minute_bars(symbols, start, end) + elapsed = time.time() - t0 + out = { + "rows": float(len(bars)), + "frame_gb": float(bars.memory_usage(deep=True).sum()) / GB, + "peak_rss_gb": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / (1024**2), + "seconds": elapsed, + } + del bars + return out + + +def per_symbol_equivalence( + provider, symbols: list[str], start, end, factor_names: tuple[str, ...] +) -> list[dict]: + """Whole-universe vs per-symbol-concatenated, per factor, on real bars.""" + bars = provider.minute_bars(symbols, start, end) + rows = [] + for name in factor_names: + factor = factor_registry.build(name, {}) + whole = minute_raw_from_bars(factor, bars).sort_index() + parts = [] + for sym in symbols: + one = bars[bars.index.get_level_values("symbol") == sym] + if one.empty: + continue + parts.append(minute_raw_from_bars(factor, one)) + per = pd.concat(parts).sort_index() if parts else pd.Series(dtype=float) + entry: dict = {"factor": name, "rows": len(whole)} + if whole.index.equals(per.index): + both = whole.notna() & per.notna() + entry["index_same"] = True + # None, not 0.0, when the two series share NO finite cell: printing a + # zero difference next to "NOT SAFE" reads as "the values agree", + # when in fact there is nothing to compare (that is exactly the + # intraday_amp_cut case, where every per-symbol value is NaN). + entry["max_abs_diff"] = ( + float((whole[both] - per[both]).abs().max()) if both.any() else None + ) + entry["compared_cells"] = int(both.sum()) + entry["nan_set_diff"] = int((whole.isna() != per.isna()).sum()) + else: + entry["index_same"] = False + entry["max_abs_diff"] = None + entry["compared_cells"] = 0 + entry["nan_set_diff"] = -1 + entry["per_symbol_safe"] = bool( + entry["index_same"] + and entry["nan_set_diff"] == 0 + and entry["max_abs_diff"] == 0.0 + ) + rows.append(entry) + return rows + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--cache-root", default=DEFAULT_CACHE_ROOT) + parser.add_argument("--universe-panel", default=DEFAULT_UNIVERSE_PANEL) + parser.add_argument("--sample-symbols", type=int, default=20) + args = parser.parse_args(argv) + + symbols = universe_from_frozen_panel(args.universe_panel) + symset = set(symbols) + floor = pd.Timestamp(CACHE_MINUTE_DATA_START) + print(f"universe: {len(symbols)} symbols | emit {EVAL_START.date()}..{EVAL_END.date()}") + print(f"declared minute floor: {floor.date()}\n") + + print("== 1. listing depth ==") + first = first_minute_month(args.cache_root, symset) + late = int((first >= EVAL_START).sum()) + print(first.dt.year.value_counts().sort_index().to_string()) + print( + f"\nsymbols whose first 1min bar is ON/AFTER emit_start: {late}" + f" -> pooled saturation CANNOT terminate early; it expands to the floor\n" + ) + + print("== 2. on-disk volume ==") + ev, full = partition_bytes(args.cache_root, symset) + print(f"evaluation window : {ev / GB:7.2f} GB") + print(f"floor depth : {full / GB:7.2f} GB ({full / max(ev, 1):.2f}x)\n") + + print("== 3. whole-universe single-frame load (measured, then extrapolated) ==") + provider = CacheMinuteProvider(args.cache_root) + sample = symbols[: args.sample_symbols] + scale = len(symbols) / len(sample) + for label, start in (("eval-window", EVAL_START), ("floor-depth", floor)): + m = measure_load(provider, sample, start, EVAL_END) + print( + f"{label:12s} {len(sample):3d} syms: rows={m['rows']:>12,.0f} " + f"frame={m['frame_gb']:6.2f} GB peakRSS={m['peak_rss_gb']:6.2f} GB " + f"load={m['seconds']:6.1f}s" + ) + print( + f"{'':12s} x{scale:5.2f} -> rows={m['rows'] * scale / 1e6:7.0f}M " + f"frame={m['frame_gb'] * scale:7.1f} GB load={m['seconds'] * scale / 60:5.1f} min" + ) + print(f"\nlive minute calls so far: {provider.live_calls} (cache-only)\n") + + print("== 4. per-symbol equivalence (the fix's decisive question) ==") + probe_syms = symbols[:12] + rows = per_symbol_equivalence( + provider, + probe_syms, + pd.Timestamp("2021-03-01"), + pd.Timestamp("2021-09-30"), + ("ridge_minute_return_20", "volume_peak_count_20", "intraday_amp_cut_10"), + ) + for r in rows: + verdict = "SAFE" if r["per_symbol_safe"] else "NOT SAFE" + diff = r["max_abs_diff"] + shown = f"{diff:.3e}" if diff is not None else "n/a (no comparable cell)" + print( + f"{r['factor']:26s} rows={r['rows']:>6,} index_same={r['index_same']!s:5s} " + f"max|diff|={shown:>22s} compared={r['compared_cells']:>5d} " + f"nan_set_diff={r['nan_set_diff']:>5d} {verdict}" + ) + print( + "\nintraday_amp_cut is expected NOT SAFE at whole-factor granularity: its " + "step-4 cross-sectional z-score needs >= AMP_CUT_MIN_CROSS_SECTION symbols " + "on a date, and a one-symbol cross-section is all-NaN by definition. The " + "per-symbol split must therefore happen BEFORE that combine, not around it." + ) + print(f"live minute calls total: {provider.live_calls}") + return 0 + + +if __name__ == "__main__": # pragma: no cover + raise SystemExit(main()) diff --git a/tests/test_exec_baseline_freeze.py b/tests/test_exec_baseline_freeze.py new file mode 100644 index 0000000..1c30509 --- /dev/null +++ b/tests/test_exec_baseline_freeze.py @@ -0,0 +1,339 @@ +"""D5 C1: the frozen exec-basis baseline must be unforgeable and unclobberable. + +The frozen 77-artifact tree is the ONLY thing the D5 reconciliation can be +judged against, and the unified runner writes to the same filenames it was +copied from. These tests lock the two properties that make the reconciliation +capable of failing at all: + +* the reader serves frozen bytes or raises — it can never silently serve + freshly generated artifacts (the ``compare_postmerge.py`` empty-reconciliation + failure mode recorded in CLAUDE.md); +* freezing never overwrites an existing baseline with different content. + +Every "X raises" test here is paired with a real, non-vacuous mutation. (The +first draft of the tamper test replaced a byte string that did not occur in the +file, so it mutated nothing and "passed" against a guard that had not been +exercised — the exact trap this suite exists to avoid.) +""" + +from __future__ import annotations + +import hashlib +import json +import re +import shutil +import subprocess +from pathlib import Path + +import pytest + +from qt.exec_baseline_freeze import ( + EXPECTED_FILE_COUNT, + MANIFEST_SCHEMA, + BaselineIntegrityError, + FrozenExecBaseline, + expected_artifact_names, + freeze, + sha256_file, +) + +REPO_ROOT = Path(__file__).resolve().parents[1] +MANIFEST = REPO_ROOT / "docs/factors/d5_exec_baseline_manifest.json" + + +def _fake_reports(root: Path) -> Path: + """A synthetic 'live reports' dir holding the full expected inventory.""" + reports = root / "reports" + reports.mkdir(parents=True) + for i, name in enumerate(expected_artifact_names()): + (reports / name).write_bytes(f"payload-{i}-{name}".encode()) + return reports + + +def _freeze_fake(tmp_path: Path) -> tuple[Path, Path, Path]: + reports = _fake_reports(tmp_path) + frozen = tmp_path / "frozen" + manifest = tmp_path / "manifest.json" + freeze(reports, frozen, manifest, repo_root=tmp_path) + return reports, frozen, manifest + + +# -------------------------------------------------------------------------- +# inventory +# -------------------------------------------------------------------------- + + +def test_exec_baseline_inventory_is_77_derived_from_the_factor_list() -> None: + names = expected_artifact_names() + assert len(names) == EXPECTED_FILE_COUNT == 77 + assert len(set(names)) == len(names) + assert sum(n.endswith(".json") for n in names) == 22 + assert sum(n.endswith("_dashboard.png") for n in names) == 22 + assert sum(n.endswith("_basis_sanity.md") for n in names) == 11 + # 22 per-book report .md + 11 basis-sanity .md = 33 .md in total + assert sum(n.endswith(".md") for n in names) == 33 + + +def test_exec_baseline_freeze_refuses_a_short_inventory(tmp_path: Path) -> None: + """A missing artifact is a loud absence, not a quietly smaller baseline.""" + reports = _fake_reports(tmp_path) + (reports / "eval_jump_amount_corr_exec_no_book.json").unlink() + with pytest.raises(FileNotFoundError, match="absent"): + freeze(reports, tmp_path / "frozen", tmp_path / "m.json", repo_root=tmp_path) + + +def test_exec_baseline_freeze_refuses_unexpected_members(tmp_path: Path) -> None: + reports = _fake_reports(tmp_path) + (reports / "eval_surprise_exec_no_book.json").write_bytes(b"x") + with pytest.raises(RuntimeError, match="unexpected"): + freeze(reports, tmp_path / "frozen", tmp_path / "m.json", repo_root=tmp_path) + + +# -------------------------------------------------------------------------- +# freeze: non-destructive, idempotent +# -------------------------------------------------------------------------- + + +def test_exec_baseline_freeze_copies_then_is_idempotent(tmp_path: Path) -> None: + reports, frozen, manifest = _freeze_fake(tmp_path) + assert len(list(frozen.iterdir())) == EXPECTED_FILE_COUNT + + again = freeze(reports, frozen, manifest, repo_root=tmp_path) + assert again.copied == () + assert len(again.already_frozen) == EXPECTED_FILE_COUNT + + +def test_exec_baseline_freeze_never_overwrites_a_differing_baseline( + tmp_path: Path, +) -> None: + """MUTATION: change the LIVE artifact after freezing. Re-freeze must refuse. + + This is the scenario that matters — the unified runner rewriting a live + artifact must never be able to launder itself into the frozen baseline. + """ + reports, frozen, manifest = _freeze_fake(tmp_path) + victim = reports / "eval_ridge_minute_return_exec_with_book.json" + before = (frozen / victim.name).read_bytes() + victim.write_bytes(victim.read_bytes() + b"!") # real, non-vacuous change + + with pytest.raises(BaselineIntegrityError, match="differs from the live"): + freeze(reports, frozen, manifest, repo_root=tmp_path) + assert (frozen / victim.name).read_bytes() == before # untouched + + +def test_exec_baseline_freeze_refuses_to_target_the_source_dir(tmp_path: Path) -> None: + reports = _fake_reports(tmp_path) + with pytest.raises(ValueError, match="live reports directory"): + freeze(reports, reports, tmp_path / "m.json", repo_root=tmp_path) + + +# -------------------------------------------------------------------------- +# reader: structural isolation +# -------------------------------------------------------------------------- + + +def test_exec_baseline_reader_rejects_tampered_bytes(tmp_path: Path) -> None: + """MUTATION: flip one byte of a frozen file; the reader must refuse it.""" + _, frozen, manifest = _freeze_fake(tmp_path) + name = "eval_jump_amount_corr_exec_no_book.json" + victim = frozen / name + before = victim.read_bytes() + after = before[:-1] + bytes([before[-1] ^ 0x01]) + assert after != before, "mutation must actually mutate" + assert hashlib.sha256(after).hexdigest() != hashlib.sha256(before).hexdigest() + victim.write_bytes(after) + + baseline = FrozenExecBaseline(frozen, manifest) + with pytest.raises(BaselineIntegrityError, match="on-disk sha256"): + baseline.read_bytes(name) + + ok, problems = baseline.verify_all() + assert (ok, len(problems)) == (EXPECTED_FILE_COUNT - 1, 1) + # the guard is targeted, not a blanket failure + baseline.read_bytes("eval_valley_price_quantile_exec_no_book.json") + + +def test_exec_baseline_reader_rejects_a_missing_file(tmp_path: Path) -> None: + _, frozen, manifest = _freeze_fake(tmp_path) + (frozen / "eval_volume_peak_count_exec_no_book.json").unlink() + baseline = FrozenExecBaseline(frozen, manifest) + with pytest.raises(FileNotFoundError): + baseline.read_bytes("eval_volume_peak_count_exec_no_book.json") + + +def test_exec_baseline_reader_refuses_the_live_reports_directory() -> None: + """Aiming the reader at artifacts/reports/ is refused by path, before hashing.""" + with pytest.raises(BaselineIntegrityError, match="LIVE artifacts directory"): + FrozenExecBaseline(REPO_ROOT / "artifacts" / "reports", MANIFEST) + + +def test_exec_baseline_reader_requires_a_manifest(tmp_path: Path) -> None: + _, frozen, _ = _freeze_fake(tmp_path) + with pytest.raises(FileNotFoundError, match="manifest"): + FrozenExecBaseline(frozen, tmp_path / "nope.json") + + +def test_exec_baseline_reader_rejects_a_foreign_manifest_schema(tmp_path: Path) -> None: + _, frozen, manifest = _freeze_fake(tmp_path) + payload = json.loads(manifest.read_text()) + payload["schema"] = "something-else/9" + manifest.write_text(json.dumps(payload)) + with pytest.raises(BaselineIntegrityError, match="schema"): + FrozenExecBaseline(frozen, manifest) + + +def test_exec_baseline_reader_rejects_unknown_names(tmp_path: Path) -> None: + _, frozen, manifest = _freeze_fake(tmp_path) + baseline = FrozenExecBaseline(frozen, manifest) + with pytest.raises(KeyError): + baseline.read_bytes("eval_not_a_factor_exec_no_book.json") + + +# -------------------------------------------------------------------------- +# the real, committed baseline +# -------------------------------------------------------------------------- + + +def test_committed_manifest_describes_the_real_frozen_baseline() -> None: + """The git-tracked manifest is well-formed, complete, and carries provenance. + + Deliberately NOT skipif-guarded: it reads only the committed manifest, so it + runs on every machine. That matters because the provenance assertions below + are the ones that catch a manifest whose history was silently restamped — + a check that skips on the hosts without the gitignored bytes would be a + check that never runs where it is needed. + """ + payload = json.loads(MANIFEST.read_text()) + assert payload["schema"] == MANIFEST_SCHEMA + assert payload["file_count"] == EXPECTED_FILE_COUNT + assert len(payload["files"]) == EXPECTED_FILE_COUNT + assert {f["name"] for f in payload["files"]} == set(expected_artifact_names()) + # repo-relative, machine-independent + assert not payload["source_dir"].startswith("/") + assert not payload["frozen_root"].startswith("/") + + # --- provenance must be present and well-formed, not merely a field --- + assert payload["source_note"].strip(), ( + "source_note is empty: the manifest no longer records WHERE the frozen " + "bytes came from. A re-freeze must inherit provenance, never blank it." + ) + assert re.fullmatch(r"[0-9a-f]{40}", payload["frozen_at_git_head"]), ( + f"frozen_at_git_head {payload['frozen_at_git_head']!r} is not a full " + "40-hex commit id" + ) + assert payload["frozen_at_utc"].startswith("20") + + for entry in payload["files"]: + assert re.fullmatch(r"[0-9a-f]{64}", entry["sha256"]), ( + f"{entry['name']}: sha256 is not 64 hex chars" + ) + assert int(entry["size_bytes"]) > 0, f"{entry['name']}: non-positive size" + + +def test_freeze_inherits_provenance_instead_of_restamping_it(tmp_path: Path) -> None: + """MUTATION: re-freeze an UNCHANGED tree. Provenance must survive verbatim. + + The regression this locks: freeze() used to rebuild frozen_at_utc / + frozen_at_git_head / source_note on every call, so the documented + verification command silently repointed the manifest at whoever re-ran it + and blanked the provenance narrative, while printing `copied: 0`. + """ + reports, frozen, manifest = _freeze_fake(tmp_path) + # give it real provenance, as the committed manifest has + original = json.loads(manifest.read_text()) + original["source_note"] = "PR #79 artifacts; bulk-copied 2026-07-21 15:33:20" + original["frozen_at_git_head"] = "a" * 40 + original["frozen_at_utc"] = "2026-07-25T00:00:00+00:00" + manifest.write_text(json.dumps(original, indent=2) + "\n") + before = manifest.read_bytes() + + result = freeze(reports, frozen, manifest, repo_root=tmp_path) + + assert result.copied == () + assert result.manifest_written is False, "an unchanged tree must not rewrite" + assert manifest.read_bytes() == before, "manifest bytes must be untouched" + after = json.loads(manifest.read_text()) + assert after["source_note"] == original["source_note"] + assert after["frozen_at_git_head"] == "a" * 40 + assert after["frozen_at_utc"] == "2026-07-25T00:00:00+00:00" + + +def test_freeze_lets_an_explicit_source_note_win(tmp_path: Path) -> None: + """Inheritance is a default, not a lock: an explicit note is still an act.""" + reports, frozen, manifest = _freeze_fake(tmp_path) + result = freeze( + reports, frozen, manifest, repo_root=tmp_path, source_note="corrected note" + ) + assert result.manifest_written is True + assert json.loads(manifest.read_text())["source_note"] == "corrected note" + + +@pytest.mark.skipif( + not (REPO_ROOT / "artifacts/refactor_baseline/exec_baseline").is_dir(), + reason="frozen baseline bytes are gitignored; present only on the run host", +) +def test_real_frozen_baseline_verifies_against_the_committed_manifest() -> None: + baseline = FrozenExecBaseline( + REPO_ROOT / "artifacts/refactor_baseline/exec_baseline", MANIFEST + ) + ok, problems = baseline.verify_all() + assert problems == [] + assert ok == EXPECTED_FILE_COUNT + + report = baseline.report_json("jump_amount_corr", "no_book") + assert set(report) >= {"sections", "spec", "thresholds", "verdict"} + + +def test_sha256_file_matches_hashlib(tmp_path: Path) -> None: + path = tmp_path / "blob.bin" + payload = b"x" * (3 << 20) + b"tail" # spans the 1MiB read chunks + path.write_bytes(payload) + assert sha256_file(path) == hashlib.sha256(payload).hexdigest() + + +def _tracked_under(prefix: str) -> list[str]: + """Paths git TRACKS under ``prefix``, read from the index. + + Queries the index rather than the filesystem on purpose. ``artifacts/`` is + routinely a symlink into another checkout, and every filesystem-walking git + command (``add``, ``check-ignore``, ``ls-files --error-unmatch`` on a path) + refuses such paths with "beyond a symbolic link" — which would make an + assertion about them pass for the wrong reason, and make it impossible to + mutate. The index has no such blind spot. + """ + proc = subprocess.run( + ["git", "-C", str(REPO_ROOT), "ls-files", "--", prefix], + capture_output=True, + text=True, + check=True, + ) + return proc.stdout.split() + + +def test_frozen_bytes_are_untracked_while_the_manifest_is_tracked() -> None: + """The split that gives the tamper check its teeth, asserted against git. + + Hashes live in version control so they can only move through a reviewable + diff; the bytes they describe do not. If both lived in git, one commit could + move the bytes and their hashes together and the tamper check would be + decorative. + + The previous version of this test claimed exactly this but never consulted + git at all — it froze a fake tree under tmp_path, deleted it, and asserted a + file it had just written was still readable. Removing ``artifacts/`` from + .gitignore left it green. + """ + assert _tracked_under("artifacts") == [], ( + "no bytes under artifacts/ may be committed; found tracked paths" + ) + manifest_rel = str(MANIFEST.relative_to(REPO_ROOT)) + assert manifest_rel in _tracked_under("docs/factors"), ( + "the manifest MUST be committed — it is the tamper reference" + ) + + +def test_manifest_outlives_the_bytes_it_describes(tmp_path: Path) -> None: + """Losing the frozen tree must not lose the record of what it contained.""" + _, frozen, manifest = _freeze_fake(tmp_path) + shutil.rmtree(frozen) + assert json.loads(manifest.read_text())["file_count"] == EXPECTED_FILE_COUNT