From 6e35d3835f2a1083a3e8c18f15e146dbd805d1d5 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Sat, 25 Jul 2026 06:48:16 -0700 Subject: [PATCH 1/5] feat(qt): freeze the exec-basis evaluation baseline before D5 touches anything The 22 exec-basis eval artifacts (+22 rendered reports, +11 basis-sanity disclosures, +22 dashboards = 77 files) are the only thing the D5 unified runner can be judged against, and they are gitignored, single-copy, and not reproducible from current code: regenerating them needs a pre-D5 tree plus a ~1.5h run. The unified runner writes to those same filenames. Running it once would destroy the baseline, and any reconciliation performed afterwards would compare the new output against a copy of itself -- the compare_postmerge.py empty-reconciliation failure mode already committed once in this repo. So the freeze lands before any D5 engine code. Bytes go to gitignored artifacts/refactor_baseline/exec_baseline/; the 77 sha256 hashes go into git. That split is what gives the tamper check teeth: FrozenExecBaseline verifies every file it serves against the committed manifest at read time, so a live artifact copied over a frozen one is a loud failure rather than a silent substitution -- and whoever overwrites the frozen tree cannot also move the hashes without a reviewable diff. Reconciliation code must go through this reader; it may not open artifacts/reports/ itself. Provenance is stated by evidence strength, not asserted. Observed: all 77 exec files share one mtime instant (2ms spread) -- a bulk copy, not a 1.5h run -- 8 minutes after PR #79 merged. Contrast: the 44 close-basis artifacts carry 11 progressive mtimes matching the recorded #74 rerun window exactly. Attributed (not observed): the content comes from the #79 exec-basis run. The manifest field is named frozen_at_git_head, not producing_sha, so the freeze instant is never misread as the producing commit. Mutation evidence, run rather than claimed: reader aimed at artifacts/reports/ raises; one flipped byte in a frozen file raises and verify_all reports 76 ok / 1 problem while untouched siblings still read; re-freezing after the live source changed refuses and leaves the frozen copy byte-identical; a short or over-full inventory raises. The first draft of the tamper test replaced a byte string absent from the file, so it mutated nothing and "passed" against a guard never exercised -- the test now asserts its own mutation is real. --- docs/factors/d5_exec_baseline_freeze.md | 121 +++++ docs/factors/d5_exec_baseline_manifest.json | 490 ++++++++++++++++++++ qt/exec_baseline_freeze.py | 394 ++++++++++++++++ tests/test_exec_baseline_freeze.py | 240 ++++++++++ 4 files changed, 1245 insertions(+) create mode 100644 docs/factors/d5_exec_baseline_freeze.md create mode 100644 docs/factors/d5_exec_baseline_manifest.json create mode 100644 qt/exec_baseline_freeze.py create mode 100644 tests/test_exec_baseline_freeze.py diff --git a/docs/factors/d5_exec_baseline_freeze.md b/docs/factors/d5_exec_baseline_freeze.md new file mode 100644 index 0000000..f6ff765 --- /dev/null +++ b/docs/factors/d5_exec_baseline_freeze.md @@ -0,0 +1,121 @@ +# 因子层重构 D5 —— exec 基准评估 artifact 冻结 + +> **状态**：D5 第一个交付物（C1），**先于任何 D5 代码改动执行**。 +> 设计出处：v3.2 §五第 3/4 腿（语义对账的比较对象）、§九 D5 行。 +> 本文是 provenance 叙述；**机器可读权威 manifest 是入 git 的 +> `docs/factors/d5_exec_baseline_manifest.json`**（77 条 sha256）。 +> 冻结的字节在 `artifacts/refactor_baseline/exec_baseline/`（gitignored，不入 git）。 + +--- + +## 一、为什么必须先冻结（读这段再动任何东西） + +D5 要在删掉 11 个旧 runner 之前证明新引擎没有悄悄改变任何东西。唯一能证明的办法是拿新 +runner 的输出和**旧 runner 的输出**比。而旧输出： + +1. **只有一份**，在 `artifacts/reports/`，**gitignored、只存在于本机**； +2. **不可从当前代码再生**——需要 pre-D5 的树 + 约 1.5 小时真实 run； +3. 设计 §五 提到的耐久副本 `scratchpad/baseline_reports/` **已不存在**（旧会话 scratchpad，已清）； +4. **新 runner 沿用同名输出路径**。 + +即：新 runner 跑一次就会覆盖唯一的基线，随后的"对账"会比对刚被自己覆盖的文件——正是 +CLAUDE.md 记录在案的 `compare_postmerge.py` **空对账**失效模式（拿新结果和自己比， +构造性恒真，`ic=1.0` vs `ic=999.0` 也会打印"精确复现"）。 + +**所以顺序不可交换：先冻结，再改代码。** 本 commit 不含任何 D5 引擎代码。 + +## 二、冻结了什么（77 个文件，按因子列表派生而非按目录扫描） + +| 类别 | 数量 | 说明 | +|---|---|---| +| `eval_{factor}_exec_{book}.json` | 22 | **对账的权威机器可读源**（IC / ICIR / 分位价差 / verdict 全在此） | +| `eval_{factor}_exec_{book}.md` | 22 | 渲染报告 | +| `eval_{factor}_exec_basis_sanity.md` | 11 | #79 的覆盖率/偏差披露（含 `coverage_bias_bad_vwap` 原话） | +| `eval_{factor}_exec_{book}_dashboard.png` | 22 | 单图 dashboard（16MB，同样不可再生） | + +11 因子 × 2 book。清单**由因子列表派生**（`expected_artifact_names()`），不是目录扫描—— +少一个文件是**可读报错**，不是一个静静变小的基线。多一个未知成员同样报错。 + +> 注：`_exec_basis_sanity.md` 与 dashboard 不在 lead 交办的「22 json + 22 md」里，是本步 +> 清点目录时发现的同族产物。它们同样单份、同样不可再生，故一并冻结——**冻结的成本是一次 +> 文件拷贝，漏冻的成本是永久丢失**。 + +**未冻结**：close 基准的 44 个 artifact。它们不是 D5 的对账参照（统一 runner 是 exec-only， +决策 4），且未被新 runner 的输出路径覆盖，原地不动。 + +## 三、Provenance（观察到的事实 vs 归因） + +**观察到的**（可复验）： + +- 77 个 exec 文件的 mtime **全部落在同一瞬间 `2026-07-21 15:33:20`**（跨度 ~2ms）。 +- 对照：44 个 **close 基准** artifact 的 mtime 呈 **11 个递进值** + `09:24:33 → 09:32:45 → … → 10:43:54`——每因子一个，正是一次真实 11 因子 run 的写入签名， + 且与 CLAUDE.md 记录的 #74 合并后全量重跑窗口（09:24:33→10:43:54）**逐值吻合**。 + +**推论**：exec 家族**不是**在 `artifacts/reports/` 里跑出来的，而是**被整体拷贝进来的** +（一次 1.5h 的 run 不可能让 77 个文件共享毫秒级 mtime）。拷贝时刻在 PR #79 合并 +（`b04c966`，15:25）之后 8 分钟——与「#79 在独立 worktree 里跑完、合并后把产物拷进主 checkout」 +的形态一致。 + +**归因（非观察）**：内容归属于 PR #79 的 exec 基准 run。这是**从 PR 记录推断的**，不是从 +文件系统观察到的。本文如此区分，是因为 #79 的 code review 未返回、靠自查合并（CLAUDE.md +已记录），其 provenance 本就该按证据强度分级陈述，而不是写成"由 SHA X 产生"。 + +**冻结时刻**：`frozen_at_git_head = 45c14aa`（D0–D4 已合并的 `main`）。这是**冻结动作**发生 +的 SHA，**不是产生 artifact 的 SHA**——manifest 字段名即为 `frozen_at_git_head`，不叫 +`producing_sha`，避免 D1 manifest 那种「producing SHA」措辞被误读。 + +## 四、结构性隔离：对账工具为什么读不到 live 路径 + +lead 的要求是「用路径断言或结构性隔离保证，不靠纪律」。实现为三层，**第 2 层承重**： + +| # | 机制 | 拦得住什么 | +|---|---|---| +| 1 | 读取器要求一份 git-tracked manifest 描述该目录 | 随手指向 `artifacts/reports/`（那里没有 manifest） | +| 2 | **每次读取都对该文件 sha256 校验 manifest** | **把 live 产物拷进冻结目录**——它问的不是"路径看着对不对"，而是"字节是不是冻结的那些字节" | +| 3 | 目录若解析为 `artifacts/reports` 直接拒绝 | 显式误用，早失败早报错 | + +**为什么 (2) 有牙**：字节在 gitignored 的 `artifacts/` 下，哈希在 git 里。覆盖冻结树的人 +无法同时悄悄移动哈希——哈希只能经一次会被 review 的 `git diff` 变动。 + +**mutation 证据（实跑，非声称）**： + +| mutation | 结果 | +|---|---| +| 读取器指向 `artifacts/reports/` | **RAISED** `refusing to read the LIVE artifacts directory` | +| 冻结文件末字节 XOR 0x01 | **RAISED** `on-disk sha256 … != manifest …`；`verify_all` **76 ok / 1 problem**；同目录未改的兄弟文件仍正常读出（守卫是定向的，不是一刀切） | +| 冻结后改 **live** 源文件再 freeze | **RAISED** `differs from the live artifact`，且冻结副本**逐字节未动** | +| 少一个 / 多一个清单成员 | **RAISED**（分别 `absent` / `unexpected`） | + +⚠️ **记一次本步自己的失误（照实记录，与项目既有先例同格式）**：tamper 测试的第一版把 +`b'"ic"'` 替换成 `b'"IC"'`，而该字节串**在文件里根本不存在** → `.replace()` 是 no-op → +"mutation" 什么也没改，测试对着一个从未被触发的守卫打印通过。它之所以被抓住，只因为同一次 +输出里 `verify_all` 报了 `ok=77 problems=0`，与"我刚破坏了一个文件"的预期矛盾。 +**这正是 §六.10「不变性测试没有 mutation 证据」要防的形态，而本步亲手造了一个。** +现版本对 mutation 本身加了断言（`assert after != before` 且 sha256 必须不同）， +使"mutation 是空的"变成测试失败而不是测试通过。 + +## 五、复验命令 + +``` +# 校验冻结树与 manifest 一致（不拷贝任何东西） +python -m qt.exec_baseline_freeze --verify + +# 再次冻结：幂等；内容不同则拒绝并保留原副本 +python -m qt.exec_baseline_freeze +``` + +冻结/校验工具：`qt/exec_baseline_freeze.py`；测试：`tests/test_exec_baseline_freeze.py`。 +读取器 `FrozenExecBaseline` 是 D5 对账**唯一**支持的入口——对账代码不得自己 `open()` +`artifacts/reports/` 下的任何文件。 + +## 六、清单摘要 + +完整 77 条 `(name, sha256, size, source_mtime_utc)` 见 +`docs/factors/d5_exec_baseline_manifest.json`。抽样： + +| 文件 | sha256（前 12） | bytes | +|---|---|---| +| `eval_amp_marginal_anomaly_vol_exec_no_book.json` | `6755304f05c7` | 19487 | +| `eval_amp_marginal_anomaly_vol_exec_no_book.md` | `2300300c7d26` | 16286 | +| `eval_amp_marginal_anomaly_vol_exec_basis_sanity.md` | `4856188e2674` | 3892 | diff --git a/docs/factors/d5_exec_baseline_manifest.json b/docs/factors/d5_exec_baseline_manifest.json new file mode 100644 index 0000000..09e8e0c --- /dev/null +++ b/docs/factors/d5_exec_baseline_manifest.json @@ -0,0 +1,490 @@ +{ + "schema": "d5-exec-baseline-freeze/1", + "frozen_at_utc": "2026-07-25T13:44:24.553842+00:00", + "frozen_at_git_head": "45c14aa036c37ddafc490bc16f961ec5212b5025", + "source_dir": "artifacts/reports", + "frozen_root": "artifacts/refactor_baseline/exec_baseline", + "file_count": 77, + "factors": [ + "jump_amount_corr", + "minute_ideal_amplitude", + "amp_marginal_anomaly_vol", + "volume_peak_count", + "intraday_amp_cut", + "peak_interval_kurtosis", + "valley_relative_vwap", + "valley_ridge_vwap_ratio", + "ridge_minute_return", + "valley_price_quantile", + "peak_ridge_amount_ratio" + ], + "books": [ + "no_book", + "with_book" + ], + "source_note": "PR #79 exec-to-exec rebasis artifacts. OBSERVED: all 77 files share mtime 2026-07-21 15:33:20 UTC-7 (bulk copy; ~2ms spread), 8 minutes after PR #79 merged as b04c966 at 15:25. Content is ATTRIBUTED to the #79 exec-basis run per the PR record; the copy instant is observed, the run is not.", + "files": [ + { + "name": "eval_amp_marginal_anomaly_vol_exec_basis_sanity.md", + "sha256": "4856188e26740f9cdf3629aae6e12d1932670c4be3ccba9779a54c74bdd63a2f", + "size_bytes": 3892, + "source_mtime_utc": "2026-07-21T22:33:20.985336+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_no_book.json", + "sha256": "6755304f05c705f5ef0551bf3e3f20a4deec9b9d45899798c1ea4a00f1ac80c3", + "size_bytes": 19487, + "source_mtime_utc": "2026-07-21T22:33:20.977414+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_no_book.md", + "sha256": "2300300c7d26f8b4d8f00d58e7a459e3b4e3664c36f0dbea70e4fa2c35918923", + "size_bytes": 16286, + "source_mtime_utc": "2026-07-21T22:33:20.978123+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_no_book_dashboard.png", + "sha256": "d39ba8fb4590c94dc747b82357dd694dd48303076b13a45d20577c0409139669", + "size_bytes": 655165, + "source_mtime_utc": "2026-07-21T22:33:20.979059+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_with_book.json", + "sha256": "e80b48c6aeec464791a210e2a89fdf14e064d9ec8fae9daa0711576ece4dd2a5", + "size_bytes": 21286, + "source_mtime_utc": "2026-07-21T22:33:20.977462+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_with_book.md", + "sha256": "f1df356c1db62e5730f261d678fd90e67252e8d36f96a451f28b7b0846da9980", + "size_bytes": 17717, + "source_mtime_utc": "2026-07-21T22:33:20.978161+00:00" + }, + { + "name": "eval_amp_marginal_anomaly_vol_exec_with_book_dashboard.png", + "sha256": "a585fcc7492c5d0138d5e13bac1764272d2e0fe77dfbe6b29f411db79fad2b2e", + "size_bytes": 665637, + "source_mtime_utc": "2026-07-21T22:33:20.979250+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_basis_sanity.md", + "sha256": "6fd77ccf8f094d862a6c49bece2c5247bdcb120ad9ee9a59c6c722e13e7e8158", + "size_bytes": 3900, + "source_mtime_utc": "2026-07-21T22:33:20.985399+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_no_book.json", + "sha256": "80cf44b25c4be0c9091e3cc43cc56fdcd6d61b5cb3b378e36b842fb2265ffad8", + "size_bytes": 21131, + "source_mtime_utc": "2026-07-21T22:33:20.977496+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_no_book.md", + "sha256": "eb80ca1dc57267f2470c393047b0acf7eea4dcbdb6e8b682545571d955035a1e", + "size_bytes": 17771, + "source_mtime_utc": "2026-07-21T22:33:20.978218+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_no_book_dashboard.png", + "sha256": "a29121a103c87b2d310ca32ffe18aa603c3b0316b2a5bc09da9488e1c8329ad0", + "size_bytes": 668176, + "source_mtime_utc": "2026-07-21T22:33:20.979444+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_with_book.json", + "sha256": "dc63b7e620d80b37482cc5b280c482de2f70e7c1b31da0b64f1047e431f7fe56", + "size_bytes": 22715, + "source_mtime_utc": "2026-07-21T22:33:20.977527+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_with_book.md", + "sha256": "61ad89dac238e11b0e5e74c24b20c15cb9b033e1d590be0080eec2865dff9936", + "size_bytes": 18994, + "source_mtime_utc": "2026-07-21T22:33:20.978252+00:00" + }, + { + "name": "eval_intraday_amp_cut_exec_with_book_dashboard.png", + "sha256": "987f7c14b86401823e747fd0c32ee6bb02ecd36973db4d23c0920e6b4a042bc5", + "size_bytes": 678388, + "source_mtime_utc": "2026-07-21T22:33:20.979649+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_basis_sanity.md", + "sha256": "a67c1e297293ae3a08904005bbd7ab18155394e9d977b7b3a290251116189db4", + "size_bytes": 3885, + "source_mtime_utc": "2026-07-21T22:33:20.985427+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_no_book.json", + "sha256": "126bdc8febb82dbf8e43c2e162e88044f2135924013518b4f03d1a89c3147b0f", + "size_bytes": 20546, + "source_mtime_utc": "2026-07-21T22:33:20.977555+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_no_book.md", + "sha256": "7fce0749339a4753c7fd508c0f86ecd4330b0cb523a93088f98425b18396bb35", + "size_bytes": 16625, + "source_mtime_utc": "2026-07-21T22:33:20.978304+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_no_book_dashboard.png", + "sha256": "8a83f2ecd73c40a85027914595bf1ac6deb23fcb9a5d9019bd16d808c24428c4", + "size_bytes": 620927, + "source_mtime_utc": "2026-07-21T22:33:20.979841+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_with_book.json", + "sha256": "08d791ad46e6793191662eb185083aa1e2a92f8fce82ad11889b6b39b464cfc6", + "size_bytes": 22093, + "source_mtime_utc": "2026-07-21T22:33:20.977582+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_with_book.md", + "sha256": "c9aea4a639c433aa2f6d4691c59025efb20f5c6a4cd79ee35eea104539feb529", + "size_bytes": 17811, + "source_mtime_utc": "2026-07-21T22:33:20.978333+00:00" + }, + { + "name": "eval_jump_amount_corr_exec_with_book_dashboard.png", + "sha256": "7a7714627f10305b1d165f68285f38f4c3c4afb1f31495bdd93b93b1cdb68cdc", + "size_bytes": 630809, + "source_mtime_utc": "2026-07-21T22:33:20.980035+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_basis_sanity.md", + "sha256": "12fa8a81a07451f960ef56ba9770ed03da00bb1a196a02f1c9a0f600575dd239", + "size_bytes": 3894, + "source_mtime_utc": "2026-07-21T22:33:20.985453+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_no_book.json", + "sha256": "495ff3189631c4ee435c3be69db9ad7bf670fa03fdf90d2015e774515f109c3e", + "size_bytes": 20409, + "source_mtime_utc": "2026-07-21T22:33:20.977612+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_no_book.md", + "sha256": "482cc7a219628fac2eb11b5b65708ace0880c4b8d4fabbb6cd3e4e61823d3e6d", + "size_bytes": 16611, + "source_mtime_utc": "2026-07-21T22:33:20.978386+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_no_book_dashboard.png", + "sha256": "878767ccae74ed0e55315d1fb064dea66e155e07f002134a25fa7acacf16933b", + "size_bytes": 572962, + "source_mtime_utc": "2026-07-21T22:33:20.980259+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_with_book.json", + "sha256": "85293e8d63edb08e4ebadcfb3fddc683d08ebb7fd7e3a28e4954d3bf27d7eae8", + "size_bytes": 21955, + "source_mtime_utc": "2026-07-21T22:33:20.977637+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_with_book.md", + "sha256": "84176a922ca7d8a83aa58aee1eff81972223ff3d3e60bcf17ea1659fe173f480", + "size_bytes": 17796, + "source_mtime_utc": "2026-07-21T22:33:20.978414+00:00" + }, + { + "name": "eval_minute_ideal_amplitude_exec_with_book_dashboard.png", + "sha256": "7241548bb091d481494fb917c9e781763acee4b8ec94c10322f17546e895b59d", + "size_bytes": 582827, + "source_mtime_utc": "2026-07-21T22:33:20.980434+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985477+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_no_book.json", + "sha256": "db9527bf4ad190e633fb0dbbf0057cdbfaee28160df8c674fdfa09e5b6f8103d", + "size_bytes": 20351, + "source_mtime_utc": "2026-07-21T22:33:20.977666+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_no_book.md", + "sha256": "622b6f17e0e83226204242096e43199b6f6592ca2dac4ee9a2ffed7957183203", + "size_bytes": 17288, + "source_mtime_utc": "2026-07-21T22:33:20.978471+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_no_book_dashboard.png", + "sha256": "291039b28536bc8084c0b64bc643910bdbb36541c774ba8e99560dd76d276542", + "size_bytes": 758107, + "source_mtime_utc": "2026-07-21T22:33:20.980605+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_with_book.json", + "sha256": "af92d2974d1dc69c13f6226cd97eae5454c15f9a240b364d3fc380995222865a", + "size_bytes": 22110, + "source_mtime_utc": "2026-07-21T22:33:20.977691+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_with_book.md", + "sha256": "d7346960f74ff95e925c191c1fdf1f506dbae5c574569c5c702fd0872b541e83", + "size_bytes": 18679, + "source_mtime_utc": "2026-07-21T22:33:20.978499+00:00" + }, + { + "name": "eval_peak_interval_kurtosis_exec_with_book_dashboard.png", + "sha256": "18b6ace5286822803b38e7630ddaff37680d052fc7f448ef2e7aa7f16c820024", + "size_bytes": 768042, + "source_mtime_utc": "2026-07-21T22:33:20.980828+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985501+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_no_book.json", + "sha256": "bc963b3007d363483a25fc10f38b4902cbb796966ee6a5d325c89b6e8a4538df", + "size_bytes": 20489, + "source_mtime_utc": "2026-07-21T22:33:20.977716+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_no_book.md", + "sha256": "2ba2f0e2dfb7b48ebbacd87d966ec75b57b79e882ef6b93e794c7fb25fb3b84f", + "size_bytes": 19109, + "source_mtime_utc": "2026-07-21T22:33:20.978554+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_no_book_dashboard.png", + "sha256": "62d8d74166e3d48d2807027aea8dc847d6a6bd36117123bc361dac1243ac7558", + "size_bytes": 767069, + "source_mtime_utc": "2026-07-21T22:33:20.981082+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_with_book.json", + "sha256": "bdb4795630a71a5097b16a4f9657da8baae61e8e5a3ddf550efb9f3b1e731e8e", + "size_bytes": 22023, + "source_mtime_utc": "2026-07-21T22:33:20.977742+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_with_book.md", + "sha256": "f0739f5ab6f6c0bb0648a3dcf218a5e0e3142d5aeb5f53a9d4920ece06a4aaf4", + "size_bytes": 20282, + "source_mtime_utc": "2026-07-21T22:33:20.978582+00:00" + }, + { + "name": "eval_peak_ridge_amount_ratio_exec_with_book_dashboard.png", + "sha256": "7da1593f9adfaf1499ac39f6948b18916450097e1f956747e473f639f55b8d90", + "size_bytes": 777536, + "source_mtime_utc": "2026-07-21T22:33:20.981301+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985526+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_no_book.json", + "sha256": "f4e1778cc22e3758d4b5ce0ce792dc61edfb704fc5e4141e830a0376c6c46f9f", + "size_bytes": 21107, + "source_mtime_utc": "2026-07-21T22:33:20.977768+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_no_book.md", + "sha256": "ed2fe47c2851f0880360985b66de0be89fe23d61ef4fb7df9352899cd7443b20", + "size_bytes": 19551, + "source_mtime_utc": "2026-07-21T22:33:20.978633+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_no_book_dashboard.png", + "sha256": "0be2a1c2a6210c7b47074af7fbd904853d6daeb65b3f7a88301205aa3c0b8d5d", + "size_bytes": 845707, + "source_mtime_utc": "2026-07-21T22:33:20.981516+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_with_book.json", + "sha256": "8fd9e6595c6a21f050b671cac0b8ca96b0bb0bd72d71912d7b5688756c41aade", + "size_bytes": 22906, + "source_mtime_utc": "2026-07-21T22:33:20.977796+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_with_book.md", + "sha256": "c7fe175a5d162e3a533c164dcc5015b0ebe39ebf260b2ebd73f866126593571c", + "size_bytes": 20989, + "source_mtime_utc": "2026-07-21T22:33:20.978660+00:00" + }, + { + "name": "eval_ridge_minute_return_exec_with_book_dashboard.png", + "sha256": "fb527aaf7134910131236dc2d5336efd65b1abcb0496f04172302349dd99e016", + "size_bytes": 854968, + "source_mtime_utc": "2026-07-21T22:33:20.981749+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_basis_sanity.md", + "sha256": "9fc709f253b8771a878b08d0035ba638fcc793d8b5d1b804215a710e90aaf05c", + "size_bytes": 3887, + "source_mtime_utc": "2026-07-21T22:33:20.985550+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_no_book.json", + "sha256": "8f29016285efb8ebf7119256955ab1e8cb54ab37a94d12b508a2ec8d749376a4", + "size_bytes": 20577, + "source_mtime_utc": "2026-07-21T22:33:20.977822+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_no_book.md", + "sha256": "a4ab52edd64c78ab3827dbda71d29a4f68eae475730df12162f9695ef1e0bf5e", + "size_bytes": 19922, + "source_mtime_utc": "2026-07-21T22:33:20.978712+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_no_book_dashboard.png", + "sha256": "6d5ac41bb26109fcd0d265bb6655eaf33472d0e196ad2ad6d869c8cabec6ed83", + "size_bytes": 777643, + "source_mtime_utc": "2026-07-21T22:33:20.982016+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_with_book.json", + "sha256": "b17311ac1ce7566c441f3d62ab373ddf07b95ad9ca9f08ae57f344197b4ee3ec", + "size_bytes": 22113, + "source_mtime_utc": "2026-07-21T22:33:20.977850+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_with_book.md", + "sha256": "c9ea08b66cd0c8ed7fba7e490041f59c8f3c5b437dc44c4a9f36d6ae49240505", + "size_bytes": 21097, + "source_mtime_utc": "2026-07-21T22:33:20.978740+00:00" + }, + { + "name": "eval_valley_price_quantile_exec_with_book_dashboard.png", + "sha256": "aa3c34db623f5efc8267aabc78be5903961a49450fe0b909d5f31501cb0be746", + "size_bytes": 787357, + "source_mtime_utc": "2026-07-21T22:33:20.982236+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.985574+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_no_book.json", + "sha256": "237af1b8ebce42b795c7d37e0c2491cee53420c2dd86637882a93a41cc85d4ab", + "size_bytes": 20492, + "source_mtime_utc": "2026-07-21T22:33:20.977883+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_no_book.md", + "sha256": "fd4be587f4344cb2b0209a9baa427333968cce280f9ba6763a864f90a5e8a37c", + "size_bytes": 17746, + "source_mtime_utc": "2026-07-21T22:33:20.978795+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_no_book_dashboard.png", + "sha256": "d92867037ebc470e3f1133a91cb93ba965e2c29157482593ed5bb08bec163f9a", + "size_bytes": 734803, + "source_mtime_utc": "2026-07-21T22:33:20.982455+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_with_book.json", + "sha256": "32e74de07d6fbdb953b38c8095589806cbda57b98620df32304a5c57d351517c", + "size_bytes": 22027, + "source_mtime_utc": "2026-07-21T22:33:20.977919+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_with_book.md", + "sha256": "6d63e704cc770d9bc9d462da60271abe02fcaa267a0e0bc85e321ec245dcf821", + "size_bytes": 18920, + "source_mtime_utc": "2026-07-21T22:33:20.978822+00:00" + }, + { + "name": "eval_valley_relative_vwap_exec_with_book_dashboard.png", + "sha256": "43d63d2db041d653839ac2c82d67962eedcbe80dede7f8e14cdb5d52dda4e25d", + "size_bytes": 745381, + "source_mtime_utc": "2026-07-21T22:33:20.982663+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.986839+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_no_book.json", + "sha256": "d9e6694f0d0a00eb36fad38131e919c0bc77f17ecd6a72798d93aa269e879eda", + "size_bytes": 21109, + "source_mtime_utc": "2026-07-21T22:33:20.977958+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_no_book.md", + "sha256": "6aa5456f2a40773920b8eb54bb5915452380cf012ddc03d9acda9908d07a58fe", + "size_bytes": 19026, + "source_mtime_utc": "2026-07-21T22:33:20.978875+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_no_book_dashboard.png", + "sha256": "aa106bd3a08a164aaa9b781d97e4f5592e8c9e015cfb104d771df4703d549a20", + "size_bytes": 805954, + "source_mtime_utc": "2026-07-21T22:33:20.982870+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_with_book.json", + "sha256": "f5b752d68097dc2d13577349188c3f2b01d89a23344f9cbd10582d6c62d07f1e", + "size_bytes": 22682, + "source_mtime_utc": "2026-07-21T22:33:20.977999+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_with_book.md", + "sha256": "59391763e075461d98bcbb5d5432e7b42a5ebec151da24a03d7df32b003e8335", + "size_bytes": 20238, + "source_mtime_utc": "2026-07-21T22:33:20.978908+00:00" + }, + { + "name": "eval_valley_ridge_vwap_ratio_exec_with_book_dashboard.png", + "sha256": "e3157782b76ad035a49cd7b9e301641905463c453aebaf98d1e42914b25becdf", + "size_bytes": 816590, + "source_mtime_utc": "2026-07-21T22:33:20.983125+00:00" + }, + { + "name": "eval_volume_peak_count_exec_basis_sanity.md", + "sha256": "1187924fb1550d314f1d1be03745a56084d1b238a007e728ee78dbf0580f6940", + "size_bytes": 3888, + "source_mtime_utc": "2026-07-21T22:33:20.986876+00:00" + }, + { + "name": "eval_volume_peak_count_exec_no_book.json", + "sha256": "acb4b510ec4f620115e4b5de1f7cbaaa9a076444a372691acb1ba41ac20b8ecd", + "size_bytes": 19773, + "source_mtime_utc": "2026-07-21T22:33:20.978030+00:00" + }, + { + "name": "eval_volume_peak_count_exec_no_book.md", + "sha256": "79c1389d26badf0c5e2dfb25fee367eae394ee454e736052961ce241736b48e5", + "size_bytes": 16506, + "source_mtime_utc": "2026-07-21T22:33:20.978963+00:00" + }, + { + "name": "eval_volume_peak_count_exec_no_book_dashboard.png", + "sha256": "5d0a6085e376d8ab5f3b82b78d0f4aa5f61c602012079e67841b14d4274de814", + "size_bytes": 666669, + "source_mtime_utc": "2026-07-21T22:33:20.983360+00:00" + }, + { + "name": "eval_volume_peak_count_exec_with_book.json", + "sha256": "f4603792251374f575d25c04dc27632e990cfda44c35fceb9c8527d33cb2383a", + "size_bytes": 21535, + "source_mtime_utc": "2026-07-21T22:33:20.978060+00:00" + }, + { + "name": "eval_volume_peak_count_exec_with_book.md", + "sha256": "4d09a3b744b0f8559ba54321e4b852dd65abad1ef487a206f8d2ef839f638e78", + "size_bytes": 17900, + "source_mtime_utc": "2026-07-21T22:33:20.979005+00:00" + }, + { + "name": "eval_volume_peak_count_exec_with_book_dashboard.png", + "sha256": "a5993d0637a158d21e1746545a91846100ed102c23af91272862276d2f3b5476", + "size_bytes": 676724, + "source_mtime_utc": "2026-07-21T22:33:20.983557+00:00" + } + ] +} diff --git a/qt/exec_baseline_freeze.py b/qt/exec_baseline_freeze.py new file mode 100644 index 0000000..c9c80a6 --- /dev/null +++ b/qt/exec_baseline_freeze.py @@ -0,0 +1,394 @@ +"""D5 exec-basis evaluation baseline freeze — the ONLY reconciliation reference. + +The 22 exec-basis evaluation artifacts (11 factors x {no_book, with_book}), their +22 rendered Markdown reports, the 11 ``_exec_basis_sanity.md`` coverage +disclosures and the 22 dashboards live in ``artifacts/reports/`` and are +**gitignored, single-copy, and not reproducible from current code**: regenerating +them needs a checkout of the pre-D5 tree plus a ~1.5h real run. The unified D5 +runner writes to those same filenames. Running it once would therefore destroy +the only thing D5 can be judged against, and any "reconciliation" performed +afterwards would compare the new output with a copy of itself — the +``compare_postmerge.py`` empty-reconciliation failure mode this repo has already +committed once and recorded in CLAUDE.md. + +So: freeze first, reconcile against the frozen copy, never against the live path. + +Structural isolation (NOT discipline) +------------------------------------- +``FrozenExecBaseline`` is the only supported reader, and it cannot be aimed at +the live artifacts directory even by accident: + +1. it requires a git-tracked manifest describing the tree it is given; +2. **every file it returns is sha256-verified against that manifest at read + time** — so a live artifact copied over a frozen one is a loud failure, not a + silent substitution. This is the load-bearing guard: it does not ask whether + the *path* looks right, it asks whether the *bytes* are the frozen bytes; +3. it refuses outright to resolve to the live reports directory. + +The manifest lives in git (``docs/factors/d5_exec_baseline_manifest.json``) while +the bytes live under gitignored ``artifacts/``. That split is deliberate and is +the reason (2) has teeth: an attacker-in-the-form-of-a-tired-engineer who +overwrites the frozen tree cannot also silently move the hashes, because the +hashes are under version control and show up in ``git diff``. + +Usage:: + + python -m qt.exec_baseline_freeze # freeze (idempotent, refuses to clobber) + python -m qt.exec_baseline_freeze --verify # re-verify the frozen tree, copy nothing +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import subprocess +import sys +from dataclasses import dataclass +from datetime import datetime, timezone +from pathlib import Path + +#: Repository-relative default locations. +DEFAULT_REPORTS_DIR = "artifacts/reports" +DEFAULT_FROZEN_ROOT = "artifacts/refactor_baseline/exec_baseline" +DEFAULT_MANIFEST = "docs/factors/d5_exec_baseline_manifest.json" + +#: Manifest schema tag. Bumping it invalidates every reader — intentional. +MANIFEST_SCHEMA = "d5-exec-baseline-freeze/1" + +#: Only the exec-basis family is frozen. The close-basis artifacts are NOT the +#: D5 reference (the unified runner is exec-only, design v3.2 decision 4) and +#: are left alone. +EXEC_MARKER = "_exec_" +FILENAME_PREFIX = "eval_" +FROZEN_SUFFIXES = (".json", ".md", ".png") + +#: The eleven closing minute factors, in the canonical order used by reports. +FACTORS = ( + "jump_amount_corr", + "minute_ideal_amplitude", + "amp_marginal_anomaly_vol", + "volume_peak_count", + "intraday_amp_cut", + "peak_interval_kurtosis", + "valley_relative_vwap", + "valley_ridge_vwap_ratio", + "ridge_minute_return", + "valley_price_quantile", + "peak_ridge_amount_ratio", +) +BOOKS = ("no_book", "with_book") + +#: Expected inventory: 22 JSON + 22 MD + 11 sanity MD + 22 PNG. +EXPECTED_FILE_COUNT = 77 + + +class BaselineIntegrityError(RuntimeError): + """Raised when frozen bytes do not match the git-tracked manifest.""" + + +def sha256_file(path: Path) -> str: + """Return the hex sha256 of a file's bytes. + + For opaque artifacts (JSON/Markdown/PNG) the byte hash IS canonical — unlike + parquet, there is no writer metadata to normalise away. + """ + digest = hashlib.sha256() + with open(path, "rb") as handle: + for chunk in iter(lambda: handle.read(1 << 20), b""): + digest.update(chunk) + return digest.hexdigest() + + +def expected_artifact_names() -> tuple[str, ...]: + """The 77 exec-basis artifact names this freeze is defined over. + + Derived from the factor list rather than from a directory listing, so a + missing file is a loud absence instead of a quietly smaller baseline. + """ + names: list[str] = [] + for factor in FACTORS: + names.append(f"eval_{factor}_exec_basis_sanity.md") + for book in BOOKS: + stem = f"eval_{factor}_exec_{book}" + names.extend((f"{stem}.json", f"{stem}.md", f"{stem}_dashboard.png")) + return tuple(sorted(names)) + + +def discover_exec_artifacts(reports_dir: Path) -> list[Path]: + """List the exec-basis artifacts actually present in ``reports_dir``.""" + found = [ + path + for path in sorted(reports_dir.iterdir()) + if path.is_file() + and path.name.startswith(FILENAME_PREFIX) + and EXEC_MARKER in path.name + and path.suffix in FROZEN_SUFFIXES + ] + return found + + +def _git_head(repo_root: Path) -> str: + try: + out = subprocess.run( + ["git", "-C", str(repo_root), "rev-parse", "HEAD"], + capture_output=True, + text=True, + check=True, + ) + return out.stdout.strip() + except (subprocess.CalledProcessError, OSError): # pragma: no cover - env dependent + return "unknown" + + +def _atomic_write_bytes(path: Path, payload: bytes) -> None: + """tmp + os.replace, the repository's durable-write pattern.""" + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(f".{path.name}.tmp{os.getpid()}") + try: + with open(tmp, "wb") as handle: + handle.write(payload) + handle.flush() + os.fsync(handle.fileno()) + os.replace(tmp, path) + finally: + if tmp.exists(): # pragma: no cover - only on write failure + tmp.unlink() + + +@dataclass(frozen=True) +class FreezeResult: + copied: tuple[str, ...] + already_frozen: tuple[str, ...] + manifest_path: Path + frozen_root: Path + + +def freeze( + reports_dir: Path, + frozen_root: Path, + manifest_path: Path, + *, + repo_root: Path, + source_note: str = "", +) -> FreezeResult: + """Copy the exec-basis artifacts into ``frozen_root`` and write the manifest. + + Idempotent and non-destructive: a destination that already exists is + verified against the freshly hashed source and left alone; a MISMATCH is an + error, never an overwrite. Freezing twice is safe; freezing over a different + baseline is impossible. + """ + reports_dir = reports_dir.resolve() + frozen_root = frozen_root.resolve() + if frozen_root == reports_dir or frozen_root.is_relative_to(reports_dir): + raise ValueError( + f"refusing to freeze into the live reports directory: {frozen_root}" + ) + + expected = set(expected_artifact_names()) + present = {path.name: path for path in discover_exec_artifacts(reports_dir)} + missing = sorted(expected - set(present)) + if missing: + raise FileNotFoundError( + f"{len(missing)} expected exec-basis artifact(s) absent from " + f"{reports_dir}: {missing[:5]}{' ...' if len(missing) > 5 else ''}" + ) + unexpected = sorted(set(present) - expected) + if unexpected: + raise RuntimeError( + "unexpected exec-basis artifact(s) present — the freeze inventory is " + f"defined by {len(expected)} names and refuses to guess: {unexpected}" + ) + + frozen_root.mkdir(parents=True, exist_ok=True) + entries: list[dict[str, object]] = [] + copied: list[str] = [] + already: list[str] = [] + + for name in sorted(expected): + src = present[name] + payload = src.read_bytes() + digest = hashlib.sha256(payload).hexdigest() + dst = frozen_root / name + if dst.exists(): + existing = sha256_file(dst) + if existing != digest: + raise BaselineIntegrityError( + f"{name}: frozen copy ({existing[:16]}) differs from the live " + f"artifact ({digest[:16]}). The frozen baseline is NOT " + "overwritten. Investigate before proceeding — one of the two " + "is not the PR #79 baseline." + ) + already.append(name) + else: + _atomic_write_bytes(dst, payload) + copied.append(name) + + stat = src.stat() + entries.append( + { + "name": name, + "sha256": digest, + "size_bytes": stat.st_size, + "source_mtime_utc": datetime.fromtimestamp( + stat.st_mtime, tz=timezone.utc + ).isoformat(), + } + ) + + manifest = { + "schema": MANIFEST_SCHEMA, + "frozen_at_utc": datetime.now(timezone.utc).isoformat(), + "frozen_at_git_head": _git_head(repo_root), + # Logical, repo-relative locations. The on-disk paths resolve through a + # gitignored symlink on some checkouts; recording the resolved absolute + # path would make this git-tracked manifest machine-specific. + "source_dir": DEFAULT_REPORTS_DIR, + "frozen_root": DEFAULT_FROZEN_ROOT, + "file_count": len(entries), + "factors": list(FACTORS), + "books": list(BOOKS), + "source_note": source_note, + "files": entries, + } + _atomic_write_bytes( + manifest_path, (json.dumps(manifest, indent=2, sort_keys=False) + "\n").encode() + ) + return FreezeResult( + copied=tuple(copied), + already_frozen=tuple(already), + manifest_path=manifest_path, + frozen_root=frozen_root, + ) + + +class FrozenExecBaseline: + """Verified read-only accessor for the frozen exec-basis baseline. + + Every read is checked against the git-tracked manifest, so this object + cannot serve bytes that are not the frozen bytes — including the case where + the caller aims it at a directory of freshly generated artifacts. + """ + + def __init__(self, frozen_root: Path, manifest_path: Path) -> None: + manifest_path = Path(manifest_path) + if not manifest_path.exists(): + raise FileNotFoundError( + f"no frozen-baseline manifest at {manifest_path}; the baseline " + "must be frozen (and its manifest committed) before any " + "reconciliation can read it" + ) + manifest = json.loads(manifest_path.read_text()) + if manifest.get("schema") != MANIFEST_SCHEMA: + raise BaselineIntegrityError( + f"manifest schema {manifest.get('schema')!r} != {MANIFEST_SCHEMA!r}" + ) + + root = Path(frozen_root).resolve() + # Refuse the live path outright. This is the cheap guard; the hash check + # below is the one that actually cannot be talked around. + if root.name == "reports" and root.parent.name == "artifacts": + raise BaselineIntegrityError( + f"refusing to read the LIVE artifacts directory as a frozen " + f"baseline: {root}. Reconciliation reads frozen copies only." + ) + if not root.is_dir(): + raise FileNotFoundError(f"frozen baseline root not found: {root}") + + self.root = root + self.manifest_path = manifest_path + self._expected: dict[str, str] = { + str(entry["name"]): str(entry["sha256"]) for entry in manifest["files"] + } + self.frozen_at_git_head = manifest.get("frozen_at_git_head", "unknown") + self.file_count = int(manifest.get("file_count", len(self._expected))) + + @property + def names(self) -> tuple[str, ...]: + return tuple(sorted(self._expected)) + + def read_bytes(self, name: str) -> bytes: + """Return the frozen bytes of ``name``, or raise.""" + if name not in self._expected: + raise KeyError(f"{name!r} is not part of the frozen exec baseline") + path = self.root / name + if not path.exists(): + raise FileNotFoundError(f"frozen artifact missing from disk: {path}") + payload = path.read_bytes() + digest = hashlib.sha256(payload).hexdigest() + if digest != self._expected[name]: + raise BaselineIntegrityError( + f"{name}: on-disk sha256 {digest[:16]} != manifest " + f"{self._expected[name][:16]}. The frozen baseline has been " + "modified; reconciliation against it would be meaningless." + ) + return payload + + def read_text(self, name: str) -> str: + return self.read_bytes(name).decode("utf-8") + + def read_json(self, name: str) -> dict: + return json.loads(self.read_text(name)) + + def report_json(self, factor: str, book: str) -> dict: + return self.read_json(f"eval_{factor}_exec_{book}.json") + + def verify_all(self) -> tuple[int, list[str]]: + """Verify every manifest entry. Returns (ok_count, problems).""" + problems: list[str] = [] + ok = 0 + for name in self.names: + try: + self.read_bytes(name) + ok += 1 + except (BaselineIntegrityError, FileNotFoundError) as exc: + problems.append(f"{name}: {exc}") + return ok, problems + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--repo-root", default=".") + parser.add_argument("--reports-dir", default=None) + parser.add_argument("--frozen-root", default=None) + parser.add_argument("--manifest", default=None) + parser.add_argument( + "--verify", + action="store_true", + help="verify the frozen tree against the manifest; copy nothing", + ) + parser.add_argument("--source-note", default="") + args = parser.parse_args(argv) + + repo_root = Path(args.repo_root).resolve() + reports_dir = Path(args.reports_dir or repo_root / DEFAULT_REPORTS_DIR) + frozen_root = Path(args.frozen_root or repo_root / DEFAULT_FROZEN_ROOT) + manifest_path = Path(args.manifest or repo_root / DEFAULT_MANIFEST) + + if args.verify: + baseline = FrozenExecBaseline(frozen_root, manifest_path) + ok, problems = baseline.verify_all() + print(f"verified {ok}/{baseline.file_count} frozen artifacts") + print(f"frozen at git head: {baseline.frozen_at_git_head}") + for problem in problems: + print(f" PROBLEM {problem}") + return 1 if problems else 0 + + result = freeze( + reports_dir, + frozen_root, + manifest_path, + repo_root=repo_root, + source_note=args.source_note, + ) + print(f"frozen root : {result.frozen_root}") + print(f"manifest : {result.manifest_path}") + print(f"copied : {len(result.copied)}") + print(f"already ok : {len(result.already_frozen)}") + return 0 + + +if __name__ == "__main__": # pragma: no cover + raise SystemExit(main(sys.argv[1:])) diff --git a/tests/test_exec_baseline_freeze.py b/tests/test_exec_baseline_freeze.py new file mode 100644 index 0000000..3b60a6f --- /dev/null +++ b/tests/test_exec_baseline_freeze.py @@ -0,0 +1,240 @@ +"""D5 C1: the frozen exec-basis baseline must be unforgeable and unclobberable. + +The frozen 77-artifact tree is the ONLY thing the D5 reconciliation can be +judged against, and the unified runner writes to the same filenames it was +copied from. These tests lock the two properties that make the reconciliation +capable of failing at all: + +* the reader serves frozen bytes or raises — it can never silently serve + freshly generated artifacts (the ``compare_postmerge.py`` empty-reconciliation + failure mode recorded in CLAUDE.md); +* freezing never overwrites an existing baseline with different content. + +Every "X raises" test here is paired with a real, non-vacuous mutation. (The +first draft of the tamper test replaced a byte string that did not occur in the +file, so it mutated nothing and "passed" against a guard that had not been +exercised — the exact trap this suite exists to avoid.) +""" + +from __future__ import annotations + +import hashlib +import json +import shutil +from pathlib import Path + +import pytest + +from qt.exec_baseline_freeze import ( + EXPECTED_FILE_COUNT, + MANIFEST_SCHEMA, + BaselineIntegrityError, + FrozenExecBaseline, + expected_artifact_names, + freeze, + sha256_file, +) + +REPO_ROOT = Path(__file__).resolve().parents[1] +MANIFEST = REPO_ROOT / "docs/factors/d5_exec_baseline_manifest.json" + + +def _fake_reports(root: Path) -> Path: + """A synthetic 'live reports' dir holding the full expected inventory.""" + reports = root / "reports" + reports.mkdir(parents=True) + for i, name in enumerate(expected_artifact_names()): + (reports / name).write_bytes(f"payload-{i}-{name}".encode()) + return reports + + +def _freeze_fake(tmp_path: Path) -> tuple[Path, Path, Path]: + reports = _fake_reports(tmp_path) + frozen = tmp_path / "frozen" + manifest = tmp_path / "manifest.json" + freeze(reports, frozen, manifest, repo_root=tmp_path) + return reports, frozen, manifest + + +# -------------------------------------------------------------------------- +# inventory +# -------------------------------------------------------------------------- + + +def test_exec_baseline_inventory_is_77_derived_from_the_factor_list() -> None: + names = expected_artifact_names() + assert len(names) == EXPECTED_FILE_COUNT == 77 + assert len(set(names)) == len(names) + assert sum(n.endswith(".json") for n in names) == 22 + assert sum(n.endswith("_dashboard.png") for n in names) == 22 + assert sum(n.endswith("_basis_sanity.md") for n in names) == 11 + # 22 report .md = 44 total .md minus the 11 sanity files... 22 + 11 = 33 + assert sum(n.endswith(".md") for n in names) == 33 + + +def test_exec_baseline_freeze_refuses_a_short_inventory(tmp_path: Path) -> None: + """A missing artifact is a loud absence, not a quietly smaller baseline.""" + reports = _fake_reports(tmp_path) + (reports / "eval_jump_amount_corr_exec_no_book.json").unlink() + with pytest.raises(FileNotFoundError, match="absent"): + freeze(reports, tmp_path / "frozen", tmp_path / "m.json", repo_root=tmp_path) + + +def test_exec_baseline_freeze_refuses_unexpected_members(tmp_path: Path) -> None: + reports = _fake_reports(tmp_path) + (reports / "eval_surprise_exec_no_book.json").write_bytes(b"x") + with pytest.raises(RuntimeError, match="unexpected"): + freeze(reports, tmp_path / "frozen", tmp_path / "m.json", repo_root=tmp_path) + + +# -------------------------------------------------------------------------- +# freeze: non-destructive, idempotent +# -------------------------------------------------------------------------- + + +def test_exec_baseline_freeze_copies_then_is_idempotent(tmp_path: Path) -> None: + reports, frozen, manifest = _freeze_fake(tmp_path) + assert len(list(frozen.iterdir())) == EXPECTED_FILE_COUNT + + again = freeze(reports, frozen, manifest, repo_root=tmp_path) + assert again.copied == () + assert len(again.already_frozen) == EXPECTED_FILE_COUNT + + +def test_exec_baseline_freeze_never_overwrites_a_differing_baseline( + tmp_path: Path, +) -> None: + """MUTATION: change the LIVE artifact after freezing. Re-freeze must refuse. + + This is the scenario that matters — the unified runner rewriting a live + artifact must never be able to launder itself into the frozen baseline. + """ + reports, frozen, manifest = _freeze_fake(tmp_path) + victim = reports / "eval_ridge_minute_return_exec_with_book.json" + before = (frozen / victim.name).read_bytes() + victim.write_bytes(victim.read_bytes() + b"!") # real, non-vacuous change + + with pytest.raises(BaselineIntegrityError, match="differs from the live"): + freeze(reports, frozen, manifest, repo_root=tmp_path) + assert (frozen / victim.name).read_bytes() == before # untouched + + +def test_exec_baseline_freeze_refuses_to_target_the_source_dir(tmp_path: Path) -> None: + reports = _fake_reports(tmp_path) + with pytest.raises(ValueError, match="live reports directory"): + freeze(reports, reports, tmp_path / "m.json", repo_root=tmp_path) + + +# -------------------------------------------------------------------------- +# reader: structural isolation +# -------------------------------------------------------------------------- + + +def test_exec_baseline_reader_rejects_tampered_bytes(tmp_path: Path) -> None: + """MUTATION: flip one byte of a frozen file; the reader must refuse it.""" + _, frozen, manifest = _freeze_fake(tmp_path) + name = "eval_jump_amount_corr_exec_no_book.json" + victim = frozen / name + before = victim.read_bytes() + after = before[:-1] + bytes([before[-1] ^ 0x01]) + assert after != before, "mutation must actually mutate" + assert hashlib.sha256(after).hexdigest() != hashlib.sha256(before).hexdigest() + victim.write_bytes(after) + + baseline = FrozenExecBaseline(frozen, manifest) + with pytest.raises(BaselineIntegrityError, match="on-disk sha256"): + baseline.read_bytes(name) + + ok, problems = baseline.verify_all() + assert (ok, len(problems)) == (EXPECTED_FILE_COUNT - 1, 1) + # the guard is targeted, not a blanket failure + baseline.read_bytes("eval_valley_price_quantile_exec_no_book.json") + + +def test_exec_baseline_reader_rejects_a_missing_file(tmp_path: Path) -> None: + _, frozen, manifest = _freeze_fake(tmp_path) + (frozen / "eval_volume_peak_count_exec_no_book.json").unlink() + baseline = FrozenExecBaseline(frozen, manifest) + with pytest.raises(FileNotFoundError): + baseline.read_bytes("eval_volume_peak_count_exec_no_book.json") + + +def test_exec_baseline_reader_refuses_the_live_reports_directory() -> None: + """Aiming the reader at artifacts/reports/ is refused by path, before hashing.""" + with pytest.raises(BaselineIntegrityError, match="LIVE artifacts directory"): + FrozenExecBaseline(REPO_ROOT / "artifacts" / "reports", MANIFEST) + + +def test_exec_baseline_reader_requires_a_manifest(tmp_path: Path) -> None: + _, frozen, _ = _freeze_fake(tmp_path) + with pytest.raises(FileNotFoundError, match="manifest"): + FrozenExecBaseline(frozen, tmp_path / "nope.json") + + +def test_exec_baseline_reader_rejects_a_foreign_manifest_schema(tmp_path: Path) -> None: + _, frozen, manifest = _freeze_fake(tmp_path) + payload = json.loads(manifest.read_text()) + payload["schema"] = "something-else/9" + manifest.write_text(json.dumps(payload)) + with pytest.raises(BaselineIntegrityError, match="schema"): + FrozenExecBaseline(frozen, manifest) + + +def test_exec_baseline_reader_rejects_unknown_names(tmp_path: Path) -> None: + _, frozen, manifest = _freeze_fake(tmp_path) + baseline = FrozenExecBaseline(frozen, manifest) + with pytest.raises(KeyError): + baseline.read_bytes("eval_not_a_factor_exec_no_book.json") + + +# -------------------------------------------------------------------------- +# the real, committed baseline +# -------------------------------------------------------------------------- + + +def test_committed_manifest_describes_the_real_frozen_baseline() -> None: + """The git-tracked manifest is well-formed and complete.""" + payload = json.loads(MANIFEST.read_text()) + assert payload["schema"] == MANIFEST_SCHEMA + assert payload["file_count"] == EXPECTED_FILE_COUNT + assert len(payload["files"]) == EXPECTED_FILE_COUNT + assert {f["name"] for f in payload["files"]} == set(expected_artifact_names()) + assert all(len(f["sha256"]) == 64 for f in payload["files"]) + # repo-relative, machine-independent + assert not payload["source_dir"].startswith("/") + assert not payload["frozen_root"].startswith("/") + + +@pytest.mark.skipif( + not (REPO_ROOT / "artifacts/refactor_baseline/exec_baseline").is_dir(), + reason="frozen baseline bytes are gitignored; present only on the run host", +) +def test_real_frozen_baseline_verifies_against_the_committed_manifest() -> None: + baseline = FrozenExecBaseline( + REPO_ROOT / "artifacts/refactor_baseline/exec_baseline", MANIFEST + ) + ok, problems = baseline.verify_all() + assert problems == [] + assert ok == EXPECTED_FILE_COUNT + + report = baseline.report_json("jump_amount_corr", "no_book") + assert set(report) >= {"sections", "spec", "thresholds", "verdict"} + + +def test_sha256_file_matches_hashlib(tmp_path: Path) -> None: + path = tmp_path / "blob.bin" + payload = b"x" * (3 << 20) + b"tail" # spans the 1MiB read chunks + path.write_bytes(payload) + assert sha256_file(path) == hashlib.sha256(payload).hexdigest() + + +def test_frozen_tree_is_not_a_git_tracked_path(tmp_path: Path) -> None: + """The bytes stay under gitignored artifacts/; only hashes are committed. + + That split is what gives the tamper check teeth: hashes move only through + a reviewable ``git diff``. + """ + _, frozen, manifest = _freeze_fake(tmp_path) + shutil.rmtree(frozen) + # manifest survives independently of the bytes it describes + assert json.loads(manifest.read_text())["file_count"] == EXPECTED_FILE_COUNT From 559a27d10b58f1fb3113d7184271d3d7de76e073 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Sat, 25 Jul 2026 06:56:17 -0700 Subject: [PATCH 2/5] feat(qt): probe whether the D4 materializer can run the D5 evaluation universe It cannot, and the margin is not close: a whole-universe floor-depth load needs an extrapolated 122 GB against 56 GB of available RAM (2.19x). Measured before committing to a multi-hour run rather than discovered ten minutes into one. Three findings, all reproducible via python -m qt.saturation_probe (cache-only, live_calls=0 asserted in the output): 1. Pooled saturation reaching the declared floor is STRUCTURAL, not incidental. The termination criterion requires every REQUESTED symbol to accumulate lookback_days valid days at or before emit_start, and 84 of the 995 CSI500 evaluation symbols have their first 1min bar on or after emit_start -- they were not listed yet, so no depth of backward expansion can satisfy them. One such symbol forces the whole universe to the 2015-01-05 floor, and a five-year CSI500 window always contains some: the index keeps admitting new listings. Floor depth is 2.01x the evaluation window on disk (16.43 vs 8.18 GB). 2. The problem is not confined to pooled factors. The bounded path loads the whole universe in one frame too (materialize.py:262), and the evaluation window alone measures an extrapolated 52.7 GB -- 0.94x available RAM, before any factor computation, once per factor. Per-symbol streaming is a structural prerequisite for all eleven factors, not a pooled-only optimization. The eleven old runners never hit this because they read symbol by symbol and aggregate to daily immediately; loading the universe at once is a D4 shape that a 40-symbol smoke could not expose. 3. The fix has a right and a wrong incision point, decided by measurement rather than assumption. Whole-universe versus per-symbol-concatenated, on real bars: ridge_minute_return and volume_peak_count agree bit-for-bit (max|diff| 0.0, identical NaN sets), while intraday_amp_cut flips all 1692 rows to NaN, because its step-4 cross-sectional z-score needs at least AMP_CUT_MIN_CROSS_SECTION=10 finite pairs on a date and a one-symbol cross-section is all-NaN by definition. So the per-symbol split must happen before that combine, not around it -- which is how the factor module and the old runner are already organised. The accompanying note argues the AMP_CUT coupling explicitly, as required: D4's warning is against per-symbol TRUNCATION, which really does move the cross-section; per-symbol SATURATION keeps every symbol's terminal depth unchanged and only stops forcing the other 911 to travel with the deepest one. That argument's load-bearing premise is measured for the ten per-symbol-pure factors and NOT yet measured for intraday_amp_cut, and the note says so: the equivalence is not to be treated as verified until that reconciliation is run. No engine code is changed here. This commit measures and documents. --- docs/factors/d5_saturation_feasibility.md | 136 ++++++++++++++ qt/saturation_probe.py | 219 ++++++++++++++++++++++ 2 files changed, 355 insertions(+) create mode 100644 docs/factors/d5_saturation_feasibility.md create mode 100644 qt/saturation_probe.py diff --git a/docs/factors/d5_saturation_feasibility.md b/docs/factors/d5_saturation_feasibility.md new file mode 100644 index 0000000..3865271 --- /dev/null +++ b/docs/factors/d5_saturation_feasibility.md @@ -0,0 +1,136 @@ +# D5 前置探测：D4 materializer 能否跑真实评估 universe + +> **状态**：D5 第二个交付物，**先于任何长 run**（lead 指令：「宁可报『这条路走不通、原因如下』， +> 也不要跑一个 2 小时后失败或产出错值的 run」）。 +> 复验：`python -m qt.saturation_probe`（cache-only，实测 `live_calls=0`）。 +> 全部数字为**实测**，标注为外推的才是外推。 + +--- + +## 结论先行 + +**D4 的 materializer 跑不了 D5 的评估 universe。** 不是慢，是 **2.19× 于本机可用内存**。 +它在 40 票热路径 smoke 上验收，评估面是 **995 票**，而它把整个 universe 装进**一个** frame。 + +且这不止影响 pooled 因子：**bounded 路径同样整 universe 一次性加载** +（`factors/materialize.py:262`），评估窗口本身就要 **52.7 GB**，已经贴着 56 GB 可用内存。 +所以「per-symbol 流式化」不是 pooled 因子的局部优化，**是 11 个因子全都需要的结构前提**。 + +--- + +## 一、饱和为什么必然走到 floor（不是偶发，是构造性的） + +`_pooled_pool_saturated` 要求**每一个被请求的 symbol** 在锁定子窗口里、在 `emit_start` +当天或之前攒够 `lookback_days` 个有效日，否则否决终止（D4 review HIGH 的修法，正确且 +故意保守）。 + +**实测**：评估 universe 995 票中，**84 票的第一根 1min bar 在 `emit_start` 2021-07-01 +当天或之后**（最晚到 2026 年）。这些票**无论往回加载多深都攒不出 emit_start 之前的有效日** +——因为那时它们还没上市。于是判据永远为 False，循环一路扩到声明 floor `2015-01-05`。 + +| 首根 bar 年份 | 票数 | +|---|---| +| 2015 | 668 | +| 2016–2020 | 223 | +| **2021（含 ≥07-01 的）** | 39 | +| **2022–2026** | **65** | + +这是 CSI500 的**正常构成**（指数持续纳入新上市公司），不是数据缺陷。**换言之：只要 +universe 里有一只在窗口内上市的票，pooled 加载就一定打到 floor。** 对 CSI500 五年窗口， +这是必然而非可能。 + +## 二、代价（实测 + 线性外推，外推处已标注） + +**磁盘**（精确，遍历 month 分区）： + +| 范围 | 分区文件 | 字节 | +|---|---|---| +| 评估窗口 2021-07..2026-06 | 57,542 | **8.18 GB** | +| floor 深度 2015-01..2026-06 | 117,971 | **16.43 GB**（2.01×） | + +**内存/耗时**（20 票**实测**，× 49.75 **外推**——单帧 concat 的线性外推在这里是保守的， +真实 concat 峰值更高）： + +| 加载 | 实测（20 票） | 外推（995 票） | +|---|---|---| +| 评估窗口 | 5.69M 行 / **1.06 GB** frame / 峰值 RSS 2.50 GB / 5.3s | 283M 行 / **52.7 GB** / ~4.4 min | +| floor 深度 | 13.21M 行 / **2.46 GB** frame / 峰值 RSS 5.57 GB / 12.7s | 657M 行 / **122.2 GB** / ~10.5 min | + +本机 `free`：总 62 GB，**可用 56 GB**。 + +- floor 深度 **122.2 GB / 56 GB = 2.19×** → **OOM，不可行**。 +- 评估窗口 **52.7 GB / 56 GB = 0.94×** → 名义装得下，但那是**因子计算开始之前**的裸 bar + 帧，且 8 个 pooled 因子每个都要一次。**实际同样不可行。** + +**为什么旧 runner 没这个问题**：11 个旧 runner 的 `_load_*_panel` **逐 symbol** 读、读完 +立刻聚合到日频，**从不同时持有整个 universe 的分钟 bar**。D4 的 materializer 改成了整批 +加载——这是 D4 引入的新形态，在 40 票上看不出来。 + +## 三、修法：per-symbol 流式化，且切口必须在 cross-section combine **之前** + +**实测的决定性证据**（12 票真实数据，整 universe 算 vs 逐 symbol 算再拼）： + +| 因子 | index 相同 | max\|diff\| | NaN 集合差异 | 判定 | +|---|---|---|---|---| +| `ridge_minute_return_20`（pooled） | True | **0.000e+00** | **0** | **SAFE** | +| `volume_peak_count_20`（pooled） | True | **0.000e+00** | **0** | **SAFE** | +| `intraday_amp_cut_10` | True | 0.000e+00 | **1692（全部行）** | **NOT SAFE** | + +前两个**逐位相同**——per-symbol 纯性在真实数据上得到确认（不是靠既有 isolation 测试推断）。 + +`intraday_amp_cut` **全部 1692 行翻成 NaN**：它的第 4 步是**按日截面 z-score**，要求当日 +finite 对数 ≥ `AMP_CUT_MIN_CROSS_SECTION=10`，而**单票截面 n=1 恒 < 10 → 定义上全 NaN**。 +这正是 D4 docstring 警告的耦合，实测复现。 + +**因此切口在哪里是有对错的**： + +- ❌ 把**整个因子**逐 symbol 算再拼 → `intraday_amp_cut` 全毁。 +- ✅ 逐 symbol 算到**该因子的 per-symbol 中间量**，拼成全 universe 面板，**再做那一次 + 截面 combine**。`intraday_amp_cut` 的模块结构本来就是这样组织的 + （`factors/compute/minute/intraday_amp_cut.py` docstring：先出两列 `(V_mean, V_std)` + 面板，**再 `combine_amp_cut_cross_section` 一次**），旧 runner 亦然。 + +### 与 `AMP_CUT_MIN_CROSS_SECTION` 耦合的显式论证（lead 要求单独论证，不许当纯优化） + +D4 的警告原文是「per-symbol 加载深度会改 `intraday_amp_cut` 的截面 z-score 组成」。 +**该警告针对的是 per-symbol *截断*，不是 per-symbol *饱和***，两者必须分开： + +- **per-symbol 截断**（每票只加载「它大概需要」的深度）：某票在日 d 本应 finite 却因加载 + 过浅变 NaN → 当日截面成员变少 → z-score 分母变、`min_cross_section` 门可能翻 → + **确实改值**。这条 D4 警告成立，**不可做**。 +- **per-symbol 饱和**（每票加载到**它自己**的饱和/floor，判据仍是 D4 那条结构性判据， + 只是逐票判而非全 universe 齐步走）：每票的 per-symbol 中间量**等于**它在全 universe + floor 加载下的值（上表前两行逐位实测 + 既有 cross-symbol isolation 性质）→ 日 d 的 + finite 成员集合**不变** → 截面 combine 输入不变 → **值不变**。 + +**即：把「全 universe 一起扩到 floor」换成「每票各自扩到自己的 floor」，对每票的 +saturation *终点* 没有放松**（84 票里任何一票该到 floor 的仍到 floor），只是不再强迫 +另外 911 票陪着一起到 floor。**这不是把判据放宽，是把判据从「universe 齐步」改成 +「逐票各自」，而逐票判据的终点集合与齐步判据完全相同。** + +⚠️ **仍属定义相邻、必须验收**：上述论证的承重前提是「per-symbol 中间量与加载几何无关」。 +该前提对 10 个 per-symbol 纯因子已实测（0.0 逐位）；对 `intraday_amp_cut` 需要**单独** +证明其 `(V_mean, V_std)` per-symbol 部分同样与几何无关，并用**全 universe floor 加载 +vs per-symbol 饱和加载**在小 universe 上做端到端逐格对账。**该对账未完成前，不得把 +per-symbol 饱和当作已验证的等价改写。** + +## 四、对 D5 计划的影响 + +1. **C5 的全量 run 在修好之前不可能执行**——不是「慢」，是 OOM。任何「先跑起来看看」 + 都会在 10 分钟的加载后死掉，或更糟：在评估窗口那档勉强不死而开始 swap。 +2. 修法（per-symbol 流式 + 切口在 combine 之前）**在 D4 已声明为 D5 工作** + （`materialize.py:397` "A per-symbol saturation start is the D5 optimization"）， + 故属 D5 范围，但**它是修改因子取值路径的改动，必须按 §五四腿验收**， + 不能作为「让 run 跑起来」的顺手改动混进去。 +3. `valley_price_quantile` 的 D5 绑定（C4 交付物）**继承同一路径**——lead 已明令 + 不许用固定深度 trim 绕过，per-symbol 饱和是它唯一合法的加载形态。 + +## 五、未做 / 留给后续 + +- **per-symbol 饱和的实现与四腿验收**：本探测只证明了「必须这么做」与「这么做在两个 + pooled 因子上逐位等价」，**没有实现它**，也没有对 `intraday_amp_cut` 做端到端对账。 +- 声明 floor 目前仍是硬编码常量 `qt/factor_hotpath_smoke.py::CACHE_MINUTE_DATA_START + = "2015-01-05"`，对所有 symbol 同值。**探测显示按票派生 floor 是有意义的**（668 票 + 确实从 2015 开始，但 327 票晚得多），但**改 floor 派生方式本身**同样是定义相邻变更， + 与 per-symbol 饱和是两件事，不要合并处理。 +- 本探测未测 `concat` 峰值与 GC 行为，线性外推对峰值是**乐观**的。 diff --git a/qt/saturation_probe.py b/qt/saturation_probe.py new file mode 100644 index 0000000..31b1f64 --- /dev/null +++ b/qt/saturation_probe.py @@ -0,0 +1,219 @@ +"""D5 feasibility probe: can the D4 materializer run the real evaluation universe? + +D4's materializer was accepted against a 40-symbol hot-path smoke. D5 needs it on +the eleven-factor evaluation plane (CSI500, 995 symbols, 2021-07-01..2026-06-30), +so before committing to a multi-hour run this probe MEASURES, on the real cache, +the four quantities that decide whether that run is possible at all: + +1. **listing depth** — per-symbol first 1min month, read from the parquet store's + directory layout (no data is read). A symbol whose bars begin at or after + ``emit_start`` can never accumulate ``lookback_days`` valid days before it, so + the pooled saturation criterion can never terminate early and the load expands + to the declared floor; +2. **on-disk volume** at the evaluation window vs at the floor; +3. **in-memory cost** of a whole-universe single-frame load, measured on a sample + and extrapolated linearly (the materializer loads every symbol into ONE frame: + ``sources.minute.minute_bars(list(symbols), ...)``); +4. **per-symbol equivalence** — whether materializing symbol-by-symbol and + concatenating reproduces the whole-universe values exactly. This is the + decisive question for the fix, and it is answered per factor rather than + assumed: ten factors are per-symbol pure, ``intraday_amp_cut`` is not (it + z-scores across the loaded cross-section, ``AMP_CUT_MIN_CROSS_SECTION=10``). + +Run: ``python -m qt.saturation_probe`` (cache-only; zero live calls). +""" + +from __future__ import annotations + +import argparse +import resource +import time +from pathlib import Path + +import pandas as pd + +from data.cache.intraday_cache import ENDPOINT as INTRADAY_ENDPOINT +from data.clean.intraday_schema import RAW_INTRADAY_FREQ +from factors import registry as factor_registry +from factors.compute.minute.binding import minute_raw_from_bars +from qt.factor_hotpath_smoke import CACHE_MINUTE_DATA_START, CacheMinuteProvider + +DEFAULT_CACHE_ROOT = "artifacts/cache/tushare/v1" +#: The frozen D1 panel is on the exact evaluation data plane, so its symbol list +#: IS the evaluation universe — no need to re-resolve PIT membership here. +DEFAULT_UNIVERSE_PANEL = "artifacts/refactor_baseline/panels/ridge_minute_return_20.parquet" +EVAL_START = pd.Timestamp("2021-07-01") +EVAL_END = pd.Timestamp("2026-06-30") +GB = 1024**3 + + +def universe_from_frozen_panel(path: str) -> list[str]: + return sorted(pd.read_parquet(path)["symbol"].unique().tolist()) + + +def first_minute_month(root: str, symbols: set[str]) -> pd.Series: + """Per-symbol earliest 1min month, from DIRECTORY STRUCTURE only.""" + base = Path(root) / INTRADAY_ENDPOINT / f"freq={RAW_INTRADAY_FREQ}" + first: dict[str, pd.Timestamp] = {} + for prefix in base.glob("symbol_prefix=*"): + for sym_dir in prefix.glob("symbol=*"): + sym = sym_dir.name.split("=", 1)[1] + if sym not in symbols: + continue + months = [ + pd.Timestamp( + year=int(yd.name.split("=", 1)[1]), + month=int(mf.name.split("=", 1)[1].replace(".parquet", "")), + day=1, + ) + for yd in sym_dir.glob("year=*") + for mf in yd.glob("month=*") + ] + if months: + first[sym] = min(months) + return pd.Series(first).sort_values() + + +def partition_bytes(root: str, symbols: set[str]) -> tuple[int, int]: + """(evaluation-window bytes, floor-depth bytes) of 1min month partitions.""" + base = Path(root) / INTRADAY_ENDPOINT / f"freq={RAW_INTRADAY_FREQ}" + ev = full = 0 + lo = (EVAL_START.year, EVAL_START.month) + hi = (EVAL_END.year, EVAL_END.month) + for prefix in base.glob("symbol_prefix=*"): + for sym_dir in prefix.glob("symbol=*"): + if sym_dir.name.split("=", 1)[1] not in symbols: + continue + for yd in sym_dir.glob("year=*"): + year = int(yd.name.split("=", 1)[1]) + for mf in yd.glob("month=*"): + month = int(mf.name.split("=", 1)[1].replace(".parquet", "")) + size = mf.stat().st_size + full += size + if lo <= (year, month) <= hi: + ev += size + return ev, full + + +def measure_load(provider, symbols: list[str], start, end) -> dict[str, float]: + t0 = time.time() + bars = provider.minute_bars(symbols, start, end) + elapsed = time.time() - t0 + out = { + "rows": float(len(bars)), + "frame_gb": float(bars.memory_usage(deep=True).sum()) / GB, + "peak_rss_gb": resource.getrusage(resource.RUSAGE_SELF).ru_maxrss / (1024**2), + "seconds": elapsed, + } + del bars + return out + + +def per_symbol_equivalence( + provider, symbols: list[str], start, end, factor_names: tuple[str, ...] +) -> list[dict]: + """Whole-universe vs per-symbol-concatenated, per factor, on real bars.""" + bars = provider.minute_bars(symbols, start, end) + rows = [] + for name in factor_names: + factor = factor_registry.build(name, {}) + whole = minute_raw_from_bars(factor, bars).sort_index() + parts = [] + for sym in symbols: + one = bars[bars.index.get_level_values("symbol") == sym] + if one.empty: + continue + parts.append(minute_raw_from_bars(factor, one)) + per = pd.concat(parts).sort_index() if parts else pd.Series(dtype=float) + entry: dict = {"factor": name, "rows": len(whole)} + if whole.index.equals(per.index): + both = whole.notna() & per.notna() + entry["index_same"] = True + entry["max_abs_diff"] = ( + float((whole[both] - per[both]).abs().max()) if both.any() else 0.0 + ) + entry["nan_set_diff"] = int((whole.isna() != per.isna()).sum()) + else: + entry["index_same"] = False + entry["max_abs_diff"] = float("nan") + entry["nan_set_diff"] = -1 + entry["per_symbol_safe"] = bool( + entry["index_same"] + and entry["nan_set_diff"] == 0 + and entry["max_abs_diff"] == 0.0 + ) + rows.append(entry) + return rows + + +def main(argv: list[str] | None = None) -> int: + parser = argparse.ArgumentParser(description=__doc__.splitlines()[0]) + parser.add_argument("--cache-root", default=DEFAULT_CACHE_ROOT) + parser.add_argument("--universe-panel", default=DEFAULT_UNIVERSE_PANEL) + parser.add_argument("--sample-symbols", type=int, default=20) + args = parser.parse_args(argv) + + symbols = universe_from_frozen_panel(args.universe_panel) + symset = set(symbols) + floor = pd.Timestamp(CACHE_MINUTE_DATA_START) + print(f"universe: {len(symbols)} symbols | emit {EVAL_START.date()}..{EVAL_END.date()}") + print(f"declared minute floor: {floor.date()}\n") + + print("== 1. listing depth ==") + first = first_minute_month(args.cache_root, symset) + late = int((first >= EVAL_START).sum()) + print(first.dt.year.value_counts().sort_index().to_string()) + print( + f"\nsymbols whose first 1min bar is ON/AFTER emit_start: {late}" + f" -> pooled saturation CANNOT terminate early; it expands to the floor\n" + ) + + print("== 2. on-disk volume ==") + ev, full = partition_bytes(args.cache_root, symset) + print(f"evaluation window : {ev / GB:7.2f} GB") + print(f"floor depth : {full / GB:7.2f} GB ({full / max(ev, 1):.2f}x)\n") + + print("== 3. whole-universe single-frame load (measured, then extrapolated) ==") + provider = CacheMinuteProvider(args.cache_root) + sample = symbols[: args.sample_symbols] + scale = len(symbols) / len(sample) + for label, start in (("eval-window", EVAL_START), ("floor-depth", floor)): + m = measure_load(provider, sample, start, EVAL_END) + print( + f"{label:12s} {len(sample):3d} syms: rows={m['rows']:>12,.0f} " + f"frame={m['frame_gb']:6.2f} GB peakRSS={m['peak_rss_gb']:6.2f} GB " + f"load={m['seconds']:6.1f}s" + ) + print( + f"{'':12s} x{scale:5.2f} -> rows={m['rows'] * scale / 1e6:7.0f}M " + f"frame={m['frame_gb'] * scale:7.1f} GB load={m['seconds'] * scale / 60:5.1f} min" + ) + print(f"\nlive minute calls so far: {provider.live_calls} (cache-only)\n") + + print("== 4. per-symbol equivalence (the fix's decisive question) ==") + probe_syms = symbols[:12] + rows = per_symbol_equivalence( + provider, + probe_syms, + pd.Timestamp("2021-03-01"), + pd.Timestamp("2021-09-30"), + ("ridge_minute_return_20", "volume_peak_count_20", "intraday_amp_cut_10"), + ) + for r in rows: + verdict = "SAFE" if r["per_symbol_safe"] else "NOT SAFE" + print( + f"{r['factor']:26s} rows={r['rows']:>6,} index_same={r['index_same']!s:5s} " + f"max|diff|={r['max_abs_diff']:.3e} nan_set_diff={r['nan_set_diff']:>5d} {verdict}" + ) + print( + "\nintraday_amp_cut is expected NOT SAFE at whole-factor granularity: its " + "step-4 cross-sectional z-score needs >= AMP_CUT_MIN_CROSS_SECTION symbols " + "on a date, and a one-symbol cross-section is all-NaN by definition. The " + "per-symbol split must therefore happen BEFORE that combine, not around it." + ) + print(f"live minute calls total: {provider.live_calls}") + return 0 + + +if __name__ == "__main__": # pragma: no cover + raise SystemExit(main()) From 5d2f31154424545cc12499230612f8c12725b8d9 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Sat, 25 Jul 2026 07:01:44 -0700 Subject: [PATCH 3/5] docs(factors): catalogue the eleven eval runners' differences before unifying them MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Design v3.2 §6.5 requires the eleven runners' differences to be catalogued BEFORE they are merged, because after the merge an uncatalogued difference is indistinguishable from a regression -- the rule is that finding one counts as an acceptance failure. This table is the difference whitelist C5's reconciliation will be read against. Method: AST body comparison across all eleven, a flattened YAML comparison of all eleven configs, and targeted reads of every diverging region. The headline is that only FOUR extension points are actually needed; the rest can be a fixed body, and §5 records the thirteen classes of meaningless drift being normalised so that normalisation is a decision on the record rather than a silent edit. The load-bearing finding is that a unified runner cannot assume the per-symbol reducer returns the factor. Six runners yield a plain series, three yield a series plus a diagnostics sink, and two (intraday_amp_cut, valley_price_quantile) do not produce the factor in the loop at all -- they emit a stats frame that a mandatory post-loop cross-sectional finalizer turns into the factor, and one of those finalizers also needs the daily close panel. That same cross-sectional step is why the D5 per-symbol streaming incision has to land before the combine, which ties this catalogue to the saturation feasibility note. Also closes the structural half of one of the three items CLAUDE.md lists as "not independently checked" from PR #79: execution_capacity passthrough is a single shared code path with no runner variation, and the disclosure dict deliberately omits the two keys the Tradable axis reads, so NOT_ASSESSED cannot vary per factor by construction. The other two items remain open. Seven defects are flagged and deliberately NOT fixed. Two were independently re-verified here rather than taken on trust: BUG 1 (CRITICAL, user-visible) -- three runners and the CLI describe a defect PR #74 already fixed. They claim the frozen layer computes sign*(gross-cost) and "is not patched"; standard.py:419 reads `sign * gross - base_cost`, with the old form described in the past tense. The v0.8 fix commit is an ancestor of the commit that last edited the ridge runner, so the text was already stale when it was last touched, and it prints to stdout on every PR-K run. This is the #76/#78/#82 failure mode again: behaviour changed, wording did not. BUG 2 (HIGH) -- aligned_spread_by_cost is extracted by two runners but no key of that name exists anywhere in analytics/. Verified against the frozen artifacts themselves: absent from both return_risk payloads, so both runners always report {}. Its only guard is `assert isinstance(m[...], dict)`, which {} satisfies unconditionally -- an impossible-to-fail test of exactly the kind the project methodology names. BUG 3 (HIGH) -- peak_ridge_amount_ratio has no runner test at all, leaving 122 lines of coverage-disclosure logic untested while all three sibling summarizers have counterfactual and empty-frame tests. BUG 6 is a direct warning to C6: a duplicate test function name across two files is currently harmless only because tests/__init__.py exists, so a consolidation that removes it would silently drop a test. Finally, one artifact difference that is expected and must be pre-registered or it will be chased as a D5 regression: the frozen spec block has 16 keys and current code emits 20, the four additions being requires/adjustment/ overnight_boundary/lookback_depth from PR #86 and #91, which reach the JSON automatically through sanitize_payload(vars(report.spec)). --- .../factors/d5_runner_difference_catalogue.md | 241 ++++++++++++++++++ 1 file changed, 241 insertions(+) create mode 100644 docs/factors/d5_runner_difference_catalogue.md diff --git a/docs/factors/d5_runner_difference_catalogue.md b/docs/factors/d5_runner_difference_catalogue.md new file mode 100644 index 0000000..afcb46d --- /dev/null +++ b/docs/factors/d5_runner_difference_catalogue.md @@ -0,0 +1,241 @@ +# D5 C2 —— 11 个 eval runner 的差异编目（统一前置） + +> **状态**：D5 交付物 C2。设计出处 v3.2 §六.5：11 个 runner 的差异**先编目再合并**， +> 合并后按 §五语义对账逐项归因——**「未编目的差异」算失败**。 +> 本表是 **C5 对账的差异白名单唯一来源**：对账中出现表外差异 = 验收不通过。 +> 编目方法：全 11 份 AST body 比对 + 11 份 config 展平 YAML 全量比对 + 逐个分歧区精读。 +> 路径均相对仓库根；行号为编目时实测。 + +**总量**：11 个 runner 共 6550 行（496–749 行/个）。 + +--- + +## 一、结论：只需要四个扩展点 + +| # | 扩展点 | 为什么不能做成固定 body | +|---|---|---| +| 1 | **因子构造 + per-symbol reducer，kwargs 不透明（2–9 个）** | 各因子预注册参数个数与名字都不同；且 reducer 返回**三种形态** | +| 2 | **可选的 post-loop 截面 finalizer** | 2 个因子的值**不是**逐 symbol 产出的；其中 1 个还要日频面板 | +| 3 | **可选的 add-Section 覆盖披露** | 4 个 runner 各有一份**形状不同**的稀缺性披露，且这是**故意的** | +| 4 | **metric 键集** | 三档 19/27/29，但**与因子无关**（见下），可收敛 | + +其余全部可归一（§四 C1–C13）。 + +## 二、(a) 真参数 —— 必须作为 runner 参数存活 + +### 2.1 per-symbol reduction 形态（**最承重的一条**） + +统一 runner **不能假设 per-symbol reducer 返回因子值**： + +| 形态 | runner 数 | 成员 | +|---|---|---| +| 纯 series | 6 | jump_amount_corr / minute_ideal_amplitude / amp_marginal_anomaly_vol / volume_peak_count / peak_interval_kurtosis / valley_relative_vwap | +| series + diagnostics sink | 3 | valley_ridge_vwap_ratio / ridge_minute_return / peak_ridge_amount_ratio | +| **stats frame + 强制截面 finalizer** | 2 | **intraday_amp_cut / valley_price_quantile** —— 循环**根本不产出因子** | + +### 2.2 post-loop 截面 finalizer（2 个） + +- `qt/eval_intraday_amp_cut.py:172-177` → `combine_amp_cut_cross_section(stats, min_cross_section=…)`（研报第 4 步截面 z-score）。其覆盖率判定测的是 `V_MEAN_COL` **与** `V_STD_COL` 双 finite（`:155`），不是"series 非 NaN"。 +- `qt/eval_valley_price_quantile.py:281-284` → `reversal_20(panel[["close"]])` + `residualize_on_reversal(raw, rev, min_cross_section=…)`。 + +> ⚠️ 与 D5 饱和探测（`docs/factors/d5_saturation_feasibility.md`）**直接耦合**： +> `intraday_amp_cut` 的截面步正是 per-symbol 流式化切口必须落在其**之前**的原因。 + +### 2.3 需要日频面板的 loader（1 个） + +`_load_valley_price_quantile_panel(cfg, symbols, spec, panel, logger, *, …)` +（`qt/eval_valley_price_quantile.py:202-207`，调用点 `:598`）——**唯一** 5 位置参数 loader， +其余 10 个都是 4 个。reversal 取自前复权面板 `panel['close']` 的 T-1。 +→ **reducer 签名必须能接住日频面板。** + +### 2.4 因子构造参数（全部为模块常量，无一 config 驱动；kwargs 个数 2–9，不能做固定列表） + +| runner | Factor 类 | lookback | 额外 loader kwargs | +|---|---|---|---| +| jump_amount_corr | JumpAmountCorrFactor | 20 | `min_pairs=10` | +| minute_ideal_amplitude | MinuteIdealAmplitudeFactor | 10 | `lam=0.25, min_minutes=1150` | +| amp_marginal_anomaly_vol | AmpMarginalAnomalyVolFactor | 20 | `min_pool=460, min_selected=20, sigma_k=1.0` | +| volume_peak_count | VolumePeakCountFactor | 20 | PRV-5 | +| intraday_amp_cut | IntradayAmpCutFactor | 10 | `lam=0.20, min_day_minutes=100, min_valid_days=6, min_cross_section=10` | +| peak_interval_kurtosis | PeakIntervalKurtosisFactor | 20 | PRV-5 + `min_intervals=20` | +| valley_relative_vwap | ValleyRelativeVwapFactor | 20 | PRV-5 + `min_valley_bars=20` | +| valley_ridge_vwap_ratio | ValleyRidgeVwapRatioFactor | 20 | PRV-5 + `min_valley_bars=20, min_ridge_bars=10` | +| ridge_minute_return | RidgeMinuteReturnFactor | 20 | PRV-5 + `min_ridge_bars=10` | +| valley_price_quantile | ValleyPriceQuantileFactor | 20 | PRV-5 + `min_valley_bars=20, min_cross_section=10, reversal_days=20` | +| peak_ridge_amount_ratio | PeakRidgeAmountRatioFactor | 20 | PRV-5 + `min_peak_bars=5, min_ridge_bars=10` | + +PRV-5 = `baseline_days=20, baseline_min_obs=10, sigma_k=1.0, min_valid_days=10, min_classifiable=100` +（`factors/compute/minute/primitives.py:55-59`，8 个 runner 共享）。 +**这些是预注册定义，参数化保留、不统一**（v3.2 §〇 总原则）。 + +## 三、(b) 真异构披露 —— 走 add-Section 扩展点 + +4 个 runner 各有一份稀缺性/中性化披露，**无共享基类**，4 个独立 frozen dataclass 各带 `render()`。 +两种机制： + +| runner | 类 | 定义行 | 机制 | +|---|---|---|---| +| valley_ridge_vwap_ratio | `RidgeCoverage` | `:110-233` | A：循环中 diagnostics sink（`:310`） | +| ridge_minute_return | `RidgeReturnCoverage` | `:120-253` | A：sink（`:328`） | +| peak_ridge_amount_ratio | `PeakCoverage` | `:122-243` | A：sink（`:320`） | +| valley_price_quantile | `NeutralizationCoverage` | `:121-182` | B：循环后从已装配面板算（`:285`），无 sink | + +**它们不是同一对象换字段名**——共有 `symbol_days / classifiable_days / valid_days / +days_below_classifiable_gate / min_classifiable`，其余各不相同： + +- `RidgeCoverage`：`ridge_percentiles / ridge_mean / valley_median`；两道 gate；反事实 `valid_days_at_valley_floor` +- `RidgeReturnCoverage`：另**追踪两个计数** `ridge_bars_mean/median`；额外属性 `return_guard_attrition`；反事实 floor `_COMPARISON_FLOOR = 20`（**硬编码**对齐 PR-J） +- `PeakCoverage`：反事实 floor `_COUNTERFACTUAL_PEAK_FLOOR = PEAK_RIDGE_MIN_RIDGE_BARS`（**派生**，非硬编码） +- `NeutralizationCoverage`：形状完全不同——`raw_rows/rev_rows/residual_rows/dates_total/dates_residualized/cross_section_min,median,max/raw_rev_spearman_mean` + +**披露措辞**（4 条 format 模板，同时进 run log 与 stdout）： +`eval_valley_ridge_vwap_ratio.py:151-168` / `eval_ridge_minute_return.py:173-189` / +`eval_peak_ridge_amount_ratio.py:164-177` / `eval_valley_price_quantile.py:293-301`。 +各类 docstring 均强调：**报出的 floor 必须是本次 run 实际施加的那个**，不是模块默认值。 + +**判定 (b) 而非 (c)**：这些是各因子**自己 gate 的实测事实**，量纲与语义都不同， +强行统一 = 丢信息。这正是 §3.6 的 add-Section 扩展点（"may ADD sections but never +drop a mandatory one"），**不是第二套 runner**（kill 档案 A8-F03）。 + +**一处归一**：PR-L 在 `qt/cli.py:525-530` 用**内联 f-string** 重渲染，而其三个兄弟走 +`.render()`（`:436` / `:479` / `:570`）→ 统一走 `.render()`。 + +## 四、全 11 份完全一致（可做固定 body，无参数） + +| 轴 | 值 | 证据 | +|---|---|---| +| **horizon** | `h = 1`，且**根本不是 runner 关切** | 来自 `FactorSpec.forward_return_horizon`，仅在共享模块 `qt/exec_basis_eval.py:213` 消费。**不要加 horizon 旋钮** | +| **`execution_capacity` 透传** | **零个 runner 设置**；仅共享模块 `qt/exec_basis_eval.py:305`(no_book) / `:315`(with_book)，同一 `disclosure` dict（`:97-151`） | 该 dict **故意省略** `tradable`/`capacity_sufficient` 两个 Tradable 轴会读的键 → 该轴恒 NOT_ASSESSED，模块 docstring `:21-27` 明写 | +| **因子簿** | `ValueFactor("value_ep"), ValueFactor("value_bp"), VolatilityFactor(window=20)` | `eval_jump_amount_corr.py:480-488`，**11 份 AST body 逐字节相同** | +| **费率/成本** | `fee_rate = cfg.cost.fee_rate`（配置 0.001）；`cost_scenarios = (1.0, 2.0, 4.0)` 11 份硬编码相同 | `qt/exec_basis_eval.py:304, 313` | +| **分钟读取窗口** | 11 份相同：`[data.start, data.end 23:59:59]` | **无 runner 向前扩窗做 warm-up** → 前 lookback_days 结构性 NaN（与 D4 饱和工作直接相关） | +| **process/中性化** | 因子一次 + 因子簿一次 `_process_factors`；`neutralization=("industry","size"), standardize="zscore", winsorize=None` | 唯一例外是 PR-L 的 reversal 残差化，发生在 loader **内部**、`_process_factors` **之前** | +| **报告/图输出** | `{stem}_no_book|_with_book` .md+.json + 2 PNG；exec 侧同形 | `eval_jump_amount_corr.py:221-228`；`qt/exec_basis_eval.py:321-330` | +| **`_exec_basis_sanity.md`** | **仅**共享模块 `qt/exec_basis_eval.py:271-283` 产出，11 份无差异，无 runner 代码触碰 | 渲染器 `qt/exec_basis_sanity.py:355+` | +| **config** | **D1 manifest 的声称经复核成立**：11 份展平 YAML 共 56 个 union 键，**恰有两个不同**：`project.name` 与 `data.output_name` | 其余逐值相同（CSI500 / 2021-07-01..2026-06-30 / split 2024-01-01 / fee 0.001 / quantiles 5 / top_n 50 / L1 / cache root / factors 列表） | + +**`execution_capacity` 附注**：CLAUDE.md 把「`execution_capacity` 透传是否在作者自带测试之外 +仍保持 Tradable 轴 `NOT_ASSESSED`」列为 **#79 未独立核**三项之一。本次编目**关闭了这一项的 +结构面**：透传是**单一共享代码路径、无 runner 变体、且不含任何能动 verdict 的键**， +故该性质**按构造**不可能逐因子变化。（另两项仍未核。） + +### metric 键集：三档，但**与因子无关** + +19 键（8 个）/ 27 键（ridge_minute_return，`:490-519`）/ 29 键（valley_price_quantile `:436-465`、 +peak_ridge_amount_ratio `:488-517`）。**多出的键全部取自每次 run 都产出的 `return_risk` / +`stability_cost` payload**——即「PR-K 之后开始多报几个」的**报告选择**，不是因子性质。 +→ **收敛到 29 键超集 减去 `aligned_spread_by_cost`**（该键不存在，见 BUG 2）。 + +## 五、(c) 无意义漂移 —— 已判定归一，逐条记录 + +| # | 漂移 | +|---|---| +| C1 | `_build_eval_config` 11 份不同 body、语义全同；差别仅 `oos is None` 错误串里的 runner 名。17 个 EvalConfig kwargs 逐字符相同 | +| C2 | `_check_preconditions` 同上，同 4 项检查，差别仅嵌入的 runner 名 | +| C3 | 11 个 result dataclass 名不同，14–16 字段相同 | +| C4 | 11 个私有 load-dataclass 名不同 | +| C5 | 局部变量命名 `jump_raw` / `amp_raw` / `factor_raw` | +| C6 | 11 个 `_load_*_panel` 函数名不同，11 行骨架相同 | +| C7 | 日志措辞（`rows read` 换行位置；`symbols with a value` vs `with a stat`） | +| C8 | `logger.info("eval config: …")` 单行 vs 折行 | +| C9 | `_write_report` 声明单行 vs 四行折行，body 相同 | +| C10 | docstring 88 vs 90 列重排——**任何两份 byte-diff 的主体其实是这个** | +| C11 | 仅 4 个大 runner import numpy（只被覆盖率 summarizer 用） | +| C12 | `qt/cli.py:206-580` 11 份 `_cmd_run_eval_*`，同 try/except + f-string | +| C13 | 两份相同 `_fmt` helper：`qt/cli.py:395-409` 与 `qt/exec_basis_eval.py:355-367` | + +## 六、发现的缺陷（**已标记，未修**） + +> 按 lead 指令「flag separately，do not fix」。以下 BUG 1/2 由本 agent **独立复核确认** +> （非仅采信编目），复核方式记在各条。 + +### BUG 1（CRITICAL，用户可见）—— runner 描述了一个 **PR #74 已经修掉**的缺陷 + +**声称**（`qt/eval_ridge_minute_return.py:477-481`，并在 `:49-56`、`:705-708` 复述）： +> "The frozen layer's `aligned_spread_*` computes `sign * (gross - cost)` … costs are added +> back rather than deducted … **The frozen layer is not patched**" + +**实际**（`analytics/eval/standard.py:419`，本 agent 直接读取确认）： +``` +aligned_base = sign * gross - base_cost +``` +上方注释以**过去时**描述旧缺陷（"The pre-v0.8 form `sign * net` expanded, at sign=-1, to +`-gross + cost`"）——即 PR #74 的 v0.8 修复**已在位**。 + +**且它每次 PR-K run 都打到 stdout**（`qt/cli.py:443` +"net long-short by cost (aligned_spread_* UNRELIABLE at sign=-1)"）并写进 run log +（`eval_ridge_minute_return.py:708`）。另有四处断言同一死声称的否定形式： +`qt/cli.py:484-485`、`:535-536`、`eval_valley_price_quantile.py:57-59` 与 `:422-424`、 +`eval_peak_ridge_amount_ratio.py:475`。 + +**祖先关系已验**：`08829bd`（v0.8 cost-correct aligned spread）**既是** HEAD 祖先、 +**也是** `36c8c85`（最后触碰 ridge runner 的 commit）的祖先 → **该文字在最后一次被编辑时 +就已经是陈旧的**。 + +**这正是 #76/#78/#82 的失效模式**：行为改了、措辞没改。按项目自定规则，修法是 +**author-once（只写一遍、其余组合引用）**，不是再复制一份正确措辞。 +→ **归属**：本条与 D5 统一 runner 天然同批（11 份措辞坍缩成一份），但**它是独立缺陷， +不因 D5 而存在**；若 D5 拆期，本条应单独修。 + +### BUG 2（HIGH）—— `aligned_spread_by_cost` 被提取，而 frozen 层从不产出它 + +`eval_valley_price_quantile.py:461` 与 `eval_peak_ridge_amount_ratio.py:513` 都做 +`dict(ret_risk.get("aligned_spread_by_cost", {}) or {})`，但 `analytics/` 全仓**不存在**该键 +（只有 `aligned_spread_annual_return|sharpe|volatility|max_drawdown|sortino|final_nav` +与 `net_long_short_by_cost`）。 + +**本 agent 独立复核**：直接读**冻结的 exec artifact** 的 `return_risk` payload， +两个因子的 `aligned_spread_by_cost` 均 **不存在**；实际 `aligned_*` 键恰为上述六个。 +→ 两个 runner 报的**永远是 `{}`**。 + +**唯一"守卫"是一个不可能失败的测试**： +`tests/test_eval_valley_price_quantile_runner.py:485` +`assert isinstance(m["aligned_spread_by_cost"], dict)` —— `{}` 是 dict，**无条件通过**。 +正是项目方法论 ① 点名的形态。 + +### BUG 3（HIGH）—— `qt/eval_peak_ridge_amount_ratio.py` **完全没有 runner 测试** + +11 个里 10 个有 `tests/test_eval__runner.py`；PR-M 没有 +（`grep -rn "qt.eval_peak_ridge_amount_ratio" tests/` 无命中）。 +后果：`PeakCoverage` + `summarize_peak_coverage`（`:122-243`，**122 行披露逻辑**） +**零测试覆盖**，而三个兄弟 summarizer 都有反事实 + 空帧测试。 +`tests/test_peak_ridge_amount_ratio_factor.py` 只覆盖因子数学。 + +### BUG 4（MEDIUM）—— PR-M 的 run log 比终端输出更贫 + +它提取全 29 键、CLI 也打印 net-spread/pearson/monotonicity（`qt/cli.py:486-489`）， +但不像 PR-K（`:708`）/ PR-L（`:654,:660`）那样发对应 `logger.info`。 + +### BUG 5（LOW）—— config 声明了一份 runner 根本不读的因子簿 + +每份 config 都有 `factors:[value_ep,value_bp,volatility_20]`，与硬编码 +`_build_book_factors()` 相同，但**无 runner 读 `cfg.factors`**。今天两者一致故无害， +但**改 config 会静默无效**。同形：`backtest.rebalance='monthly'` 而 +`_build_eval_config` 硬编码 `rebalance='daily'`；整个 `backtest:/portfolio:/alpha:` 块对这些 run 惰性。 + +### BUG 6（LOW / 近失）—— 重名测试函数，**目前**两份都被收集 + +`test_summarize_ridge_coverage_handles_no_frames` 同时定义于 +`tests/test_eval_ridge_minute_return_runner.py:246` 与 +`tests/test_eval_valley_ridge_vwap_ratio_runner.py:232`。 +**因为 `tests/__init__.py` 存在**，两份都被收集（实测该两文件收 10+7=17）。 +→ ⚠️ **对 C6 的直接警告**：若统一 runner 的测试整合过程中删掉 `tests/__init__.py`， +这条会立刻变成静默丢测试。 + +### BUG 7（LOW）—— `eval_ridge_minute_return.py:471` 自称"Beyond the fields PR-C..PR-J surfaced"， +但 PR-J（valley_ridge_vwap_ratio）属 19 键基础档，它点名的对照集在自己的边界上就是错的。 + +## 七、与 C5 对账的接口 + +**本表 §二/§三 = 允许出现的差异白名单；§四 = 必须逐值一致的项；§五 = 已判定归一（对账中 +不应产生数值差异）。** + +另有一条**已知且必然**的 artifact 差异，来自 D1/D3 而非 D5，**必须计入白名单** +（本 agent 独立复核确认）： + +> **`spec` 块键数 16 → 20**。冻结 artifact 的 `spec` 有 16 键，当前代码产出 20 键， +> 新增恰为 `requires` / `adjustment` / `overnight_boundary` / `lookback_depth`，**零删除**。 +> 成因：`render.py:156` 的 `sanitize_payload(vars(report.spec))` 会把 FactorSpec 的**任何** +> 新字段自动带进 JSON，而这四个字段是 PR #86（契约 v1.0）与 PR #91（v1.1 `lookback_depth`） +> 加的。`requires` 经 `clean_value` 的 `str()` 兜底渲染成 **repr 字符串列表**。 +> **属预期漂移，非回归**——但若不预先登记，会在对账时被当成 D5 引入的差异去追。 From d42602eb37e25a77ba983576114a01782811f034 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Sat, 25 Jul 2026 11:11:19 -0700 Subject: [PATCH 4/5] fix(qt): stop the freeze command from silently restamping its own provenance Review BLOCK, one HIGH and two MEDIUM, plus five smaller corrections. HIGH -- freeze() rewrote the manifest unconditionally, regardless of whether it had copied anything, and source_note came from a CLI argument that defaulted to empty and was never inherited. So running the command this repo's own documentation tells you to run, on a completely unchanged tree, printed `copied: 0 / already ok: 77` while the git diff showed frozen_at_utc and frozen_at_git_head repointed at whoever re-ran it and the entire provenance narrative blanked. What broke was the one thing this PR exists to protect, the documentation actively steered people into it, and all sixteen tests passed through the destruction. The restamped frozen_at_git_head was also an active false claim -- it would assert the baseline was frozen at the re-runner's HEAD, the exact misreading the doc explains the field name was chosen to avoid. Provenance is now inherited from an existing manifest unless explicitly overridden, and an unchanged tree writes nothing at all, so the freeze is byte-idempotent and the doc says "byte-idempotent" rather than "idempotent". Inheritance only takes what is actually present: a manifest that never carried provenance gets fresh values rather than an inherited empty string, which would trade a restamp for a silent blank. The always-run manifest test now asserts source_note is non-empty, frozen_at_git_head is 40 hex, every sha256 is valid hex rather than merely 64 characters, and every size is positive. MEDIUM -- test_frozen_tree_is_not_a_git_tracked_path claimed the bytes stay untracked while the hashes are committed, but never consulted git: it froze a fake tree under tmp_path, deleted it, and asserted a file it had just written was readable. It now queries the git index. The index rather than the filesystem because artifacts/ is routinely a symlink and every filesystem-walking git command refuses such paths with "beyond a symbolic link" -- which is also why the first version of this mutation proved nothing. MEDIUM -- the catalogue only recorded qt/cli.py as importing the eleven runners. Three more do, and all three are D1/D2 acceptance tools, including qt/panel_freeze.py, which produced the very baseline D5's panel leg compares against. Deleting the runners in C6 would take the ability to regenerate or re-verify that baseline with them -- the same asymmetric loss that justifies C1. Recorded with a required disposition, alongside the note that d1_panel_freeze_manifest.md's "rerun command" section goes stale either way. Smaller: the close-basis family is 66 files, not 44 -- the miscount dropped the dashboards, in a document that two paragraphs earlier explains dashboards were found and frozen on the exec side. The exec mtime spread is 9.46 ms, not ~2 ms, and the "11 progressive values" is a JSON-only count (27 across all 66), so the caliber is now stated. The probe printed max|diff|=0.000e+00 next to NOT SAFE where the two series share no comparable cell at all; it prints "n/a (no comparable cell)" with an explicit compared-count, which also shows the SAFE rows agree over 728 and 1524 real comparisons rather than vacuously. Both docs now state the gitignored-artifacts prerequisite and the nested-symlink trap. Mutation evidence, re-run against the final code: blanking the manifest provenance fails the always-run test (rc=1) where all sixteen previously passed; injecting a frozen path into the git index moves tracked-under-artifacts 0->1->0 and fails the tracking test (rc=1); running freeze on an unchanged tree reports "manifest: unchanged" and leaves zero porcelain lines. The second of those mutations was itself empty on the first attempt -- git add -f refused to traverse the symlink, so nothing changed and the test "passed" for the wrong reason. That is the second vacuous mutation in this work, both caught by an observation contradicting the expectation rather than by the test. Both are recorded in the freeze document. --- docs/factors/d5_exec_baseline_freeze.md | 49 ++++++-- .../factors/d5_runner_difference_catalogue.md | 26 ++++ docs/factors/d5_saturation_feasibility.md | 23 +++- qt/exec_baseline_freeze.py | 61 ++++++++-- qt/saturation_probe.py | 15 ++- tests/test_exec_baseline_freeze.py | 115 ++++++++++++++++-- 6 files changed, 256 insertions(+), 33 deletions(-) diff --git a/docs/factors/d5_exec_baseline_freeze.md b/docs/factors/d5_exec_baseline_freeze.md index f6ff765..dfa3cff 100644 --- a/docs/factors/d5_exec_baseline_freeze.md +++ b/docs/factors/d5_exec_baseline_freeze.md @@ -5,6 +5,12 @@ > 本文是 provenance 叙述；**机器可读权威 manifest 是入 git 的 > `docs/factors/d5_exec_baseline_manifest.json`**（77 条 sha256）。 > 冻结的字节在 `artifacts/refactor_baseline/exec_baseline/`（gitignored，不入 git）。 +> +> ⚠️ **前置条件**：本文所有命令都需要 gitignored 的 `artifacts/` 树。**在没有它的 worktree 里 +> 开箱跑不起来**——需先对主 checkout 的 `artifacts` 建符号链接，或用 `--repo-root` 指向 +> 一个有该树的 checkout（先例：`qt/factor_hotpath_smoke.py` docstring 的同类说明）。 +> 建符号链接时注意：若 worktree 里已存在真实 `artifacts/` 目录，`ln -s` 会嵌套成 +> `artifacts/artifacts`（本步实际踩到过）。 --- @@ -40,20 +46,25 @@ CLAUDE.md 记录在案的 `compare_postmerge.py` **空对账**失效模式（拿 > 清点目录时发现的同族产物。它们同样单份、同样不可再生，故一并冻结——**冻结的成本是一次 > 文件拷贝，漏冻的成本是永久丢失**。 -**未冻结**：close 基准的 44 个 artifact。它们不是 D5 的对账参照（统一 runner 是 exec-only， -决策 4），且未被新 runner 的输出路径覆盖，原地不动。 +**未冻结**：close 基准的 **66** 个 artifact（22 json + 22 md + **22 png**）。它们不是 D5 的对账 +参照（统一 runner 是 exec-only，决策 4），且未被新 runner 的输出路径覆盖，原地不动。 +（本数曾误写 44——**漏的正是 dashboard**，而上一段刚说明 exec 侧发现 dashboard 才一并冻结， +close 侧没回头改数。同一次清点里，同一类东西数了两遍、两次结果不同。） ## 三、Provenance（观察到的事实 vs 归因） **观察到的**（可复验）： -- 77 个 exec 文件的 mtime **全部落在同一瞬间 `2026-07-21 15:33:20`**（跨度 ~2ms）。 -- 对照：44 个 **close 基准** artifact 的 mtime 呈 **11 个递进值** - `09:24:33 → 09:32:45 → … → 10:43:54`——每因子一个，正是一次真实 11 因子 run 的写入签名， - 且与 CLAUDE.md 记录的 #74 合并后全量重跑窗口（09:24:33→10:43:54）**逐值吻合**。 +- 77 个 exec 文件的 mtime **全部落在同一秒 `2026-07-21 15:33:20`**（实测跨度 **9.46 ms**）。 +- 对照：66 个 close 基准 artifact 里，**22 个 JSON 的 mtime 呈 11 个递进秒值** + `09:24:33 → 09:32:45 → … → 10:43:54`——每因子一个（同因子的 no_book/with_book 同秒）， + 正是一次真实 11 因子 run 的写入签名，且与 CLAUDE.md 记录的 #74 合并后全量重跑窗口 + （09:24:33→10:43:54）**逐值吻合**。 + ⚠️ **「11」是按 JSON 计的**；全部 66 个文件（含 md/png）共 **27 个不同秒值**，因为每因子 + 的多个产物写在相邻但不同的秒里。引用本条时必须带口径，否则 11 与 27 会看起来矛盾。 **推论**：exec 家族**不是**在 `artifacts/reports/` 里跑出来的，而是**被整体拷贝进来的** -（一次 1.5h 的 run 不可能让 77 个文件共享毫秒级 mtime）。拷贝时刻在 PR #79 合并 +（一次 1.5h 的 run 不可能让 77 个文件落在同一个 9.46ms 窗口里；对照组恰恰不是这样）。拷贝时刻在 PR #79 合并 （`b04c966`，15:25）之后 8 分钟——与「#79 在独立 worktree 里跑完、合并后把产物拷进主 checkout」 的形态一致。 @@ -86,6 +97,15 @@ lead 的要求是「用路径断言或结构性隔离保证，不靠纪律」。 | 冻结文件末字节 XOR 0x01 | **RAISED** `on-disk sha256 … != manifest …`；`verify_all` **76 ok / 1 problem**；同目录未改的兄弟文件仍正常读出（守卫是定向的，不是一刀切） | | 冻结后改 **live** 源文件再 freeze | **RAISED** `differs from the live artifact`，且冻结副本**逐字节未动** | | 少一个 / 多一个清单成员 | **RAISED**（分别 `absent` / `unexpected`） | +| **清空 manifest 的 `source_note` + `frozen_at_git_head`** | **rc=1 FAILED** `test_committed_manifest_describes_the_real_frozen_baseline`（新增的恒跑 provenance 断言；**修复前 16 个测试对此全绿**） | +| **把一个冻结文件塞进 git 索引**（`update-index --cacheinfo`，绕开 symlink） | **rc=1 FAILED** `test_frozen_bytes_are_untracked_…`；索引内 artifacts 路径 **0→1→0** | +| 在未改动的树上跑 `freeze`（真实执行） | `copied: 0 / already ok: 77 / manifest: **unchanged**`，manifest porcelain **0 行** | + +⚠️ **本步第二次空 mutation（照实记录）**：上表倒数第二行的**第一版**用 `git add -f`， +但 git 对符号链接下的路径报 `beyond a symbolic link` 而**拒绝执行**——mutation 什么也没改， +测试于是"通过"，且是**为错误的理由通过**。改用 `git update-index --cacheinfo` 直接写索引 +（不走文件系统、不受 symlink 影响）才真正把它变红。**同一份工作里两次写出不会失败的 +mutation**，都靠"预期与观察不符"而非靠测试本身抓住——见 §四末与本行。 ⚠️ **记一次本步自己的失误（照实记录，与项目既有先例同格式）**：tamper 测试的第一版把 `b'"ic"'` 替换成 `b'"IC"'`，而该字节串**在文件里根本不存在** → `.replace()` 是 no-op → @@ -101,10 +121,23 @@ lead 的要求是「用路径断言或结构性隔离保证，不靠纪律」。 # 校验冻结树与 manifest 一致（不拷贝任何东西） python -m qt.exec_baseline_freeze --verify -# 再次冻结：幂等；内容不同则拒绝并保留原副本 +# 再次冻结：**字节幂等**——树没变则一个字节都不写（含 manifest）；内容不同则拒绝并保留原副本 python -m qt.exec_baseline_freeze ``` +⚠️ **「字节幂等」这个限定词是修出来的，不是一开始就对**（评审 HIGH，照实记录）：初版 +`freeze()` 在末尾**无条件重写 manifest**，与本轮有没有拷贝文件无关，且 `source_note` 来自 +CLI 参数、默认空、**不从既有 manifest 继承**。于是在一棵完全没变的树上跑上面这条本文档 +**主动教人跑**的命令，会打印 `copied: 0 / already ok: 77`（纯成功），而 git diff 里 +`frozen_at_utc` / `frozen_at_git_head` 已改指向重跑者、**§三整段 provenance 叙述被清空**—— +**坏掉的恰是本 PR 最下功夫保护的东西**，而且 16 个测试在它被清空后全部通过。 + +重写后的 `frozen_at_git_head` 还是一个**主动的假声明**（它会宣称基线冻结于重跑者的 HEAD）， +正是本文 §三末尾专门解释过要避免的那种误读——同 #76/#78/#82 形态。 + +**修法**：provenance 三字段从既有 manifest **继承**（显式 `--source-note` 才覆盖）； +内容不变则**完全不写**。现在 `--verify` 与 `freeze` 都不会改动 provenance。 + 冻结/校验工具：`qt/exec_baseline_freeze.py`；测试：`tests/test_exec_baseline_freeze.py`。 读取器 `FrozenExecBaseline` 是 D5 对账**唯一**支持的入口——对账代码不得自己 `open()` `artifacts/reports/` 下的任何文件。 diff --git a/docs/factors/d5_runner_difference_catalogue.md b/docs/factors/d5_runner_difference_catalogue.md index afcb46d..83dcdd9 100644 --- a/docs/factors/d5_runner_difference_catalogue.md +++ b/docs/factors/d5_runner_difference_catalogue.md @@ -225,6 +225,32 @@ aligned_base = sign * gross - base_cost ### BUG 7（LOW）—— `eval_ridge_minute_return.py:471` 自称"Beyond the fields PR-C..PR-J surfaced"， 但 PR-J（valley_ridge_vwap_ratio）属 19 键基础档，它点名的对照集在自己的边界上就是错的。 +## 六之二、C6 爆炸半径：谁 import 这 11 个 runner（**删之前必须处置**） + +严格说这不是「runner 之间的差异」，但删除动作的影响面必须落在纸面上——与 BUG 6 +（`tests/__init__.py` 收集陷阱）同一接口。**非测试模块共四个**（实测 +`grep -rln "qt\.eval_" --include=*.py | grep -v ^./tests/`）： + +| 模块 | 用途 | C6 处置 | +|---|---|---| +| `qt/cli.py` | 11 个 `_cmd_run_eval_*` 子命令（§五 C13） | 随统一 runner 收敛为一个子命令 | +| **`qt/panel_freeze.py`** | **生产 D1 冻结基线的工具**——调各 runner 私有 `_load_*_panel` + `_build_book_factors`，「零公式重抄」正是靠 import 它们实现的 | **删 runner 会打断「重新生成 / 重新验证 D1 基线」的能力** | +| **`qt/panel_reconcile.py`** | D2 逐格对账工具（同上依赖） | 同上 | +| **`qt/hand_anchors_engine_values.py`** | D2 手算锚的引擎侧取值 | 同上 | + +⚠️ **后三个全是 D1/D2 的验收工具**，而 **D5 面板腿的比较对象正是 `panel_freeze.py` 的产物**。 +按设计 v3.2 §五第 4 腿的 **provenance 规则**，基线只许从钉住的 pre-D2 SHA 重新生成—— +所以严格说 C6 之后从**当前树**重跑它们本来就不合法。但「工具还在、只是不该从这里跑」与 +「工具已被删、想复核也无从下手」是两回事：**这与 C1 存在的理由是同一种不对称损失** +（复核能力一旦丢失，代价远大于保留一份不再调用的代码）。 + +**处置要求（C6 执行时逐条落实，不得默认删掉了事）**：① 三个工具**要么保留**（连同它们 +import 的 runner loader，即便统一 runner 已上线）、**要么显式记录**「D1/D2 基线自此不可从 +本仓再生，只能依赖已冻结的 artifact + manifest 哈希」并让 lead 知情裁定； +② 无论哪种，`docs/factors/d1_panel_freeze_manifest.md` 的「重跑命令」一节都会变成陈旧描述， +**必须同批更新**——否则就是本项目 #76/#78/#82 那条形态的又一次复发（文档教人跑一条已经 +跑不了的命令）。 + ## 七、与 C5 对账的接口 **本表 §二/§三 = 允许出现的差异白名单；§四 = 必须逐值一致的项；§五 = 已判定归一（对账中 diff --git a/docs/factors/d5_saturation_feasibility.md b/docs/factors/d5_saturation_feasibility.md index 3865271..12ee7a1 100644 --- a/docs/factors/d5_saturation_feasibility.md +++ b/docs/factors/d5_saturation_feasibility.md @@ -3,6 +3,11 @@ > **状态**：D5 第二个交付物，**先于任何长 run**（lead 指令：「宁可报『这条路走不通、原因如下』， > 也不要跑一个 2 小时后失败或产出错值的 run」）。 > 复验：`python -m qt.saturation_probe`（cache-only，实测 `live_calls=0`）。 +> ⚠️ **前置条件**：需要 gitignored 的 `artifacts/`（分钟缓存 + D1 冻结面板）。**在没有它的 +> worktree 里开箱跑不起来**——先对主 checkout 的 `artifacts` 建符号链接，或用 +> `--cache-root` / `--universe-panel` 指向绝对路径（先例：`qt/factor_hotpath_smoke.py`）。 +> 建链接时注意：worktree 里若已存在真实 `artifacts/` 目录，`ln -s` 会嵌套成 +> `artifacts/artifacts`（本步实际踩到过）。 > 全部数字为**实测**，标注为外推的才是外推。 --- @@ -70,13 +75,19 @@ universe 里有一只在窗口内上市的票，pooled 加载就一定打到 flo **实测的决定性证据**（12 票真实数据，整 universe 算 vs 逐 symbol 算再拼）： -| 因子 | index 相同 | max\|diff\| | NaN 集合差异 | 判定 | -|---|---|---|---|---| -| `ridge_minute_return_20`（pooled） | True | **0.000e+00** | **0** | **SAFE** | -| `volume_peak_count_20`（pooled） | True | **0.000e+00** | **0** | **SAFE** | -| `intraday_amp_cut_10` | True | 0.000e+00 | **1692（全部行）** | **NOT SAFE** | +| 因子 | index 相同 | 可比 cell 数 | max\|diff\| | NaN 集合差异 | 判定 | +|---|---|---|---|---|---| +| `ridge_minute_return_20`（pooled） | True | **728** | **0.000e+00** | **0** | **SAFE** | +| `volume_peak_count_20`（pooled） | True | **1524** | **0.000e+00** | **0** | **SAFE** | +| `intraday_amp_cut_10` | True | **0** | **n/a（无可比 cell）** | **1692（全部行）** | **NOT SAFE** | -前两个**逐位相同**——per-symbol 纯性在真实数据上得到确认（不是靠既有 isolation 测试推断）。 +前两个在 728 / 1524 个**真实可比 cell** 上逐位相同——per-symbol 纯性在真实数据上得到确认 +（不是靠既有 isolation 测试推断，也不是在空集合上真空通过）。 + +⚠️ 第三行的 `max|diff|` **必须显示 `n/a` 而不是 `0.000e+00`**（评审 NIT，已改）：per-symbol +一侧全是 NaN，两边**没有任何一个 cell 可比**，而"0.000e+00"紧挨着 "NOT SAFE" 会被读成 +「值其实一致、只是 NaN 标记不同」——恰好相反。`compared=0` 这一列就是为了让"没得比" +无法被误读成"比过了且相等"。 `intraday_amp_cut` **全部 1692 行翻成 NaN**：它的第 4 步是**按日截面 z-score**，要求当日 finite 对数 ≥ `AMP_CUT_MIN_CROSS_SECTION=10`，而**单票截面 n=1 恒 < 10 → 定义上全 NaN**。 diff --git a/qt/exec_baseline_freeze.py b/qt/exec_baseline_freeze.py index c9c80a6..de8ee01 100644 --- a/qt/exec_baseline_freeze.py +++ b/qt/exec_baseline_freeze.py @@ -31,9 +31,16 @@ overwrites the frozen tree cannot also silently move the hashes, because the hashes are under version control and show up in ``git diff``. -Usage:: +Both entry points are BYTE-idempotent on an unchanged tree: re-running them +writes nothing, including the manifest, so ``git diff`` stays clean. Provenance +fields (``frozen_at_utc`` / ``frozen_at_git_head`` / ``source_note``) are +inherited from an existing manifest rather than restamped — a verification run +must not be able to re-date the history it is verifying. - python -m qt.exec_baseline_freeze # freeze (idempotent, refuses to clobber) +Usage (needs the gitignored ``artifacts/`` tree; in a worktree without one, pass +``--repo-root`` pointing at a checkout that has it):: + + python -m qt.exec_baseline_freeze # freeze; refuses to clobber python -m qt.exec_baseline_freeze --verify # re-verify the frozen tree, copy nothing """ @@ -163,6 +170,17 @@ class FreezeResult: already_frozen: tuple[str, ...] manifest_path: Path frozen_root: Path + #: False when an existing manifest was already correct and was left alone. + manifest_written: bool = True + + +def _load_manifest_if_present(manifest_path: Path) -> dict | None: + if not manifest_path.exists(): + return None + try: + return json.loads(manifest_path.read_text()) + except json.JSONDecodeError: + return None def freeze( @@ -238,10 +256,30 @@ def freeze( } ) + # PROVENANCE IS INHERITED, NEVER SILENTLY RESTAMPED. + # + # An earlier version rebuilt these three fields on every invocation, so the + # documented "re-run me to check the freeze" command quietly repointed + # frozen_at_git_head at whoever re-ran it and blanked source_note — while + # reporting `copied: 0` and passing every test. That turned the freeze tool + # into a generator of the exact false provenance claim it exists to prevent + # (the #76/#78/#82 shape: the behaviour changed, the wording did not). + # + # So: an existing manifest supplies these unless the caller explicitly + # overrides them. Re-freezing verifies bytes; it does not re-date history. + # Inherit only what is actually there: a manifest that never carried + # provenance has nothing to preserve, and inheriting "" from it would put an + # empty field where the always-run manifest assertions expect a real one — + # trading a restamp for a silent blank. + previous = _load_manifest_if_present(manifest_path) or {} + frozen_at_utc = str(previous.get("frozen_at_utc") or datetime.now(timezone.utc).isoformat()) + frozen_at_head = str(previous.get("frozen_at_git_head") or _git_head(repo_root)) + note = source_note or str(previous.get("source_note") or "") + manifest = { "schema": MANIFEST_SCHEMA, - "frozen_at_utc": datetime.now(timezone.utc).isoformat(), - "frozen_at_git_head": _git_head(repo_root), + "frozen_at_utc": frozen_at_utc, + "frozen_at_git_head": frozen_at_head, # Logical, repo-relative locations. The on-disk paths resolve through a # gitignored symlink on some checkouts; recording the resolved absolute # path would make this git-tracked manifest machine-specific. @@ -250,17 +288,23 @@ def freeze( "file_count": len(entries), "factors": list(FACTORS), "books": list(BOOKS), - "source_note": source_note, + "source_note": note, "files": entries, } - _atomic_write_bytes( - manifest_path, (json.dumps(manifest, indent=2, sort_keys=False) + "\n").encode() - ) + # BYTE-idempotent: an unchanged tree rewrites nothing at all, so re-running + # the documented verification command leaves a clean `git diff`. + manifest_written = manifest != previous + if manifest_written: + _atomic_write_bytes( + manifest_path, + (json.dumps(manifest, indent=2, sort_keys=False) + "\n").encode(), + ) return FreezeResult( copied=tuple(copied), already_frozen=tuple(already), manifest_path=manifest_path, frozen_root=frozen_root, + manifest_written=manifest_written, ) @@ -387,6 +431,7 @@ def main(argv: list[str] | None = None) -> int: print(f"manifest : {result.manifest_path}") print(f"copied : {len(result.copied)}") print(f"already ok : {len(result.already_frozen)}") + print(f"manifest : {'REWRITTEN' if result.manifest_written else 'unchanged'}") return 0 diff --git a/qt/saturation_probe.py b/qt/saturation_probe.py index 31b1f64..9034102 100644 --- a/qt/saturation_probe.py +++ b/qt/saturation_probe.py @@ -129,13 +129,19 @@ def per_symbol_equivalence( if whole.index.equals(per.index): both = whole.notna() & per.notna() entry["index_same"] = True + # None, not 0.0, when the two series share NO finite cell: printing a + # zero difference next to "NOT SAFE" reads as "the values agree", + # when in fact there is nothing to compare (that is exactly the + # intraday_amp_cut case, where every per-symbol value is NaN). entry["max_abs_diff"] = ( - float((whole[both] - per[both]).abs().max()) if both.any() else 0.0 + float((whole[both] - per[both]).abs().max()) if both.any() else None ) + entry["compared_cells"] = int(both.sum()) entry["nan_set_diff"] = int((whole.isna() != per.isna()).sum()) else: entry["index_same"] = False - entry["max_abs_diff"] = float("nan") + entry["max_abs_diff"] = None + entry["compared_cells"] = 0 entry["nan_set_diff"] = -1 entry["per_symbol_safe"] = bool( entry["index_same"] @@ -201,9 +207,12 @@ def main(argv: list[str] | None = None) -> int: ) for r in rows: verdict = "SAFE" if r["per_symbol_safe"] else "NOT SAFE" + diff = r["max_abs_diff"] + shown = f"{diff:.3e}" if diff is not None else "n/a (no comparable cell)" print( f"{r['factor']:26s} rows={r['rows']:>6,} index_same={r['index_same']!s:5s} " - f"max|diff|={r['max_abs_diff']:.3e} nan_set_diff={r['nan_set_diff']:>5d} {verdict}" + f"max|diff|={shown:>22s} compared={r['compared_cells']:>5d} " + f"nan_set_diff={r['nan_set_diff']:>5d} {verdict}" ) print( "\nintraday_amp_cut is expected NOT SAFE at whole-factor granularity: its " diff --git a/tests/test_exec_baseline_freeze.py b/tests/test_exec_baseline_freeze.py index 3b60a6f..1c30509 100644 --- a/tests/test_exec_baseline_freeze.py +++ b/tests/test_exec_baseline_freeze.py @@ -20,7 +20,9 @@ import hashlib import json +import re import shutil +import subprocess from pathlib import Path import pytest @@ -68,7 +70,7 @@ def test_exec_baseline_inventory_is_77_derived_from_the_factor_list() -> None: assert sum(n.endswith(".json") for n in names) == 22 assert sum(n.endswith("_dashboard.png") for n in names) == 22 assert sum(n.endswith("_basis_sanity.md") for n in names) == 11 - # 22 report .md = 44 total .md minus the 11 sanity files... 22 + 11 = 33 + # 22 per-book report .md + 11 basis-sanity .md = 33 .md in total assert sum(n.endswith(".md") for n in names) == 33 @@ -193,17 +195,78 @@ def test_exec_baseline_reader_rejects_unknown_names(tmp_path: Path) -> None: def test_committed_manifest_describes_the_real_frozen_baseline() -> None: - """The git-tracked manifest is well-formed and complete.""" + """The git-tracked manifest is well-formed, complete, and carries provenance. + + Deliberately NOT skipif-guarded: it reads only the committed manifest, so it + runs on every machine. That matters because the provenance assertions below + are the ones that catch a manifest whose history was silently restamped — + a check that skips on the hosts without the gitignored bytes would be a + check that never runs where it is needed. + """ payload = json.loads(MANIFEST.read_text()) assert payload["schema"] == MANIFEST_SCHEMA assert payload["file_count"] == EXPECTED_FILE_COUNT assert len(payload["files"]) == EXPECTED_FILE_COUNT assert {f["name"] for f in payload["files"]} == set(expected_artifact_names()) - assert all(len(f["sha256"]) == 64 for f in payload["files"]) # repo-relative, machine-independent assert not payload["source_dir"].startswith("/") assert not payload["frozen_root"].startswith("/") + # --- provenance must be present and well-formed, not merely a field --- + assert payload["source_note"].strip(), ( + "source_note is empty: the manifest no longer records WHERE the frozen " + "bytes came from. A re-freeze must inherit provenance, never blank it." + ) + assert re.fullmatch(r"[0-9a-f]{40}", payload["frozen_at_git_head"]), ( + f"frozen_at_git_head {payload['frozen_at_git_head']!r} is not a full " + "40-hex commit id" + ) + assert payload["frozen_at_utc"].startswith("20") + + for entry in payload["files"]: + assert re.fullmatch(r"[0-9a-f]{64}", entry["sha256"]), ( + f"{entry['name']}: sha256 is not 64 hex chars" + ) + assert int(entry["size_bytes"]) > 0, f"{entry['name']}: non-positive size" + + +def test_freeze_inherits_provenance_instead_of_restamping_it(tmp_path: Path) -> None: + """MUTATION: re-freeze an UNCHANGED tree. Provenance must survive verbatim. + + The regression this locks: freeze() used to rebuild frozen_at_utc / + frozen_at_git_head / source_note on every call, so the documented + verification command silently repointed the manifest at whoever re-ran it + and blanked the provenance narrative, while printing `copied: 0`. + """ + reports, frozen, manifest = _freeze_fake(tmp_path) + # give it real provenance, as the committed manifest has + original = json.loads(manifest.read_text()) + original["source_note"] = "PR #79 artifacts; bulk-copied 2026-07-21 15:33:20" + original["frozen_at_git_head"] = "a" * 40 + original["frozen_at_utc"] = "2026-07-25T00:00:00+00:00" + manifest.write_text(json.dumps(original, indent=2) + "\n") + before = manifest.read_bytes() + + result = freeze(reports, frozen, manifest, repo_root=tmp_path) + + assert result.copied == () + assert result.manifest_written is False, "an unchanged tree must not rewrite" + assert manifest.read_bytes() == before, "manifest bytes must be untouched" + after = json.loads(manifest.read_text()) + assert after["source_note"] == original["source_note"] + assert after["frozen_at_git_head"] == "a" * 40 + assert after["frozen_at_utc"] == "2026-07-25T00:00:00+00:00" + + +def test_freeze_lets_an_explicit_source_note_win(tmp_path: Path) -> None: + """Inheritance is a default, not a lock: an explicit note is still an act.""" + reports, frozen, manifest = _freeze_fake(tmp_path) + result = freeze( + reports, frozen, manifest, repo_root=tmp_path, source_note="corrected note" + ) + assert result.manifest_written is True + assert json.loads(manifest.read_text())["source_note"] == "corrected note" + @pytest.mark.skipif( not (REPO_ROOT / "artifacts/refactor_baseline/exec_baseline").is_dir(), @@ -228,13 +291,49 @@ def test_sha256_file_matches_hashlib(tmp_path: Path) -> None: assert sha256_file(path) == hashlib.sha256(payload).hexdigest() -def test_frozen_tree_is_not_a_git_tracked_path(tmp_path: Path) -> None: - """The bytes stay under gitignored artifacts/; only hashes are committed. +def _tracked_under(prefix: str) -> list[str]: + """Paths git TRACKS under ``prefix``, read from the index. - That split is what gives the tamper check teeth: hashes move only through - a reviewable ``git diff``. + Queries the index rather than the filesystem on purpose. ``artifacts/`` is + routinely a symlink into another checkout, and every filesystem-walking git + command (``add``, ``check-ignore``, ``ls-files --error-unmatch`` on a path) + refuses such paths with "beyond a symbolic link" — which would make an + assertion about them pass for the wrong reason, and make it impossible to + mutate. The index has no such blind spot. """ + proc = subprocess.run( + ["git", "-C", str(REPO_ROOT), "ls-files", "--", prefix], + capture_output=True, + text=True, + check=True, + ) + return proc.stdout.split() + + +def test_frozen_bytes_are_untracked_while_the_manifest_is_tracked() -> None: + """The split that gives the tamper check its teeth, asserted against git. + + Hashes live in version control so they can only move through a reviewable + diff; the bytes they describe do not. If both lived in git, one commit could + move the bytes and their hashes together and the tamper check would be + decorative. + + The previous version of this test claimed exactly this but never consulted + git at all — it froze a fake tree under tmp_path, deleted it, and asserted a + file it had just written was still readable. Removing ``artifacts/`` from + .gitignore left it green. + """ + assert _tracked_under("artifacts") == [], ( + "no bytes under artifacts/ may be committed; found tracked paths" + ) + manifest_rel = str(MANIFEST.relative_to(REPO_ROOT)) + assert manifest_rel in _tracked_under("docs/factors"), ( + "the manifest MUST be committed — it is the tamper reference" + ) + + +def test_manifest_outlives_the_bytes_it_describes(tmp_path: Path) -> None: + """Losing the frozen tree must not lose the record of what it contained.""" _, frozen, manifest = _freeze_fake(tmp_path) shutil.rmtree(frozen) - # manifest survives independently of the bytes it describes assert json.loads(manifest.read_text())["file_count"] == EXPECTED_FILE_COUNT From 6987952e5e23c733de79e23940b11447bdb0cb23 Mon Sep 17 00:00:00 2001 From: shaofl <2899218482@qq.com> Date: Sat, 25 Jul 2026 11:21:56 -0700 Subject: [PATCH 5/5] docs(qt): tell first-time freezers to pass --source-note (review NIT-A) --- qt/exec_baseline_freeze.py | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/qt/exec_baseline_freeze.py b/qt/exec_baseline_freeze.py index de8ee01..cee3041 100644 --- a/qt/exec_baseline_freeze.py +++ b/qt/exec_baseline_freeze.py @@ -42,6 +42,17 @@ python -m qt.exec_baseline_freeze # freeze; refuses to clobber python -m qt.exec_baseline_freeze --verify # re-verify the frozen tree, copy nothing + +A FIRST freeze (no manifest to inherit from) must pass ``--source-note``: the +committed manifest is asserted to carry a non-empty provenance note, so a +freeze without one is fail-closed at test time rather than at review time:: + + python -m qt.exec_baseline_freeze --source-note "" + +Re-freezing an unchanged tree inherits the existing provenance and writes no +bytes. Deleting the frozen tree and re-freezing from a DIFFERENT live source +would carry the old note onto new bytes — the 77 hash lines in the manifest +diff make that loud, but pass ``--source-note`` when the source really changed. """ from __future__ import annotations