Skip to content

fix(review): coverage evidence tooling for numpy and Rust repositories - #611

Closed
seonghobae wants to merge 7 commits into
mainfrom
claude/fast-mlsirm-pr-review-mt2e1z
Closed

fix(review): coverage evidence tooling for numpy and Rust repositories#611
seonghobae wants to merge 7 commits into
mainfrom
claude/fast-mlsirm-pr-review-mt2e1z

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

Problem

Since 2026-07-14 the offline coverage sandbox has structurally been unable to produce passing evidence for any repository that needs third-party Python packages or a modern Rust toolchain. Every such PR (all 47 open fast-mlsirm PRs, including the #160 model-design PR) is stuck at REQUEST_CHANGES with "coverage-evidence result was failure":

  • Python: the sandbox runs the repository's pytest suite with only the trusted toolchain installed (coverage, interrogate, pytest, pytest-cov, uv). Target dependencies are never installed (scripts/ci/install_python_requirements_for_coverage.py exists but is not wired into any workflow), so every test module fails collection with ModuleNotFoundError: No module named 'numpy' — see run 29875093850 for fast-mlsirm#160.
  • Rust: cargo llvm-cov aborts with failed to find llvm-tools-preview. The image only has the Ubuntu-noble rustc/cargo 1.75, which additionally cannot parse version-4 Cargo.lock files (fast-mlsirm, wardnet, …), and the --network=none sandbox has no way to download crates even if it could build.

Fix (fail-closed offline design unchanged)

  1. numpy in the trusted toolchainnumpy==2.5.1 added to requirements-opencode-review-ci.txt; the hashes file regenerated with uv pip compile --generate-hashes --no-header --no-annotate --python-version 3.12 --python-platform x86_64-manylinux_2_28. Empirically verified against fast-mlsirm main: with numpy alone the suite passes (156 passed, 72 skipped — the Rust-parity tests importorskip cleanly when fast_mlsirm._core is absent).
  2. Pinned rustup toolchain in the image — sha256-pinned rustup-init 1.28.2 provisions Rust 1.94.1 with llvm-tools-preview under /usr/local/rustup, symlinked into /usr/local/bin so it shadows the distribution rustc. No mutable sh.rustup.rs installer is used (the existing contract test forbidding it still passes).
  3. Offline crate cache — the online coverage-source-tree job now runs cargo fetch --locked into the tree-local CARGO_HOME (.opencode-sandbox-home/.cargo) that the sandbox already sets, and the sandbox exports CARGO_NET_OFFLINE=true so cargo resolves strictly from that cache. cargo fetch downloads content-addressed crates and executes no build scripts; a failed prefetch keeps today's behavior (the offline build surfaces the missing dependencies).

Verification

  • coverage run -m pytest tests634 passed, coverage report100% over scripts/ci
  • interrogate100.0%
  • Workflow YAML parses; contract assertions about the sandbox (docker build … --network=default before docker run … --network=none, https://sh.rustup.rs absent, pinned cargo-llvm-cov retained, pytest-cov==7.1.0 hash retained) all hold.

Developer experience: unblocks the fast-mlsirm review/merge queue (and every other numpy/Rust repository) without weakening any gate — evidence is still produced offline, from pinned tooling, under the same sandbox identity.

User experience: no user-facing behavior change; review verdicts for scientific-Python/Rust repositories become meaningful instead of unconditionally failing.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ERVynrYEjd2TwYvusVpA9f


Generated by Claude Code

pytest ran with no target dependencies installed and cargo llvm-cov had
no llvm-tools or crate cache, so coverage evidence always failed for
scientific-Python and Rust repositories (every fast-mlsirm PR has been
stuck at REQUEST_CHANGES since 2026-07-14).

Fixes, keeping the fail-closed offline sandbox design intact:

1. Add numpy==2.5.1 to the trusted hash-pinned coverage toolchain
   (hashes regenerated with uv pip compile --generate-hashes
   --no-header --no-annotate --python-version 3.12
   --python-platform x86_64-manylinux_2_28). Verified against
   fast-mlsirm main: suite passes with numpy alone (156 passed,
   72 skipped; Rust-parity tests skip cleanly).
2. Provision a pinned rustup toolchain (1.94.1 + llvm-tools-preview)
   via the sha256-pinned rustup-init 1.28.2 binary, symlinked into
   /usr/local/bin so it shadows the distribution rustc (1.75), which
   cannot parse version-4 Cargo.lock files.
3. Pre-fetch the locked crate graph in the online coverage-source-tree
   job into the tree-local CARGO_HOME the sandbox already uses
   (cargo fetch --locked executes no build scripts), and export
   CARGO_NET_OFFLINE=true in the sandbox so cargo resolves strictly
   from that cache.

Local gates: 634 contract tests pass, scripts/ci coverage 100%,
interrogate 100%, workflow YAML parses.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ERVynrYEjd2TwYvusVpA9f
Copilot AI review requested due to automatic review settings July 21, 2026 23:16
This was referenced Jul 21, 2026
pip-audit fails on main: pyasn1 0.6.3 in requirements-strix-ci-hashes.txt carries CVE-2026-59886 (univ.Real big-exponent DoS), fixed in 0.6.4. Add the floor to requirements-strix-ci.txt and regenerate the lockfile with the exact header command; only pyasn1 changes.
Copilot AI review requested due to automatic review settings July 21, 2026 23:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Comment thread .github/workflows/opencode-review.yml Outdated
Address review feedback: the crate prefetch now walks every tracked Cargo.toml with a sibling Cargo.lock (root and nested workspaces alike, matching the sandbox's nested-manifest coverage discovery) instead of only the repository root.
Copilot AI review requested due to automatic review settings July 21, 2026 23:34

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (1)

.github/workflows/opencode-review.yml:306

  • git ls-files 'Cargo.toml' '*/Cargo.toml' only matches Cargo.toml at repo root or one directory deep, so nested workspaces like crates/foo/Cargo.toml (and any deeper) won’t have their locked crate graphs prefetched. That can leave the offline sandbox without a complete crate cache even though the comment says nested manifests are covered.
          prefetch_manifests="$(git -C "$COVERAGE_SOURCE_WORKDIR" ls-files 'Cargo.toml' '*/Cargo.toml')"

seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…or re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change in the changelog and produces a fresh head so the
scheduler dispatches a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…resh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…esh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…efresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…ad for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…s; refresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…fresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 22, 2026
…es; refresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 26, 2026
…or re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change in the changelog and produces a fresh head so the
scheduler dispatches a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 26, 2026
…resh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 26, 2026
…esh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 26, 2026
…efresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 26, 2026
…s; refresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 26, 2026
…fresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.

Copy link
Copy Markdown
Contributor Author

Closing as functionally superseded. While this PR waited out the provider-pool outage, equivalent fixes landed on main through the parallel repair effort: target-repository Python requirements are now materialized for the offline coverage sandbox (scripts/ci/materialize_base_python_requirements.py, refined by #644), the pnpm/Node coverage path gained the same offline prefetch treatment (#641#643, #645), and the model pool gained a public free-model fallback (#646). End-to-end proof: fast-mlsirm dependabot PRs #194/#195/#197 were reviewed, approved, and auto-merged on 2026-07-26, which requires both the Python and Rust coverage evidence paths to succeed for that repository.

The diagnosis this PR documented (numpy-less pytest collection failures, missing llvm-tools/v4-lockfile-incapable distro cargo, no offline crate cache) remains accurate as history; if Rust coverage regressions resurface for cargo llvm-cov repositories, the pinned-rustup + cargo fetch --locked prefetch approach here is available to revive from branch claude/fast-mlsirm-pr-review-mt2e1z.


Generated by Claude Code

@seonghobae seonghobae closed this Jul 29, 2026
seonghobae added a commit to ContextualWisdomLab/fast-mlsirm that referenced this pull request Jul 31, 2026
* 🎨 Palette: 정적 리포트에 prefers-reduced-motion 및 print 미디어 쿼리 적용

`.Jules/palette.md`의 접근성 가이드에 따라 `scripts/build_*.py`의 HTML 출력에 `@media (prefers-reduced-motion: reduce)` 및 `@media print` CSS를 추가했습니다.

이를 통해 전정기관 장애가 있는 사용자를 위한 과도한 모션을 비활성화하고, 문서를 인쇄하거나 PDF로 저장할 때 색상이 올바르게 렌더링되도록 개선했습니다.

* docs(changelog): record report media-query accessibility support; refresh head for re-review

The previous CHANGES_REQUESTED verdict on this head was an infrastructure
failure (the central coverage-evidence sandbox could not install numpy —
fixed by ContextualWisdomLab/.github#611), not a code judgment. This commit
documents the change and produces a fresh head so the scheduler dispatches
a new review under the repaired pipeline.

* docs(css): note why reduced-motion durations stay non-zero

---------

Co-authored-by: seonghobae <8172694+seonghobae@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants