Skip to content

Stabilize Bridge numerical CI tests - #1654

Merged
jlarson4 merged 1 commit into
TransformerLensOrg:dev-4.xfrom
emerardd:fix/bridge-ci-numerical-stability
Aug 12, 2026
Merged

Stabilize Bridge numerical CI tests#1654
jlarson4 merged 1 commit into
TransformerLensOrg:dev-4.xfrom
emerardd:fix/bridge-ci-numerical-stability

Conversation

@emerardd

Copy link
Copy Markdown
Contributor

Description

Stabilize two TransformerBridge integration assertions that cross the numerical envelope of different GitHub-hosted CPU runners. This is independent of the Qwen gated-query change in #1653; that PR exposed the existing baseline failures through its full-coverage run.

The left-padding neighbour test previously compared a three-row batch against separate one-row forwards. Those different matrix shapes can select different CPU/BLAS kernels and produced a 9.1552734375e-05 maximum absolute difference despite equivalent position handling. The test now compares two equal-shaped batches that differ only in whether the middle row requires derived positions, isolating the behaviour the regression test is intended to protect.

The optimizer parity test used a step-1 post-update mean-logit threshold of 1e-3. Multiple GitHub CPU runners reproducibly report 0.0010319872526451945, so this one empirical threshold is recalibrated to 2e-3; the initial-forward, maximum-logit, loss, parameter-update, and step-10 thresholds remain unchanged.

The exact pair of failures appeared in these unrelated dev-4.x runs:

The current #1653 failure is:

The same base SHA also completed successfully in:

No production code or dependencies are changed.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Screenshots

Not applicable.

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

The documentation change is limited to the affected test docstring and explanatory comment; no user-facing API documentation changes are needed. The full unit suite was not rerun because this PR changes only two integration tests.

Validation

  • Exact two previously failing tests: 2 passed
  • Both complete integration test files with CI-style -n 2 --dist loadscope, OMP_NUM_THREADS=1, and MKL_NUM_THREADS=1: 31 passed
  • Full-repository pycln, isort, and Black checks pass
  • uv run mypy .: Success: no issues found in 431 source files
  • git diff --check passes

@emerardd
emerardd marked this pull request as ready for review August 12, 2026 09:28
@jlarson4

Copy link
Copy Markdown
Collaborator

Looks great @emerardd, thanks for cleaning this up!

@jlarson4
jlarson4 merged commit b67e6cd into TransformerLensOrg:dev-4.x Aug 12, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants