Skip to content

Bug/numeric parity fixes - #1659

Merged
jlarson4 merged 2 commits into
devfrom
bug/numeric-parity-fixes
Aug 12, 2026
Merged

Bug/numeric parity fixes#1659
jlarson4 merged 2 commits into
devfrom
bug/numeric-parity-fixes

Conversation

@jlarson4

Copy link
Copy Markdown
Collaborator

Description

  • MPT's MptMLP adds the residual inside its forward, so blocks.N.hook_mlp_out exposed resid_post instead of the additive contribution. This is a reoccurrence of the [Bug Report] BLOOM TransformerBridge maps hook_attn_out / hook_mlp_out to residual-added states, breaking HookedTransformer residual identities #1639 BLOOM issue in another arch. The alias now targets the down-projection output (same pattern as the BLOOM Fix BLOOM residual branch hook semantics #1640 fix), and MPT joins the residual-decomposition-identity integration fixture.
  • Fixing MPT exposed general MLPBridge hook-fire bugs, symmetric on both edges: forward unconditionally re-fired out.hook_out with the module's final output (stamping the residual-added state over the projection output on residual-inside MLPs, double-applying interventions on ordinary container MLPs), and pre-fired in.hook_in on a tensor the replaced in projection then hooked again inside the wrapped forward. Both are now single-fire: the out-side re-fire is a fallback gated on whether the projection actually ran (fired flag on LinearBridge), and the in-side pre-fire suppresses the inner duplicate for one call. GatedMLPBridge's processed-weights functional path had the mirror-image gap — it bypasses the wrapped projections and never fired out.hook_out at all; it now fires it with the down-projection output. The fired/suppress flags are a declared class-level protocol on GeneralizedComponent, implemented identically by LinearBridge and Conv1DBridge (GPT-2-family Conv1D projections included), with object.__setattr__ writes to stay off the nn.Module setattr path.
  • MLAAttentionBridge hardcoded qk_head_dim^-0.5, dropping the yarn mscale² factor HF folds into self.scaling — DeepSeek-V3/R1 and GLM4-MoE-Lite configs score ~1.87× too small ([Bug Report] TransformerBridge activations on gpt-oss-20b disagree with the HF model it wraps #1618-class silent divergence). The bridge now scores with the wrapped module's scaling, falling back to softmax_scale for trust_remote_code DeepSeek-V2 modules that use that name for the same quantity.
  • MPTALiBiAttentionBridge hardcoded the same base scale, ignoring the module's resolved softmax_scale, and gated clip_qkv on is not None where HF gates on truthiness — a checkpoint spelling "disabled" as 0.0 clamped Q/K/V to zeros. Both now match HF; the Q/K/V clamp itself is extracted to a shared clamp_qkv utility used by both bridges, and the reconstructed path refuses (loudly) the untested clip-plus-post-reshape-norm combination no arch currently has.
  • Falcon-H1 multiplies K by a learned key_multiplier inside its attention forward; the reconstructed path dropped it. PositionEmbeddingsAttentionBridge now applies it post-projection/pre-RoPE, matching HF order.
  • OLMo v1 / OLMoE clip_qkv was never applied under hooks: the reconstruction had no clamp, and the old OLMo prepare_model patch additionally disabled the native module's clamp — so nothing clamped anywhere. The reconstruction now clamps Q/K/V out-of-place, after the pre-reshape qk-norm (OLMoE norms first, then clamps) and before RoPE; the patch is deleted from both adapters.
  • HookedTransformer mirror: new HookedTransformerConfig.clip_qkv field, plumbed from the HF config in the OLMo-1B/7B and OLMoE branches, applied in abstract_attention between qk-norm and rotary/cache-append. OLMo checkpoints are attention-bias-free, so weight processing does not shift the clamp threshold.
  • Tests, every one verified to fail on the unfixed tree: bridge-vs-HF parity against live OlmoAttention, OlmoeAttention (pins the norm-before-clamp order), FalconH1Attention, and DeepseekV3Attention modules — all eager with an explicit 4D causal mask, so parity is tight enough (~1e-5) that each restored term breaks it when reverted; a backward test that registers real full backward hooks on the projections, which an in-place clamp_ fails (the property the deleted patch protected); an HT test proving the clamp sits between projection and scores; and MLP-bridge hook-semantics tests asserting in.hook_in/out.hook_out fire exactly once with the projection's own tensor, that additive interventions apply exactly once, that residual-inside MLPs keep the projection output on out.hook_out, and that the gated processed path fires it. Each parity test also carries a negative control proving the guarded term is active. All test setup runs under torch.random.fork_rng, leaving global RNG state untouched.

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Checklist:

  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have not rewritten tests relating to key interfaces which would affect backward compatibility

@jlarson4
jlarson4 merged commit c7ff4cd into dev Aug 12, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant