[Common] Support scaled & clamped swiglu, srelu for BF16 - #3132

Merged

vthumbe1503 merged 18 commits into

NVIDIA:mainfrom

zhongbozhu:add_support_fused_swiglu

Jul 14, 2026

Commits on Jul 9, 2026

support scaled swiglu, scaled srelu and scaled clamp swiglu
zhongbozhu
committed
ddfe2ca
vectorized loading improvement
zhongbozhu
committed
671cf17
fix bug for backward kernel
zhongbozhu
committed
5f1b8bf
optimize
zhongbozhu
committed
28154b2
fix unit test failure
zhongbozhu
committed
67e61df
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
zhongbozhu
committed
280965c
Update tests/cpp/operator/test_scaled_activation.cu

vthumbe1503
authored and
zhongbozhu
committed
c8c19a9
resolve comments
zhongbozhu
committed
bbd9451

Commits on Jul 10, 2026

refactor, resolve comments
zhongbozhu
committed
33e8c36

Commits on Jul 14, 2026

address review comment
vthumbe1503
committed
842d8b0
adaptive cta to fix slow block reduce for scale grads
vthumbe1503
committed
f451e97
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
8d44626
refactor to have gated and unary activation in activation infra
vthumbe1503
committed
f1951a4
resolve merge conflict
vthumbe1503
committed
92f35ed
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
63dbb78
Merge branch 'main' into add_support_fused_swiglu
vthumbe1503
authored
3729c29
reuse scale grad kernel for non scale grad since it is faster anyway
vthumbe1503
committed
794ee5e
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot]
committed
cb354b4