Skip to content

[SGLang-Diffusion] Add offline throughput benchmark script for multi-modal models #18154

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
BBuf merged 6 commits into from
Mar 4, 2026
+795 −350
Merged

[SGLang-Diffusion] Add offline throughput benchmark script for multi-modal models #18154

Changes from 2 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
50c4d0f
Add offline throughput benchmark script for multi-modal models
Feb 2, 2026
1687b2b
[SGLang-Diffusion] Fix fake impl eps default for torch.compile tracing
zhaochenyang20 Mar 3, 2026
da59f86
add Task name for benchmark identification
zhaochenyang20 Mar 3, 2026
50f682a
Merge branch 'fix/fake-impl-eps-default-for-torch-compile' into offli…
zhaochenyang20 Mar 3, 2026
8effef7
self fixing comments
zhaochenyang20 Mar 3, 2026
a4b3d41
Merge branch 'main' into offline_bench
BBuf Mar 3, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions python/sglang/jit_kernel/diffusion/cutedsl/scale_residual_norm_scale_shift.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -341,7 +341,7 @@ def fused_norm_scale_shift(


@fused_norm_scale_shift.register_fake
def _fused_norm_scale_shift_fake(x, weight, bias, scale, shift, norm_type, eps):
def _fused_norm_scale_shift_fake(x, weight, bias, scale, shift, norm_type, eps=1e-5):
y = x.new_empty(x.shape)
return y

Expand Down Expand Up @@ -424,7 +424,7 @@ def fused_scale_residual_norm_scale_shift(

@fused_scale_residual_norm_scale_shift.register_fake
def _fused_scale_residual_norm_scale_shift_fake(
residual, x, gate, weight, bias, scale, shift, norm_type, eps
residual, x, gate, weight, bias, scale, shift, norm_type, eps=1e-5
):
y = x.new_empty(x.shape)
residual_out = x.new_empty(x.shape)
Expand Down
Loading