[None][test] Add e2e example tests for flux1/2, ltx2, wan_i2v, and cosmos3 - #15126
Conversation
…smos3 The five newly added example scripts (flux1.py, flux2.py, ltx2.py, wan_i2v.py, cosmos3_ti2v.py) under examples/visual_gen/models/ had no integration test coverage. Add one smoke test per script that: - loads the local checkpoint via LLM_MODELS_ROOT (skips if absent), - passes the shared per-model YAML from examples/visual_gen/configs/, - runs the script end-to-end, - asserts the expected output file is produced. Register all five new tests in l0_b200.yml so they run in the B200 post-merge CI stage alongside the existing test_wan_t2v_example. Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
📝 WalkthroughWalkthroughThis PR expands VisualGen integration test coverage by adding a model checkpoint constant and five end-to-end example tests (Flux1, Flux2, LTX-2, Wan I2V, Cosmos3), then registering those tests in the pre-merge CI configuration. ChangesVisualGen Example Tests
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Possibly related PRs
Suggested labels
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/integration/defs/examples/test_visual_gen.py (1)
1423-1429: 💤 Low valueConsider using
_lpips_model_path()helper for consistency.While the current checkpoint resolution pattern (direct
os.path.join(conftest.llm_models_root(), ...)) matches the existingtest_wan_t2v_example, the other four new tests in this PR use the_lpips_model_path()helper. For consistency within the new code, consider:- scratch_space = conftest.llm_models_root() - model_path = os.path.join(scratch_space, WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH) - if not os.path.isdir(model_path): - pytest.skip( - f"Model not found: {model_path} " - f"(set LLM_MODELS_ROOT or place {WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH} under models root)" - ) + model_path = _lpips_model_path(WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH) + _skip_if_missing(model_path, "Wan 2.2 I2V checkpoint", is_dir=True)This would align with
test_flux1_example,test_flux2_example,test_ltx2_example, andtest_cosmos3_example.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@tests/integration/defs/examples/test_visual_gen.py` around lines 1423 - 1429, Replace the ad-hoc model path construction in tests/integration/defs/examples/test_visual_gen.py that uses scratch_space = conftest.llm_models_root() and os.path.join(..., WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH) with the existing helper _lpips_model_path() used by the other new tests; call _lpips_model_path(WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH) (or the equivalent helper import) to obtain model_path and keep the same pytest.skip behavior if the returned path is missing so the test matches test_flux1_example, test_flux2_example, test_ltx2_example, and test_cosmos3_example.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@tests/integration/defs/examples/test_visual_gen.py`:
- Around line 1423-1429: Replace the ad-hoc model path construction in
tests/integration/defs/examples/test_visual_gen.py that uses scratch_space =
conftest.llm_models_root() and os.path.join(...,
WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH) with the existing helper _lpips_model_path()
used by the other new tests; call
_lpips_model_path(WAN22_I2V_A14B_NVFP4_MODEL_SUBPATH) (or the equivalent helper
import) to obtain model_path and keep the same pytest.skip behavior if the
returned path is missing so the test matches test_flux1_example,
test_flux2_example, test_ltx2_example, and test_cosmos3_example.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 91452b27-922f-4ae1-aa7d-31d32381173f
📒 Files selected for processing (2)
tests/integration/defs/examples/test_visual_gen.pytests/integration/test_lists/test-db/l0_b200.yml
|
/bot run |
|
PR_Github #52875 [ run ] triggered by Bot. Commit: |
yibinl-nvidia
left a comment
There was a problem hiding this comment.
LGTM for the LTX-2 example test
|
PR_Github #52875 [ run ] completed with state
|
…e2e tests - test_ltx2_example: pass specific safetensors file (ltx-2-19b-dev.safetensors) instead of directory; LTX-2 VisualGen loader requires a checkpoint file path - test_cosmos3_example: set TRTLLM_DISABLE_COSMOS3_GUARDRAILS=1 to avoid cosmos_guardrail package dependency that is absent in CI Signed-off-by: Chang Liu <9713593+chang-l@users.noreply.github.com>
|
/bot run --disable-fail-fast |
|
PR_Github #52947 [ run ] triggered by Bot. Commit: |
|
PR_Github #52947 [ run ] completed with state |
Summary
Five example scripts were recently added under
examples/visual_gen/models/(
flux1.py,flux2.py,ltx2.py,wan_i2v.py,cosmos3_ti2v.py) but nonehad integration test coverage. This PR adds one smoke test per script and wires
them into CI.
Tests added (
tests/integration/defs/examples/test_visual_gen.py):test_flux1_example— runsflux1.py+configs/flux1-dev-fp4-1gpu.yamltest_flux2_example— runsflux2.py+configs/flux2-dev-fp4-1gpu.yamltest_ltx2_example— runsltx2.py+configs/ltx2-t2v-fp4-1gpu.yaml(passes--text_encoder_pathseparately since the shared YAML omits it intentionally)test_wan_i2v_example— runswan_i2v.py+configs/wan2.2-i2v-fp4-1gpu.yaml(uses NVFP4 pre-quantized checkpoint, defaultcat_piano.pnginput image)test_cosmos3_example— runscosmos3_ti2v.py+configs/cosmos3-nano-1gpu.yamlEach test:
LLM_MODELS_ROOTexamples/visual_gen/configs/.pngor.mp4) is producedCI registration (
tests/integration/test_lists/test-db/l0_b200.yml):All five tests registered in the B200 post-merge stage alongside the existing
test_wan_t2v_example.Test plan
Validate test list entries exist in source files (AST)pre-commit hook passes (already verified locally)🤖 Generated with Claude Code
Summary by CodeRabbit