Inquiry about support/memory usage when LoRA fine-tuning Wan2.1-14B

[Radioheading]

Hi, nice work! I want to use multiple GPUs or multiple nodes for WAN2.1- 14 B LoRA training, and I want to check SimpleTuner's support for multi-GPU/multi-node LoRA training of WAN, as well as the memory consumption. This would be very helpful. Many thanks!

[bghira]

ok, we have FSDP2, DeepSpeed, musubi block swap, RamTorch, and Diffusers' group offload available. some are compatible with torch inductor too.