fix(trainer): add torch.cuda.empty_cache() after FSDP update_actor

[Leon-Algo]

Summary

• Add torch.cuda.empty_cache() at the end of update_actor() in fsdp_workers.py to release GPU memory held by PyTorch's caching allocator after backward passes
• This prevents memory_reserved from growing monotonically across training steps, which eventually starves vLLM during weight synchronization in colocate mode

Problem

In colocate mode (vLLM + FSDP on the same GPU), PyTorch's caching allocator reserves GPU memory during backward passes and never voluntarily releases it back to CUDA. Over multiple training steps, memory_reserved grows monotonically:

Step 1: ~30.1 GiB
Step 5: ~34.0 GiB
Step 9:  38.2 GiB → OOM during vLLM weight sync (attempted 970 MiB, only 758 MiB available)

This happens because fsdp_workers.py's training path had zero empty_cache() calls, while:

• megatron_workers.py already calls aggressive_empty_cache(force_sync=True) at the end of update_actor (line 887) and in several other locations
• The vLLM worker's update_weight() already has empty_cache() for the inference side

The FSDP training path was the only backend missing this cleanup.

Fix

Add torch.cuda.empty_cache() at the end of update_actor(), after the offload section and before return output. This matches the existing pattern in megatron_workers.py.

Result with fix (Qwen3.5-0.8B, L20 46 GiB, colocate mode):

Step 1:  31.4 GiB
Step 2:  32.5 GiB
Step 5+: 32.9 GiB (stable, no further growth)
Step 10: 32.9 GiB ✓

Testing

Validated on L20 (46 GiB) with Qwen3.5-0.8B in colocate mode:

• 6 explore steps + 10 train steps completed with zero OOM
• max_memory_reserved stabilized at 32.9 GiB (vs. 38.2 GiB without fix)
• max_memory_allocated stable at 28.1-28.4 GiB
• GRPO advantages non-zero (max 0.92-1.50) throughout training

Files Changed

• trinity/trainer/verl/fsdp_workers.py: Add torch.cuda.empty_cache() at end of update_actor() (+6 lines)

🤖 Generated with Claude Code

[pan-x-c]

/unittest-pattern-TestTrainerGSM8K_1

[github-actions]

Summary

Tests 📝	Passed ✅	Failed ❌	Skipped ⏭️	Other ❓	Flaky 🍂	Duration ⏱️
1	1	0	0	0	0	5m 41s

Tests

Test Name	Status	Flaky	Duration
tests/trainer/trainer_test.py::TestTrainerGSM8K_1_fsdp2::test_trainer	✅		5m 29s

Github Test Reporter by CTRF 💚

[pan-x-c]

Thanks for pointing it out. LGTM