Tianye/miles 2gpu overlap

[TianyeGGBond]

PR: 2-GPU full-overlap planner starvation recovery

Background

User-requested 2ppl partial-overlap 拓扑，2 块 GPU 都被两条 pipeline 同时当 infer GPU 用：

P1: train=[0]  infer=[0,1]
P2: train=[1]  infer=[0,1]

和 scripts/run_smoke_dual.sh（4-GPU disjoint pool）相比，这里每个 train GPU 同时也是对面 pipeline 的 infer GPU，触发 scheduler.py:1027-1073 的 donor-shrink-before-receiver-expand 路径。在 11 次 attempt 的 debug（见 debug_log.md）里发现一个新死锁，本 PR 修掉它。

Root cause

死锁在 rlix/scheduler/planner.py:222-232 的 gap-ratio planner：

1. P2 进入 INIT (priority=INITIALIZATION=0)，把 P1 的 actor_infer 两个 engine 全 donor-shrink 走
2. P2 INIT 完成、释放回 GPU
3. P1 还从未跑过一次 rollout → step_target == 0
4. P1 的 INIT-期 PendingRequest 已被 signal 消费 → pending_bucket_gen 里没 P1 的 GEN 请求
5. planner 走 step_target_estimate is None: continue，把 P1 完全踢出 pipeline_states
6. 后续分配根本看不到 P1，P1 永远拿不回 GPU → 单向死锁

P1 starve 后没有任何外部触发能把它救出来：F40 Runtime 的 _expand_workers 只在自己的 after_training 里 fire，而 after_training 必须先有一次成功 rollout，rollout 又需要 ≥1 个 active engine — 闭环。

The fix

在 plan_generation_gap_ratio 里把 step_target_estimate is None: continue 改成三分支 if/elif/else：

if step_target_estimate is not None:
    # 原 pending GEN 路径，行为不变
    ...
elif cluster_id in active_allocations:
    # 饥饿/无需求恢复：scheduler 仍记录 P1 持有 actor_infer，
    # 给 floor demand=1，下游 max(rounded*tp_size, tp_size) 至少恢复一个 TP bundle
    remaining = 1.0
    step_target = 1.0
    percent_remaining = 1.0
else:
    continue

active_allocations 守卫的语义关键——它区分两种情况：

• ✅ 持有 cluster 但 0 demand（被饿死 / 无 pending request）→ 进 pipeline_states，让 planner 同时把它当 receiver（恢复）和 donor（让出给真有需求的 peer）
• ❌ 主动 release（notify_release_gpus 会清掉 active_allocations）→ 走 continue，保持原行为

Verification

Smoke: scripts/run_smoke_dual_2gpu.sh（本 PR 新增，从 run_smoke_dual.sh 改的 2-GPU 变体）

完整时间线（attempt 11 v7b）：

• 18:16:13 P1 init complete + activate_routing=[0,1]
• 18:16:34 ~ 18:19:19 P1 被 P2 INIT 抢占多轮，饥饿守卫数次救场自动恢复
• 18:19:46 P2 拿到 engine 0（gap-ratio 把 P1 一个 engine donate 给 P2）
• 18:20:59 / 18:21:01 两条 rollout 完成
• 18:22:26 Megatron train end elapsed=23.5s, step 0 metrics 全有
• 18:22:27 cpu_bucket_cache published + 两条 training loop complete
• 18:22:28 shutdown_hard complete，EXIT_CODE=0
• post-run nvidia-smi: GPU 0 = 1 MiB, GPU 1 = 1 MiB（清理干净）

回归保护: tests/test_gap_ratio.py 7 个用例，新分支不影响任何现有 case。

无 race / 无震荡:

• if step_target_estimate is not None 仍然先判，pending 路径优先级最高
• active_allocations 是 scheduler 内部状态，每个 plan cycle 在锁内读取
• floor demand=1 只让 P1 排到一份 TP bundle；如果 sibling 满载，_compute_shrink_budget_by_pipeline_id 会保护对方 ≥ target_gpu_count
• P1 拿回 ≥1 engine、跑完第一次 rollout 后，下一周期 progress_totals_fn 返回真实 step_target > 0，本分支不再触发

Files changed

• rlix/scheduler/planner.py (+37 / -4) — 饥饿恢复分支 + _STARVATION_FLOOR_STEP_DEMAND 常量
• scripts/run_smoke_dual_2gpu.sh (+133) — 新增，2-GPU 全 overlap smoke

Notes

• 上游 4-GPU smoke 跑得通是因为每条 pipeline 都有独占 infer GPU，永远不会被完全饿死。本 PR 不影响 4-GPU 路径。
• miles 仓库零改动

[taoluo]

I think the root issue is that pending_bucket_gen is only a transient signal queue, not durable rollout intent. Original approach of rely on it is too hacky.

Once a pending GEN request is consumed, absence from the bucket no longer proves the pipeline has no rollout GPU demand. It may simply mean the signal was consumed before rollout data was actually produced.

A more reliable signal may be the pipeline-side batch state. If the pipeline has opened a rollout batch and is blocked in get_batch(), then rollout GPU demand is still live until that batch is produced/consumed or explicitly closed by release/cancel/failure.

So instead of using active_allocations plus a synthetic floor demand, can we recover the intended step target from the open rollout-batch / waiting-for-get_batch() state?