delete unimplemented broadcast transport path [need discussion]#24
delete unimplemented broadcast transport path [need discussion]#24TianyeGGBond wants to merge 1 commit into
Conversation
sync_selected_workers advertised a two-path transport via a public broadcast_local_ranks kwarg, but any non-empty value immediately raised NotImplementedError because the cache-owner sender-side NCCL path was never wired. In practice only the cpu_serialize path runs. Remove the broadcast kwarg, validation, and NotImplementedError so the service exposes only the path it actually implements. The wire plan is unchanged: broadcast_local_ranks is still emitted (empty), comm_ranks are all 0, and world_size is 1 — exactly the values the cpu_serialize-only path already produced, so the MILES run_sync_session contract is untouched. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
Evidence for the "[need discussion]" from the F1-F12 review's deep pass over both sides of this path:
So +1 to deletion from our side; the sender-side NCCL work is cleanly re-addable later and the plan-shape ( |
|
Update on the "[need discussion]" — leaving this open rather than merging or closing it, so the team can make the call. Since opening this I'd lean the other way: keep the broadcast path. It's the normal, faster GPU transport, and the reason it's unimplemented isn't that we don't want it — it's that we don't currently have the setup to test the sender-side NCCL. Deleting it now means re-adding it later, so I'd rather leave it guarded behind @JunzheJoe your deletion evidence stands — the path really is double-guarded dead today, and the port-claim machinery really does run every sync for it. So this is a judgement call, not a correctness dispute, and the two PRs are mutually exclusive:
My vote is the first. Leaving this open so it's an explicit team decision rather than me unilaterally deleting a capability we intend to use. |
Context
MilesModelUpdateService.sync_selected_workerswas named/documented as atwo-path weight transport (cpu_serialize + NCCL broadcast) and exposed a
public
broadcast_local_rankskwarg. But any non-empty value immediatelyraised
NotImplementedError: the cache-owner sender-side NCCL path(
init_process_group+ per-bucketdist.broadcast) was never wired — onlythe receiver-side fan-out existed.
So the only path that ever runs is cpu_serialize. The broadcast kwarg,
subset validation, NCCL rank assignment, and
world_sizemath are deadcode that makes the surface look like it offers something it doesn't.
Change
pipeline/miles_model_update_service.py:broadcast_local_rankskwarg fromsync_selected_workers,along with the subset validation and the
NotImplementedErrorbranch.Every target now goes over cpu_serialize.
broadcast_setparameter threaded through_run_atomic_unitand
_build_plan, and the broadcast field from the done-log line._build_plan, replace the broadcast NCCL rank loop /world_sizecalculation with single-path constants.
Wire compatibility
The wire plan shape is intentionally unchanged.
broadcast_local_ranksisstill present (empty),
comm_ranksare all 0, andworld_sizeis 1 — theexact values the cpu_serialize-only path already produced. The MILES
run_sync_sessionplan-key validator sees no difference.Net: +20 / -60 lines, no behavior change on the live path.