Skip to content

Refactor: drop Dist prefix from runtime types, files, and constants#587

Merged
ChaoWao merged 1 commit intohw-native-sys:mainfrom
ChaoWao:refactor/drop-dist-prefix
Apr 17, 2026
Merged

Refactor: drop Dist prefix from runtime types, files, and constants#587
ChaoWao merged 1 commit intohw-native-sys:mainfrom
ChaoWao:refactor/drop-dist-prefix

Conversation

@ChaoWao
Copy link
Copy Markdown
Collaborator

@ChaoWao ChaoWao commented Apr 17, 2026

Summary

Mechanical rename sweep across the hierarchical runtime — no behavioural changes. Final PR in the refactor chain (follows #563, #564, #570, #572, #575, #578, #583).

File renames (history preserved via git mv)

  • src/common/distributed/dist_*.{h,cpp}*.{h,cpp}
  • tests/ut/cpp/test_dist_*.cpptest_*.cpp
  • tests/ut/py/test_dist_worker/test_worker/
  • python/bindings/dist_worker_bind.hworker_bind.h

Type renames

DistWorkerWorker, DistOrchestratorOrchestrator, DistSchedulerScheduler, DistWorkerManagerWorkerManager, DistRingRing, DistScopeScope, DistTensorMapTensorMap, DistReadyQueueReadyQueue, DistSubmitResultSubmitResult, DistTaskSlotStateTaskSlotState, DistAllocResultAllocResult.

Constant renames

DIST_MAX_RING_DEPTHMAX_RING_DEPTH, DIST_MAILBOX_SIZEMAILBOX_SIZE, DIST_HEAP_ALIGNHEAP_ALIGN, DIST_INVALID_SLOTINVALID_SLOT, DIST_MAX_SCOPE_DEPTHMAX_SCOPE_DEPTH, DIST_ALLOC_TIMEOUT_MSALLOC_TIMEOUT_MS, DIST_DEFAULT_HEAP_RING_SIZEDEFAULT_HEAP_RING_SIZE, DIST_MAILBOX_ARGS_CAPACITYMAILBOX_ARGS_CAPACITY.

Python binding naming

The C++ types Worker and Orchestrator are bound as _Worker and _Orchestrator to avoid colliding with the user-facing Python simpler.worker.Worker factory and simpler.orchestrator.Orchestrator facade, which wrap them. Matches the existing _ChipWorkerChipWorker pattern.

Roadmap cleanup

Removes three stale roadmap docs now that the refactor chain is complete:

  • docs/roadmap.md — every bullet is now covered by the per-component docs (orchestrator/scheduler/worker-manager/task-flow). The "Behavioural notes" content (release_ref +1 threshold, fork hygiene setenv + pthread_atfork) already lives in orchestrator.md §8 and §"Fork hygiene before fork". Also removes six forward-reference blockquotes ("see roadmap.md for landed-vs-planned") from the per-component docs and the README doc table entry.
  • src/{a2a3,a5}/runtime/tensormap_and_ringbuffer/docs/ROADMAP.md — all three proposed features have landed, via a different API than the roadmap described. MixedKernels + atomic cluster dispatch replaced the allocate_cluster/free_cluster proposal (explicitly "Out of Scope" in SUBMIT_BY_CLUSTER.md); PTO2LaunchSpec covers SPMD→MPMD block_incore expansion; cube+vector co-scheduling rides on the MixedKernels path. Current runtime design lives in RUNTIME_LOGIC.md + SUBMIT_BY_CLUSTER.md.

Test plan

  • pip install -e . builds cleanly
  • Python unit tests: 160 passed, 5 skipped (3 pre-existing failures require torch, unrelated)
  • C++ unit tests: 7/7 passed via ctest (test_tensormap, test_ring, test_scope, test_orchestrator, test_scheduler, test_a2a3_pto2_fatal, test_a5_pto2_fatal)
  • grep -rnE "Dist[A-Z]|DIST_" src/ python/ returns no results
  • ls src/common/distributed/dist_* / ls tests/ut/cpp/test_dist_* return nothing
  • Hardware tests (no device available in authoring environment)

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request implements a major refactor of the distributed runtime by dropping the "Dist" prefix from class names, constants, and source files, and removing the project roadmap and associated status documentation. Feedback highlights a significant risk of name collisions resulting from moving generic names into the global namespace, suggesting the use of a proper C++ namespace for these definitions. Furthermore, inconsistencies were noted in the documentation updates, specifically regarding the naming of configuration types.

Comment thread src/common/hierarchical/types.h
Comment thread src/common/hierarchical/ring.h
Comment thread docs/chip-level-arch.md Outdated
Mechanical rename sweep — no behavioural changes.

File renames (git mv, history preserved):
- src/common/distributed/ → src/common/hierarchical/
- src/common/{hierarchical}/dist_*.{h,cpp} → *.{h,cpp}
- tests/ut/cpp/test_dist_*.cpp → test_*.cpp
- tests/ut/py/test_dist_worker/ → test_worker/
- python/bindings/dist_worker_bind.h → worker_bind.h
- docs/distributed_level_runtime.md → docs/hierarchical_level_runtime.md

Type renames:
- DistWorker → Worker, DistOrchestrator → Orchestrator,
  DistScheduler → Scheduler, DistWorkerManager → WorkerManager,
  DistRing → Ring, DistScope → Scope, DistTensorMap → TensorMap,
  DistReadyQueue → ReadyQueue, DistSubmitResult → SubmitResult,
  DistTaskSlotState → TaskSlotState, DistAllocResult → AllocResult.

Constant renames:
- DIST_MAX_RING_DEPTH → MAX_RING_DEPTH, DIST_MAILBOX_SIZE → MAILBOX_SIZE,
  DIST_HEAP_ALIGN → HEAP_ALIGN, DIST_INVALID_SLOT → INVALID_SLOT,
  DIST_MAX_SCOPE_DEPTH → MAX_SCOPE_DEPTH, DIST_ALLOC_TIMEOUT_MS →
  ALLOC_TIMEOUT_MS, DIST_DEFAULT_HEAP_RING_SIZE → DEFAULT_HEAP_RING_SIZE,
  DIST_MAILBOX_ARGS_CAPACITY → MAILBOX_ARGS_CAPACITY.

Folder / doc rename: src/common/distributed/ → src/common/hierarchical/
to match the "hierarchical runtime" terminology used throughout the
level-composition docs. The corresponding doc file
docs/distributed_level_runtime.md is renamed to
docs/hierarchical_level_runtime.md for consistency. Internal identifiers
touched by the folder rename:
- CMake vars DISTRIBUTED_SRC{,_DIR}/DISTRIBUTED_SOURCES → HIERARCHICAL_*
- CMake project `distributed_ut` → `hierarchical_ut`;
  function `add_distributed_test` → `add_hierarchical_test`
- Python private methods `_init_distributed` / `_start_distributed`
  (and the `_distributed_started` flag) → `_init_hierarchical` /
  `_start_hierarchical` / `_hierarchical_started`.

Python bindings keep the underscore-prefix convention so they don't
collide with the user-facing Python wrapper classes: C++ Worker is
bound as _Worker, C++ Orchestrator as _Orchestrator. The Python
simpler.worker.Worker factory and simpler.orchestrator.Orchestrator
facade wrap them.

Historical references to the pre-PR-D-2 classes DistChipProcess and
DistSubWorker in comments and docs have been updated to ChipProcess /
SubWorker.

Also removes three stale roadmap docs now that the refactor chain is
complete:
- docs/roadmap.md — every bullet is now covered by the per-component
  docs (orchestrator/scheduler/worker-manager/task-flow); the
  "Behavioural notes" content (release_ref +1 threshold, fork hygiene
  setenv + pthread_atfork) already lives in orchestrator.md §8 and
  §"Fork hygiene before fork". Removes six forward-reference
  blockquotes ("see roadmap.md for landed-vs-planned") from the
  per-component docs and the README doc table entry.
- src/{a2a3,a5}/runtime/tensormap_and_ringbuffer/docs/ROADMAP.md — all
  three proposed features have landed (via a different API than the
  roadmap described): MixedKernels + atomic cluster dispatch replaced
  the allocate_cluster/free_cluster proposal (explicitly "Out of
  Scope" in SUBMIT_BY_CLUSTER.md); PTO2LaunchSpec covers SPMD→MPMD
  block_incore expansion; cube+vector co-scheduling rides on the
  MixedKernels path. Current runtime design lives in
  RUNTIME_LOGIC.md + SUBMIT_BY_CLUSTER.md.
@ChaoWao ChaoWao force-pushed the refactor/drop-dist-prefix branch from 8576df7 to d11e7e2 Compare April 17, 2026 03:41
@ChaoWao ChaoWao merged commit 6881fcc into hw-native-sys:main Apr 17, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant