feat(ds4): run CUDA and HIP expert parallelism in one process

[davide221]

Summary

• Run DeepSeek V4 CUDA and HIP expert owners in one process; no IPC daemon is required.
• Load the non-primary runtime as an isolated backend module, with distinct runtime identities and safe executable-relative discovery.
• Place experts from a routing profile, submit the two owner branches independently, and join them through the common multi-backend scheduler.
• Batch unlike-runtime transfers through reusable pinned host arenas with one source wait per split.
• Keep the mixed path opt-in and retain model-neutral policy names with the existing DeepSeek-specific names as compatibility aliases.

Review fixes

• Fixed canonical route-order branch scheduling so both independently expanded owner branches are preserved.
• Fixed scheduler staging lifetime races across async reset, graph resize, blocking fallback, and scheduler destruction.
• Fixed CUDA/HIP registry identity collisions that could permit unsafe native casts and copies.
• Fixed grouped projection strides after cross-runtime copies lose their view metadata.
• Made explicit backend paths authoritative and removed current-working-directory module loading.
• Scoped the shared-ggml CMake requirement to mixed builds, reset module cache flags on reconfigure, and added CUDA runtime rpaths.
• Added an early, target-aware Linux-only configuration guard.
• Consolidated heterogeneous MoE environment policy into a typed, model-neutral interface and removed unused placement API.

Lucebox2 validation

Hardware: Strix Halo 128 GiB + RTX 3090 24 GiB.

• HIP-primary + CUDA-module configure/build: passed.
• CUDA-primary + HIP-module configure/build: passed.
• Mixed scheduler tests in both directions, batched and unbatched: passed.
• Async reset, arena resize, multi-source join, and free lifecycle cases: passed.
• Ordinary non-mixed HIP build: passed.
• Full non-mixed suite: 339/339 passed.
• Reused-build-directory cache regression: mixed module flags and BUILD_SHARED_LIBS reset correctly.
• Unsupported-host configuration fails before GPU compiler discovery with the intended message.

Correctness

The meaningful all-6-expert mixed run placed 9,632 expert tensors on Strix and 1,376 on the 3090. Reference-exact mixed decode produced the same 128-token response hash as Strix-only:

0f785a7ffa406498aafb14553966eaed0f52220fed0f7cc016b66921d104d194

That run measured 15.751 tok/s versus 21.452 tok/s for Strix-only. Reference-exact mode is intended for identity validation, not peak throughput.

Performance profile

The tuned top-4 mixed profile completed 5/5 deterministic 128-token runs at 48.059 tok/s median (47.444-49.136), with a stable response hash. A 1,956-token prompt plus 256-token decode measured 41.181 tok/s.

Top-4 routing, sparse prefill, and owner-local sums are explicit performance choices. Owner-local sums change floating-point association and are not the byte-identity mode.

Defaults and risk

Mixed CUDA+HIP execution and its performance switches remain disabled by default and documented as burn-in. Existing single-runtime builds do not load a peer module or enable heterogeneous scheduling.

Merge status

Ready for review. All six CI checks for the final hardening commit are green.

[cubic-dev-ai]

All reported issues were addressed across 21 files

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>