feat(rocm): unified memory true by construction on integrated APUs (issue #41 F6, approach (b)) - #144
Closed
localai-bot wants to merge 2 commits into
Closed
feat(rocm): unified memory true by construction on integrated APUs (issue #41 F6, approach (b))#144localai-bot wants to merge 2 commits into
localai-bot wants to merge 2 commits into
Conversation
mudler
added a commit
that referenced
this pull request
Aug 8, 2026
…egrated APUs Implements the maintainer-ratified approach (b) from issue #41 F6. On a device probing hipDeviceAttributeIntegrated=1 + ManagedMemory=1 + ConcurrentManagedAccess=1, RocmBackend::Alloc allocates through hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory() returns true exactly then: host access to every Backend::Alloc block becomes API-guaranteed rather than architecturally incidental, which is what the CPU reference tier's host-dereference contract needs on XNACK-less RDNA3 APUs (gfx1151/gfx1103 measure PageableMemoryAccess=0, vetoing the W0 CUDA-shaped probe even though the aliasing demonstrably holds there). - The W0 conjunction (integrated AND pageable) stays as ground 1; Free stays hipFree, API-documented for both alloc paths; the inherited AllocPinned delegates to Alloc (backend.cpp:19) and rides the branch, coherent with its unified-memory contract (backend.h:76-78). - Discrete devices byte-identical to W0: the branch is provably dead at Integrated=0, asserted by a new runtime test through the new HIP-free probes ManagedAllocActive/IntegratedDevice (rocm_runtime.h). - F6's decisive experiment is now a standing test: host-write inputs with no Copy, native RmsNorm kernel reads them, host-read the kernel-written output with no Copy. - CMake F1/F3 absorption: when ROCM_PATH exists, derive CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS and export ROCM_PATH, each only when unset, before check_language(HIP) — Arch/TheRock layouts configure flag-free (F2 was downstream of the unidentified compiler; no separate change). - No upstream analog to mirror (vLLM allocates via torch; rocm.py:75-77, 909-910 only name the APUs): recorded as ADDITIVE deviation, porting-inventory §9.14. Spec: .agents/specs/rocm-unified-memory-b.md (decision quoted verbatim + the community verification table). Lands via row/ROCM-UNIFIED-MEMORY-B (PR #144). BLIND-WRITTEN: no AMD GPU or hipcc on the authoring machine; the .hip delta has never been compiled. CPU gates green (preflight EXIT=0; -Werror object-compile of the platform + test TUs; compile-mutation of the new probe API red then green on restore). Community evidence owed per the spec table; PR #144 carries the ask. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
localai-bot
marked this pull request as ready for review
August 8, 2026 07:49
mudler
added a commit
that referenced
this pull request
Aug 8, 2026
…egrated APUs Implements the maintainer-ratified approach (b) from issue #41 F6. On a device probing hipDeviceAttributeIntegrated=1 + ManagedMemory=1 + ConcurrentManagedAccess=1, RocmBackend::Alloc allocates through hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory() returns true exactly then: host access to every Backend::Alloc block becomes API-guaranteed rather than architecturally incidental, which is what the CPU reference tier's host-dereference contract needs on XNACK-less RDNA3 APUs (gfx1151/gfx1103 measure PageableMemoryAccess=0, vetoing the W0 CUDA-shaped probe even though the aliasing demonstrably holds there). - The W0 conjunction (integrated AND pageable) stays as ground 1; Free stays hipFree, API-documented for both alloc paths; the inherited AllocPinned delegates to Alloc (backend.cpp:19) and rides the branch, coherent with its unified-memory contract (backend.h:76-78). - Discrete devices byte-identical to W0: the branch is provably dead at Integrated=0, asserted by a new runtime test through the new HIP-free probes ManagedAllocActive/IntegratedDevice (rocm_runtime.h). - F6's decisive experiment is now a standing test: host-write inputs with no Copy, native RmsNorm kernel reads them, host-read the kernel-written output with no Copy. - CMake F1/F3 absorption: when ROCM_PATH exists, derive CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS and export ROCM_PATH, each only when unset, before check_language(HIP) — Arch/TheRock layouts configure flag-free (F2 was downstream of the unidentified compiler; no separate change). - No upstream analog to mirror (vLLM allocates via torch; rocm.py:75-77, 909-910 only name the APUs): recorded as ADDITIVE deviation, porting-inventory §9.14. Spec: .agents/specs/rocm-unified-memory-b.md (decision quoted verbatim + the community verification table). Lands via row/ROCM-UNIFIED-MEMORY-B (PR #144). BLIND-WRITTEN: no AMD GPU or hipcc on the authoring machine; the .hip delta has never been compiled. CPU gates green (preflight EXIT=0; -Werror object-compile of the platform + test TUs; compile-mutation of the new probe API red then green on restore). Community evidence owed per the spec table; PR #144 carries the ask. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
mudler
force-pushed
the
row/ROCM-UNIFIED-MEMORY-B
branch
from
August 8, 2026 07:54
c0b3e8f to
9abd304
Compare
…emory by construction Issue #41's F6 fork is decided: on integrated managed-capable devices the backend will allocate through hipMallocManaged so host access is API-guaranteed, and UnifiedMemory() returns true exactly then (maintainer comment, 2026-08-08, quoted in docs/ROCM.md §3.1). This commit is the contributor-facing statement: the decision and its behavior table (integrated vs discrete), the community-verified W0 state (M0/M1 MET on gfx1151/gfx1103/gfx1100/gfx1201), the two teardown known-issues, and the exact post-fix sequence for board owners (§5.2). The implementation follows in the next commit. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
…egrated APUs Implements the maintainer-ratified approach (b) from issue #41 F6. On a device probing hipDeviceAttributeIntegrated=1 + ManagedMemory=1 + ConcurrentManagedAccess=1, RocmBackend::Alloc allocates through hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory() returns true exactly then: host access to every Backend::Alloc block becomes API-guaranteed rather than architecturally incidental, which is what the CPU reference tier's host-dereference contract needs on XNACK-less RDNA3 APUs (gfx1151/gfx1103 measure PageableMemoryAccess=0, vetoing the W0 CUDA-shaped probe even though the aliasing demonstrably holds there). - The W0 conjunction (integrated AND pageable) stays as ground 1; Free stays hipFree, API-documented for both alloc paths; the inherited AllocPinned delegates to Alloc (backend.cpp:19) and rides the branch, coherent with its unified-memory contract (backend.h:76-78). - Discrete devices byte-identical to W0: the branch is provably dead at Integrated=0, asserted by a new runtime test through the new HIP-free probes ManagedAllocActive/IntegratedDevice (rocm_runtime.h). - F6's decisive experiment is now a standing test: host-write inputs with no Copy, native RmsNorm kernel reads them, host-read the kernel-written output with no Copy. - CMake F1/F3 absorption: when ROCM_PATH exists, derive CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS and export ROCM_PATH, each only when unset, before check_language(HIP) — Arch/TheRock layouts configure flag-free (F2 was downstream of the unidentified compiler; no separate change). - No upstream analog to mirror (vLLM allocates via torch; rocm.py:75-77, 909-910 only name the APUs): recorded as ADDITIVE deviation, porting-inventory §9.14. Spec: .agents/specs/rocm-unified-memory-b.md (decision quoted verbatim + the community verification table). Lands via row/ROCM-UNIFIED-MEMORY-B (PR #144). BLIND-WRITTEN: no AMD GPU or hipcc on the authoring machine; the .hip delta has never been compiled. CPU gates green (preflight EXIT=0; -Werror object-compile of the platform + test TUs; compile-mutation of the new probe API red then green on restore). Community evidence owed per the spec table; PR #144 carries the ask. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
mudler
force-pushed
the
row/ROCM-UNIFIED-MEMORY-B
branch
from
August 8, 2026 08:26
9abd304 to
5c8969f
Compare
mudler
added a commit
that referenced
this pull request
Aug 8, 2026
…proach (b) from issue #41 + Arch/TheRock CMake absorption (#144) Implements the ratified maintainer decision on issue #41 F6: RocmBackend::Alloc branches to hipMallocManaged(hipMemAttachGlobal) when Integrated && ManagedMemory && ConcurrentManagedAccess, and UnifiedMemory() returns true exactly then - host access becomes API-guaranteed rather than architecturally incidental, so the zero-kernel CPU-reference tier installs and M2 (model e2e) unblocks on the community boards (Strix Halo gfx1151, 780M gfx1103). Discrete devices provably unchanged (branch dead at Integrated=0, asserted by a new runtime-gated test; F6's kernel-write->host-read-no-copy experiment is now a standing gate). hipFree serves both alloc paths (no hipFreeManaged exists; stated in-code as a falsifiable assumption). CMake absorbs the F1/F3 Arch/TheRock layout hints (ROCM_PATH-derived, only-when-unset, non-ROCm machines byte-identical); F2 documented as environment-side. Upstream has no analog (vLLM allocates via torch) - recorded as an additive deviation in porting-inventory 9.14, with the ROCM_PATH handling mirrored from vllm/CMakeLists.txt:54-60 at the pin. WRITTEN BLIND per this lane's policy (no AMD hardware here): all HIP-side evidence is PENDING-community, itemized in the spec + PR; the handoff comment on issue #41 gives the three board owners the branch, the now-flag-free configure, and the exact report format. Operator landing basis: PR CI 12/12 green on this head (full CPU build + both sanitizers), local -Werror syntax-check target green, nine record gates green; a mutation-review round was deliberately skipped as disproportionate for a blind-HIP lane whose real gate is community hardware (the skeleton precedent) - disclosed here. FOLLOWING_AGENTS_PROTOCOL Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
Collaborator
Author
|
Landed on main as e9f3561 (mudler-authored squash, lease-guarded push). Basis: PR CI 12/12 green on the head (full CPU build + both sanitizers), local -Werror syntax-check target green, nine record gates green; the mutation-review round was deliberately skipped as disproportionate for a blind-HIP lane whose real gate is community hardware (disclosed in the commit). @jimmykarily @arch-btw @tbrasser — approach (b) is now on main: pull main (or keep testing the branch, identical content), run the flag-free configure + ctest -R 'rocm|cross_device' + the M2 attempt per the issue-41 handoff comment, and post the table. A compile error remains the most useful thing you can post. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
The ratified approach (b) from issue #41's F6: on a ROCm device probing
hipDeviceAttributeIntegrated=1+ManagedMemory=1+ConcurrentManagedAccess=1,Backend::AllocuseshipMallocManaged(hipMemAttachGlobal)andUnifiedMemory()returns true exactly then — host access becomes API-guaranteed rather than
architecturally incidental, unblocking the zero-kernel reference-tier M2 path on
Strix Halo (gfx1151) and the 780M (gfx1103), where XNACK-less RDNA3 probes
PageableMemoryAccess=0and the W0 CUDA-shaped conjunction read false.Draft opened at start per the helper protocol; full work lands on this branch.
Honesty statement — read before reviewing
Written blind. No AMD GPU and no hipcc exist on the authoring machine; the
.hipdelta in this PR has never been compiled. That is this lane's standingpolicy (the W0 skeleton landed the same way and was then verified by four
community boards on #41). CPU-side gates are green here; the HIP-side evidence
is owed by board owners, itemized below. A compile error in this PR is useful
data, not a review failure.
The change
src/vt/rocm/rocm_backend.hip— probe the two managed attributes (failuredefaults to 0, never unregisters the device);
UseManagedAlloc()predicate;managed branch in
Alloc(bytes==0 stays onhipMalloc);UnifiedMemory() = managed_branch || (integrated && pageable)(W0 ground kept intact);Freestays
hipFree— the HIP API documents it as the release call for bothhipMallocandhipMallocManaged(mirrorscudaFree; nohipFreeManagedexists), certainty stated in the code.
Allocbranches;Freesingle-path;inherited
AllocPinned/FreePinneddelegate toAlloc/Free(
src/vt/backend.cpp:19-20) and so ride the same branch, coherent with thepinned contract (
include/vt/backend.h:76-78);rocm_rmsnorm.hiphas zeroallocation sites; no pool paths exist in the skeleton. Discrete devices are
byte-identical to W0: the branch is provably dead at
Integrated=0, and aruntime test asserts it.
include/vt/rocm/rocm_runtime.h— HIP-free introspection probesManagedAllocActive(index)/IntegratedDevice(index)(the testable seam,also what board owners report).
tests/vt/test_rocm_backend.cpp— two runtime-gated cases: (1) alloc pathand
UnifiedMemory()move together (discrete: both dead/false; integrated:managed active and unified true, loud failure with the probe triple
otherwise); (2) issue ROCm (AMD GPU) backend #41 F6's decisive experiment as a standing gate —
host-writes inputs with no Copy, the native RmsNorm kernel reads them,
host-reads the kernel-written output with no Copy. Both compile on every
build via the
-Werrorsyntax-check object; both no-op without a device.CMakeLists.txt— F1/F3 absorption: whenROCM_PATHexists, deriveCMAKE_HIP_COMPILER_ROCM_ROOT, seed--rocm-pathintoCMAKE_HIP_FLAGS,export
ROCM_PATHinto the environment — each ONLY when unset, beforecheck_language(HIP). Arch/TheRock should now configure with no manualflags. F2 was downstream of the unidentified compiler and disappears with
F1/F3 (documented, no separate change).
docs/ROCM.md§3.1 (decision + behavior table) and §5.2(board-owner sequence),
docs/BUILD.md, spec.agents/specs/rocm-unified-memory-b.md(decision quoted verbatim +community verification table), backend-matrix row, porting-inventory §9.14
(ADDITIVE deviation: upstream vLLM allocates via torch and has no analog —
rocm.py:75-77,909-910only name the APUs), STATUS/BENCHMARKS/FEATURESrows, state + NOW.
Gates run here (CPU only)
scripts/agent-preflight.shEXIT=0 (all record gates + mutation suites).-Werrorobject-compile of the platform and test TUs(
vllm_rocm_platform_syntax_check): clean; compile-mutation of the newprobe API went red (2 errors) and green on restore.
.hipfiles remain excluded from non-HIP builds — byte-identical CPUbuilds.
Evidence the community owes (PENDING — the M0/M1 table shape from #41)
.hipcompile of this deltactest -R 'rocm|cross_device'incl. the 2 new casesintegrated/managed-alloc/UnifiedMemorytriple0/0/false--device cpuVT_OP_PROVIDER_STATS=1fallback listSequencing note: composes with #140 (discrete gfx1201 lane) — this PR touches
the integrated-path branch only; will rebase onto main if #140 lands first.
Closes the F6 blocker on #41 (task #286).