Skip to content

feat(rocm): unified memory true by construction on integrated APUs (issue #41 F6, approach (b)) - #144

Closed
localai-bot wants to merge 2 commits into
mainfrom
row/ROCM-UNIFIED-MEMORY-B
Closed

feat(rocm): unified memory true by construction on integrated APUs (issue #41 F6, approach (b))#144
localai-bot wants to merge 2 commits into
mainfrom
row/ROCM-UNIFIED-MEMORY-B

Conversation

@localai-bot

Copy link
Copy Markdown
Collaborator

What this is

The ratified approach (b) from issue #41's F6: on a ROCm device probing
hipDeviceAttributeIntegrated=1 + ManagedMemory=1 + ConcurrentManagedAccess=1,
Backend::Alloc uses hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory()
returns true exactly then — host access becomes API-guaranteed rather than
architecturally incidental, unblocking the zero-kernel reference-tier M2 path on
Strix Halo (gfx1151) and the 780M (gfx1103), where XNACK-less RDNA3 probes
PageableMemoryAccess=0 and the W0 CUDA-shaped conjunction read false.

Draft opened at start per the helper protocol; full work lands on this branch.

Honesty statement — read before reviewing

Written blind. No AMD GPU and no hipcc exist on the authoring machine; the
.hip delta in this PR has never been compiled.
That is this lane's standing
policy (the W0 skeleton landed the same way and was then verified by four
community boards on #41). CPU-side gates are green here; the HIP-side evidence
is owed by board owners, itemized below. A compile error in this PR is useful
data, not a review failure.

The change

  • src/vt/rocm/rocm_backend.hip — probe the two managed attributes (failure
    defaults to 0, never unregisters the device); UseManagedAlloc() predicate;
    managed branch in Alloc (bytes==0 stays on hipMalloc); UnifiedMemory() = managed_branch || (integrated && pageable) (W0 ground kept intact); Free
    stays hipFree — the HIP API documents it as the release call for both
    hipMalloc and hipMallocManaged (mirrors cudaFree; no hipFreeManaged
    exists), certainty stated in the code.
  • Allocation-site audit (all of them): Alloc branches; Free single-path;
    inherited AllocPinned/FreePinned delegate to Alloc/Free
    (src/vt/backend.cpp:19-20) and so ride the same branch, coherent with the
    pinned contract (include/vt/backend.h:76-78); rocm_rmsnorm.hip has zero
    allocation sites; no pool paths exist in the skeleton. Discrete devices are
    byte-identical to W0: the branch is provably dead at Integrated=0, and a
    runtime test asserts it.
  • include/vt/rocm/rocm_runtime.h — HIP-free introspection probes
    ManagedAllocActive(index) / IntegratedDevice(index) (the testable seam,
    also what board owners report).
  • tests/vt/test_rocm_backend.cpp — two runtime-gated cases: (1) alloc path
    and UnifiedMemory() move together (discrete: both dead/false; integrated:
    managed active and unified true, loud failure with the probe triple
    otherwise); (2) issue ROCm (AMD GPU) backend #41 F6's decisive experiment as a standing gate —
    host-writes inputs with no Copy, the native RmsNorm kernel reads them,
    host-reads the kernel-written output with no Copy. Both compile on every
    build via the -Werror syntax-check object; both no-op without a device.
  • CMakeLists.txt — F1/F3 absorption: when ROCM_PATH exists, derive
    CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS,
    export ROCM_PATH into the environment — each ONLY when unset, before
    check_language(HIP). Arch/TheRock should now configure with no manual
    flags. F2 was downstream of the unidentified compiler and disappears with
    F1/F3 (documented, no separate change).
  • Docs + records: docs/ROCM.md §3.1 (decision + behavior table) and §5.2
    (board-owner sequence), docs/BUILD.md, spec
    .agents/specs/rocm-unified-memory-b.md (decision quoted verbatim +
    community verification table), backend-matrix row, porting-inventory §9.14
    (ADDITIVE deviation: upstream vLLM allocates via torch and has no analog —
    rocm.py:75-77, 909-910 only name the APUs), STATUS/BENCHMARKS/FEATURES
    rows, state + NOW.

Gates run here (CPU only)

  • scripts/agent-preflight.sh EXIT=0 (all record gates + mutation suites).
  • CPU configure + -Werror object-compile of the platform and test TUs
    (vllm_rocm_platform_syntax_check): clean; compile-mutation of the new
    probe API went red (2 errors) and green on restore.
  • The .hip files remain excluded from non-HIP builds — byte-identical CPU
    builds.

Evidence the community owes (PENDING — the M0/M1 table shape from #41)

Gate Board Report
Configure with NO manual flags gfx1151, gfx1103 (Arch/TheRock) configure log
.hip compile of this delta any first error text, or clean
ctest -R 'rocm|cross_device' incl. the 2 new cases all four boards pass/fail + the printed integrated/managed-alloc/UnifiedMemory triple
Discrete branch provably dead gfx1100, gfx1201 triple = 0/0/false
M2: small dense model, greedy parity vs --device cpu gfx1151, gfx1103 tokens + VT_OP_PROVIDER_STATS=1 fallback list

Sequencing note: composes with #140 (discrete gfx1201 lane) — this PR touches
the integrated-path branch only; will rebase onto main if #140 lands first.

Closes the F6 blocker on #41 (task #286).

mudler added a commit that referenced this pull request Aug 8, 2026
…egrated APUs

Implements the maintainer-ratified approach (b) from issue #41 F6. On a
device probing hipDeviceAttributeIntegrated=1 + ManagedMemory=1 +
ConcurrentManagedAccess=1, RocmBackend::Alloc allocates through
hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory() returns true
exactly then: host access to every Backend::Alloc block becomes
API-guaranteed rather than architecturally incidental, which is what the
CPU reference tier's host-dereference contract needs on XNACK-less RDNA3
APUs (gfx1151/gfx1103 measure PageableMemoryAccess=0, vetoing the W0
CUDA-shaped probe even though the aliasing demonstrably holds there).

- The W0 conjunction (integrated AND pageable) stays as ground 1;
  Free stays hipFree, API-documented for both alloc paths; the inherited
  AllocPinned delegates to Alloc (backend.cpp:19) and rides the branch,
  coherent with its unified-memory contract (backend.h:76-78).
- Discrete devices byte-identical to W0: the branch is provably dead at
  Integrated=0, asserted by a new runtime test through the new HIP-free
  probes ManagedAllocActive/IntegratedDevice (rocm_runtime.h).
- F6's decisive experiment is now a standing test: host-write inputs with
  no Copy, native RmsNorm kernel reads them, host-read the kernel-written
  output with no Copy.
- CMake F1/F3 absorption: when ROCM_PATH exists, derive
  CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS and
  export ROCM_PATH, each only when unset, before check_language(HIP) —
  Arch/TheRock layouts configure flag-free (F2 was downstream of the
  unidentified compiler; no separate change).
- No upstream analog to mirror (vLLM allocates via torch; rocm.py:75-77,
  909-910 only name the APUs): recorded as ADDITIVE deviation,
  porting-inventory §9.14. Spec: .agents/specs/rocm-unified-memory-b.md
  (decision quoted verbatim + the community verification table).

Lands via row/ROCM-UNIFIED-MEMORY-B (PR #144).

BLIND-WRITTEN: no AMD GPU or hipcc on the authoring machine; the .hip
delta has never been compiled. CPU gates green (preflight EXIT=0; -Werror
object-compile of the platform + test TUs; compile-mutation of the new
probe API red then green on restore). Community evidence owed per the
spec table; PR #144 carries the ask.

FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
@localai-bot
localai-bot marked this pull request as ready for review August 8, 2026 07:49
@localai-bot localai-bot mentioned this pull request Aug 8, 2026
6 tasks
mudler added a commit that referenced this pull request Aug 8, 2026
…egrated APUs

Implements the maintainer-ratified approach (b) from issue #41 F6. On a
device probing hipDeviceAttributeIntegrated=1 + ManagedMemory=1 +
ConcurrentManagedAccess=1, RocmBackend::Alloc allocates through
hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory() returns true
exactly then: host access to every Backend::Alloc block becomes
API-guaranteed rather than architecturally incidental, which is what the
CPU reference tier's host-dereference contract needs on XNACK-less RDNA3
APUs (gfx1151/gfx1103 measure PageableMemoryAccess=0, vetoing the W0
CUDA-shaped probe even though the aliasing demonstrably holds there).

- The W0 conjunction (integrated AND pageable) stays as ground 1;
  Free stays hipFree, API-documented for both alloc paths; the inherited
  AllocPinned delegates to Alloc (backend.cpp:19) and rides the branch,
  coherent with its unified-memory contract (backend.h:76-78).
- Discrete devices byte-identical to W0: the branch is provably dead at
  Integrated=0, asserted by a new runtime test through the new HIP-free
  probes ManagedAllocActive/IntegratedDevice (rocm_runtime.h).
- F6's decisive experiment is now a standing test: host-write inputs with
  no Copy, native RmsNorm kernel reads them, host-read the kernel-written
  output with no Copy.
- CMake F1/F3 absorption: when ROCM_PATH exists, derive
  CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS and
  export ROCM_PATH, each only when unset, before check_language(HIP) —
  Arch/TheRock layouts configure flag-free (F2 was downstream of the
  unidentified compiler; no separate change).
- No upstream analog to mirror (vLLM allocates via torch; rocm.py:75-77,
  909-910 only name the APUs): recorded as ADDITIVE deviation,
  porting-inventory §9.14. Spec: .agents/specs/rocm-unified-memory-b.md
  (decision quoted verbatim + the community verification table).

Lands via row/ROCM-UNIFIED-MEMORY-B (PR #144).

BLIND-WRITTEN: no AMD GPU or hipcc on the authoring machine; the .hip
delta has never been compiled. CPU gates green (preflight EXIT=0; -Werror
object-compile of the platform + test TUs; compile-mutation of the new
probe API red then green on restore). Community evidence owed per the
spec table; PR #144 carries the ask.

FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
@mudler
mudler force-pushed the row/ROCM-UNIFIED-MEMORY-B branch from c0b3e8f to 9abd304 Compare August 8, 2026 07:54
mudler added 2 commits August 8, 2026 08:23
…emory by construction

Issue #41's F6 fork is decided: on integrated managed-capable devices the
backend will allocate through hipMallocManaged so host access is
API-guaranteed, and UnifiedMemory() returns true exactly then (maintainer
comment, 2026-08-08, quoted in docs/ROCM.md §3.1). This commit is the
contributor-facing statement: the decision and its behavior table
(integrated vs discrete), the community-verified W0 state (M0/M1 MET on
gfx1151/gfx1103/gfx1100/gfx1201), the two teardown known-issues, and the
exact post-fix sequence for board owners (§5.2). The implementation follows
in the next commit.

FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
…egrated APUs

Implements the maintainer-ratified approach (b) from issue #41 F6. On a
device probing hipDeviceAttributeIntegrated=1 + ManagedMemory=1 +
ConcurrentManagedAccess=1, RocmBackend::Alloc allocates through
hipMallocManaged(hipMemAttachGlobal) and UnifiedMemory() returns true
exactly then: host access to every Backend::Alloc block becomes
API-guaranteed rather than architecturally incidental, which is what the
CPU reference tier's host-dereference contract needs on XNACK-less RDNA3
APUs (gfx1151/gfx1103 measure PageableMemoryAccess=0, vetoing the W0
CUDA-shaped probe even though the aliasing demonstrably holds there).

- The W0 conjunction (integrated AND pageable) stays as ground 1;
  Free stays hipFree, API-documented for both alloc paths; the inherited
  AllocPinned delegates to Alloc (backend.cpp:19) and rides the branch,
  coherent with its unified-memory contract (backend.h:76-78).
- Discrete devices byte-identical to W0: the branch is provably dead at
  Integrated=0, asserted by a new runtime test through the new HIP-free
  probes ManagedAllocActive/IntegratedDevice (rocm_runtime.h).
- F6's decisive experiment is now a standing test: host-write inputs with
  no Copy, native RmsNorm kernel reads them, host-read the kernel-written
  output with no Copy.
- CMake F1/F3 absorption: when ROCM_PATH exists, derive
  CMAKE_HIP_COMPILER_ROCM_ROOT, seed --rocm-path into CMAKE_HIP_FLAGS and
  export ROCM_PATH, each only when unset, before check_language(HIP) —
  Arch/TheRock layouts configure flag-free (F2 was downstream of the
  unidentified compiler; no separate change).
- No upstream analog to mirror (vLLM allocates via torch; rocm.py:75-77,
  909-910 only name the APUs): recorded as ADDITIVE deviation,
  porting-inventory §9.14. Spec: .agents/specs/rocm-unified-memory-b.md
  (decision quoted verbatim + the community verification table).

Lands via row/ROCM-UNIFIED-MEMORY-B (PR #144).

BLIND-WRITTEN: no AMD GPU or hipcc on the authoring machine; the .hip
delta has never been compiled. CPU gates green (preflight EXIT=0; -Werror
object-compile of the platform + test TUs; compile-mutation of the new
probe API red then green on restore). Community evidence owed per the
spec table; PR #144 carries the ask.

FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
@mudler
mudler force-pushed the row/ROCM-UNIFIED-MEMORY-B branch from 9abd304 to 5c8969f Compare August 8, 2026 08:26
mudler added a commit that referenced this pull request Aug 8, 2026
…proach (b) from issue #41 + Arch/TheRock CMake absorption (#144)

Implements the ratified maintainer decision on issue #41 F6:
RocmBackend::Alloc branches to hipMallocManaged(hipMemAttachGlobal)
when Integrated && ManagedMemory && ConcurrentManagedAccess, and
UnifiedMemory() returns true exactly then - host access becomes
API-guaranteed rather than architecturally incidental, so the
zero-kernel CPU-reference tier installs and M2 (model e2e) unblocks
on the community boards (Strix Halo gfx1151, 780M gfx1103). Discrete
devices provably unchanged (branch dead at Integrated=0, asserted by
a new runtime-gated test; F6's kernel-write->host-read-no-copy
experiment is now a standing gate). hipFree serves both alloc paths
(no hipFreeManaged exists; stated in-code as a falsifiable
assumption). CMake absorbs the F1/F3 Arch/TheRock layout hints
(ROCM_PATH-derived, only-when-unset, non-ROCm machines
byte-identical); F2 documented as environment-side. Upstream has no
analog (vLLM allocates via torch) - recorded as an additive deviation
in porting-inventory 9.14, with the ROCM_PATH handling mirrored from
vllm/CMakeLists.txt:54-60 at the pin.

WRITTEN BLIND per this lane's policy (no AMD hardware here): all
HIP-side evidence is PENDING-community, itemized in the spec + PR;
the handoff comment on issue #41 gives the three board owners the
branch, the now-flag-free configure, and the exact report format.
Operator landing basis: PR CI 12/12 green on this head (full CPU
build + both sanitizers), local -Werror syntax-check target green,
nine record gates green; a mutation-review round was deliberately
skipped as disproportionate for a blind-HIP lane whose real gate is
community hardware (the skeleton precedent) - disclosed here.

FOLLOWING_AGENTS_PROTOCOL
Assisted-by: Claude Code:claude-fable-5 [ClaudeCode]
@localai-bot

Copy link
Copy Markdown
Collaborator Author

Landed on main as e9f3561 (mudler-authored squash, lease-guarded push). Basis: PR CI 12/12 green on the head (full CPU build + both sanitizers), local -Werror syntax-check target green, nine record gates green; the mutation-review round was deliberately skipped as disproportionate for a blind-HIP lane whose real gate is community hardware (disclosed in the commit). @jimmykarily @arch-btw @tbrasser — approach (b) is now on main: pull main (or keep testing the branch, identical content), run the flag-free configure + ctest -R 'rocm|cross_device' + the M2 attempt per the issue-41 handoff comment, and post the table. A compile error remains the most useful thing you can post.

@localai-bot localai-bot closed this Aug 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants