Skip to content

common : default GPU_MAX_HW_QUEUES=1 on HIP to fix idle GPU load - #24237

Closed
liminfei-amd wants to merge 1 commit into
ggml-org:masterfrom
liminfei-amd:amd-rocm/23965-hip-queue-default
Closed

common : default GPU_MAX_HW_QUEUES=1 on HIP to fix idle GPU load#24237
liminfei-amd wants to merge 1 commit into
ggml-org:masterfrom
liminfei-amd:amd-rocm/23965-hip-queue-default

Conversation

@liminfei-amd

@liminfei-amd liminfei-amd commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Overview

On HIP/ROCm, llama-server with --mmproj can leave the GPU pinned at high utilization and
maximum core clock while all slots are idle. This is a known ROCm/amdgpu MES multi-queue behavior
(see #3929, #21430) and is reported for the mmproj idle case in #23965. Limiting the number of HW
queues to 1 avoids the idle busy-loop.

In common_init(), for HIP builds only, default GPU_MAX_HW_QUEUES to 1 when the user has
not already set it. An explicitly-set value is always respected; non-HIP builds are untouched:

#if defined(GGML_USE_HIP)
    if (!std::getenv("GPU_MAX_HW_QUEUES")) {
        setenv("GPU_MAX_HW_QUEUES", "1", 0);
    }
#endif

Fixes #23965.

Additional information

Validated on real AMD ROCm hardware:

  • Hardware: Radeon RX 9070 (gfx1201, RDNA4) · ROCm 7.2.1
  • Before: with --mmproj, an idle server holds the core clock at ~3.2 GHz.
  • After: with the default in place, an idle server settles to its low idle clock for the large
    majority of samples (a single transient boost spike is normal).

Note on the approach: this sets an environment default rather than changing mmproj logic. If
maintainers prefer documenting GPU_MAX_HW_QUEUES=1 for ROCm consumer GPUs instead of a code
default, I'm glad to switch to a docs-only change.

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES — AI was used in an assistive capacity only (locating the relevant
    code paths and drafting wording). I authored and reviewed the change, understand it fully, and can
    explain every line; the fix and the on-hardware validation are mine. No AI-written code was
    submitted without manual review.

On HIP/ROCm, llama-server with --mmproj can leave the GPU pinned at high
utilization and maximum clocks while all slots are idle, due to a
ROCm/amdgpu MES multi-queue behavior (see ggml-org#3929, ggml-org#21430, ggml-org#23965). Limiting
the number of hardware queues to 1 avoids the idle busy-loop.

Default GPU_MAX_HW_QUEUES to 1 in common_init() for HIP builds when the
user has not set it; an explicit value is always respected. Non-HIP builds
are unaffected.

Fixes ggml-org#23965

Signed-off-by: liminfei-amd <91481003+liminfei-amd@users.noreply.github.com>
@liminfei-amd

Copy link
Copy Markdown
Contributor Author

Friendly ping on this one.

The original reporter (@zzgundam) has since tested this change and confirmed it resolves the issue: they closed #23965 as completed, noting that "#24237 just fixed this problem".

The change is a small, HIP-only default (GPU_MAX_HW_QUEUES=1, set only when the user hasn't already set it), validated on a Radeon RX 9070 (gfx1201, ROCm 7.2.1). Would a maintainer be able to take a look when convenient? As noted above, if documenting GPU_MAX_HW_QUEUES=1 for ROCm consumer GPUs is preferred over a built-in default, I'm happy to switch to a docs-only change instead.

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: llama.cpp consistently fully utilizes the GPU with the --mmproj mmproj-BF16.ggufoption.

1 participant