kleidiai: Add runtime feature detection mechanism for aarch64/kleidiai by JonathanC-ARM · Pull Request #26076 · ggml-org/llama.cpp

JonathanC-ARM · 2026-07-24T11:07:10Z

Overview

This PR improves upon a runtime feature detection mechanism located in ggml/src/ggml-cpu/arch/arm/cpu-feats.cpp
This mechanism has been centralized into a new header file which can be used by aarch64 systems to determine which processor features are available at runtime.

ggml/src/ggml-cpu/kleidiai/kleidiai.cpp has been reworked to use this feature detection as part of its kernel selection.
Additionally improvements have been made to SME unit detection.

Additional information

This pr is intended to be 1/3 in a series of pr's to improve the KleidiAI with its cpu feature detection, build system and documentation.

PR 2 - #26077
PR 3 - #26078

Requirements

I have read and agree with the contributing guidelines - YES
AI usage disclosure: YES - I used AI to learn about llama.cpp and to understand the codebase

Testing

I've tested this series of patches combined on several devices to ensure there are no performance regressions tested matched parent/patch builds across the following devices:

Apple M4 Pro
AWS Graviton 3
AWS Graviton 4
OPPO Find X9 Pro
Google Pixel 9 Pro

For the SME-capable devices:

Apple M4 Pro: the apple_m4 backend was selected with a score of 83.
SME2 KleidiAI kernels were selected for Q4, Q8 and F32, and the runtime SME thread cap was detected as 2.

OPPO Find X9 Pro: the ARMv9.2 SME2 backend was selected with a score of 127.
SME2 KleidiAI kernels were selected for Q4, Q8 and F32, and the runtime SME thread cap was detected as 1.

I also validated the unsupported-feature paths:

Graviton 3: selected ARMv8.6/SVE; SVE2 and SME variants scored zero and were rejected.
Graviton 4: selected ARMv8.6/SVE2; SME variants scored zero and were rejected.
Pixel 9 Pro: selected ARMv9.0/I8MM; both SME variants scored zero and were rejected.

I ran CPU-only performance comparisons using matched Release builds and Qwen3 0.6B Q4_0 and Q8_0 models. Representative patch deltas were:

M4 Pro: approximately +0.8% to +1.6%, with Q8 generation noisy but positive
Graviton 3: approximately -1.17% to +0.02%
Graviton 4: approximately -0.02% to +1.62%
OPPO Find X9 Pro: approximately -0.8% to +6.2%
Pixel 9 Pro: approximately -4.4% to +0.5%

The Android results were particularly sensitive to thermal throttling and run order. After mixing the runs the runs and repeating them after cooldown periods, I could not reproduce a performance change attributable to the new detection or thread-cap logic. Overall, the results were within device/run noise, and no reproducible regression was found.

One limitation is that the Linux systems available for this testing do not expose SME, so they validate the Linux unsupported-feature and rejection paths rather than the positive Linux NSMC path. The positive SME-capacity results currently come from the M4 Pro and OPPO Find X9 Pro.

I tested seperately also, other Devices like M5 Series and Neo Macbooks to for SMCU detection updates, ensuring that the values reported are those we know to be correct based on testing.

JonathanC-ARM · 2026-07-24T11:34:51Z

Just said I'd comment here also I opened 3 pr's as a stack to give context to the work here in this pr, a bot has commented on the other two stating that its against the contributing guidelines which is correct as I am a new contributor, however they are draft and the intent is to give context to the intent of this work and subsequent work in this area. Originally all 3 were a single PR however for the sake of review it was decided to split into 3. I can close them if required but I felt it necessary to open and link at least initially.

taronaeo · 2026-07-25T01:21:13Z

@ggml-gh-bot review

ggml-gh-bot · 2026-07-25T04:59:49Z

Automated code review

I have enough to write the review.

Static review of PR #26076 - runtime feature detection for aarch64/kleidiai

Running the General, ggml/backend, Security, and Approach and design checklists (the diff adds a new shared runtime-detection header and rewrites SMCU detection, all under ggml/src).

Blocking

Wrong HWCAP2 fall-back bit for HWCAP2_SME2 - ggml/src/ggml-aarch64.h:26
#define HWCAP2_SME2 (1ULL << 37). The Linux uapi value for HWCAP2_SME2 is bit 24 (HWCAP2_SME is bit 23); bit 37 is HWCAP2_LSE128. This fall-back only activates on a sysroot whose <asm/hwcap.h> does not define HWCAP2_SME2 (older toolchain running on a newer kernel). On such a build, runtime_feat.has_sme2 = !!(hwcap2 & (1ULL<<37)) reads the LSE128 bit instead. On a CPU with LSE128 but no SME2, this reports SME2 present, which later sets CPU_FEATURE_SME2 and can select an SME2 kernel, resulting in SIGILL. Fix to (1ULL << 24) (and double-check the other fall-backs against arch/arm64/include/uapi/asm/hwcap.h). This bug was inherited from the old kleidiai.cpp #define HWCAP2_SME2 (1UL << 37), but promoting it into a shared header makes it worth fixing now.
SMIDR_EL1.NSMC interpretation looks wrong - ggml/src/ggml-cpu/kleidiai/kleidiai.cpp:148-175
Per the Arm ARM, SMIDR_EL1.NSMC (bits [63:56]) is "number of SMCUs implemented" (system/domain count), not "number of PEs sharing this SMCU". The new code computes shared_count = (nsmc==0xF) ? 1 : nsmc+1 and then adds that value for each distinct shared id: total = num_private + sum(shared_count_per_id) (kleidiai.cpp:213-218). On a part with several distinct shared SMCUs, this multiplies a global count across each id and can wildly overcount SMCUs, inflating sme_thread_cap. The old code used shared_ids.size() (one SMCU per distinct shared id).
Either (a) revert to counting one SMCU per distinct shared id (matching the old behaviour), or (b) cite the exact Arm ARM wording for NSMC and SH that justifies nsmc+1 as the per-SMCU multiplicity, and add that citation to the comment at kleidiai.cpp:149-151. As written a reviewer cannot confirm it.

Will slow the review

Reuse vs new infrastructure - ggml/src/ggml-aarch64.h (new file)
The codebase already has runtime aarch64 capability state in ggml/src/ggml-cpu/ggml-cpu.c (ggml_arm_arch_features, ggml_cpu_has_*, ggml_cpu_get_sve_cnt). It currently only covers SVE length and the __ARM_FEATURE_* compile-time checks. Before landing a parallel, self-contained runtime detector, consider populating ggml_arm_arch_features with the runtime fields (dotprod/fp16/sve/sve2/i8mm/sme/sme2/sve_cnt) once and having both kleidiai.cpp and cpu-feats.cpp read from it, so there is a single source of truth for aarch64 capability detection. If the feats target must stay self-contained to avoid arch-instruction leakage (the LTO comment in CMakeLists.txt:1-13), say so in the header comment so the design choice is explicit. Also: placement at ggml/src/ggml-aarch64.h (top of the ggml tree) is unusual for an arch-private helper; ggml/src/ggml-cpu/arch/arm/ would be a more natural home.
has_sme2 flip changes aarch64 backend-variant scoring - ggml/src/ggml-cpu/arch/arm/cpu-feats.cpp:14-33
The score now gates on af.has_sve / af.has_sve2 (runtime) for GGML_USE_SVE/GGML_USE_SVE2. Previously feats did not check SVE/SVE2 at all, so an SVE/SVE2-built variant could win even on a CPU that disables SVE at runtime (e.g. via prctl PR_SVE_SET_VL_OFF). The new behaviour is safer (avoids selecting an SVE-built backend that would SIGILL), but it is a behaviour change to backend selection - call it out in the commit/PR description and ideally exercise it (e.g. run with SVE disabled and confirm the SVE variant scores out).
No tests / perf data for the new detection paths
The SME thread cap is a performance heuristic with three platform branches (Linux sysfs, Apple sysctl perf-levels, Windows unknown). There are no tests and no calibration numbers. Per the ggml/backend checklist, new capability-detection that drives scheduling is expected to come with concrete data on real hardware (the detected SMCU count vs. the actual SME throughput, and the resulting thread cap). At minimum, add the model/SoC the heuristic was calibrated on and the counts it produced, so reviewers can sanity-check the Apple performance/super name matching (kleidiai.cpp:136-146) and the Linux NSMC handling.
Apple SMCU estimate is L2-cluster-count, not per-core - ggml/src/ggml-cpu/kleidiai.cpp:242-258
units += ceil(physical_cpus / cpus_per_l2) counts performance L2 clusters, but FEAT_SME/SME2 are per-core. If the intent is "conservative lower bound for the SME thread cap", say so explicitly and explain why cluster count is the right ceiling (it leaves SME threads on the table on parts where each P-core has its own SME unit). The previous hard-coded M4 table used per-SiC counts that do not obviously match this formula, so the replacement changes Apple results for already-listed chips; confirm on M4/M4 Pro/Max that the new generic formula returns the same values the old table did.

Nits

ggml/src/ggml-aarch64.h is C++ only (uses static_cast<bool>, nullptr) but uses the C typedef struct {...} _t; idiom and a .h extension. Pick one convention; if it stays .h/C-style, note that it is C++-only, or rename to .hpp.
add_smcus_from_smidr (kleidiai.cpp:148) is defined unconditionally but only referenced in the #if defined(__linux__) && defined(__aarch64__) branch; on Apple/Windows builds it is an unused static. It does not error today (-Wno-unused-function is set, ggml/src/CMakeLists.txt:39), but guarding it with the same #if as parse_cpu_dir_name/detect_cpu_ids would be cleaner.
ggml/src/ggml-cpu/kleidiai/kleidiai.cpp:265 Windows branch comment uses the misspelling "diagnostics/debug override" wording is fine, but the comment block at kleidiai.cpp:266-268 ("No verified Windows ... Return unknown and use ...") repeats information also stated elsewhere; keep it concise.
(int)detected_smcus (kleidiai.cpp:336) and (int)v from parse_uint_env then sme_cores - detected_smcus is size_t; even though values are bounded in practice, clamp to INT_MAX for clarity since the cap flows into thread counts and GGML_LOG_INFO %d.
runtime_feat.sve_cnt == QK8_0 (kleidiai.cpp:312) depends on QK8_0 from ggml-common.h; the include order happens to work, but the comparison is clearer as runtime_feat.has_sve && runtime_feat.sve_cnt == QK8_0 to express the original intent explicitly.
Copyright line "Copyright 2026 Arm Limited" (ggml-aarch64.h:1) uses a bare year while the neighbouring Arm files use "Copyright 2025-2026 Arm Limited and/or its affiliates"; align for consistency.

This review was generated automatically by pi coding agent using zai-org/GLM-5.2. It may contain mistakes. Maintainers make the final call.

taronaeo

Prefer sized integer types where possible

cc: @chaxu01

taronaeo · 2026-07-25T06:54:49Z

+    errno = 0;
+    const unsigned long long value = strtoull(name + 3, &end, 10);
+    if (errno != 0 || *end != '\0' ||
+        value > (unsigned long long) std::numeric_limits<size_t>::max()) {


Suggested change

value > (unsigned long long) std::numeric_limits<size_t>::max()) {

value > (size_t) std::numeric_limits<size_t>::max()) {

I rewrote the function altogether into more modern c++ which removes the need for the unsigned types also.

taronaeo · 2026-07-25T07:02:57Z

Btw can you see if the review from the bot makes sense? #26076 (comment)

chaxu01 · 2026-07-27T12:05:28Z

+    // conservative policy and only treat zero affinity as private.
+    const uint32_t sh = (uint32_t)((smidr >> 13) & 0x3);
+    const uint32_t nsmc = (uint32_t)((smidr >> 56) & 0xF);
+    const size_t shared_count = nsmc == 0xF ? 1 : (size_t)nsmc + 1;


SMIDR_EL1.NSMC == 0b1111 is a reserved encoding. Mapping it to one shared SMCU seems unsafe, as it treats an invalid value as valid. Could we return here instead?

So while its true that if we had an nsmc value of 0xF its a reserved value and something that I wouldn't expect so see (currently) the reason we assume 1 is that the there is no valid encoding for 0 smcu's. So if SH and affinity can reliably identify a shared group then it must indicate that the group contains a minimum of 1.

Returning if we seen this value, might cause us to incorrectly assume there are no SMCU and not leverage the absolute minimum number. I think maybe a log message might be appropriate if we encounter this instead of a return? What do you think @chaxu01

That sounds reasonable. A one-time warning (or documenting it as "unknown count, assuming a minimum of one shared SMCU") would address that for me.

Added the log warning in the latest push @chaxu01

JonathanC-ARM · 2026-07-27T16:20:21Z

Wrong HWCAP2 fall-back bit for HWCAP2_SME2 - ggml/src/ggml-aarch64.h:26
#define HWCAP2_SME2 (1ULL << 37). The Linux uapi value for HWCAP2_SME2 is bit 24 (HWCAP2_SME is bit 23); bit 37 is HWCAP2_LSE128. This fall-back only activates on a sysroot whose <asm/hwcap.h> does not define HWCAP2_SME2 (older toolchain running on a newer kernel). On such a build, runtime_feat.has_sme2 = !!(hwcap2 & (1ULL<<37)) reads the LSE128 bit instead. On a CPU with LSE128 but no SME2, this reports SME2 present, which later sets CPU_FEATURE_SME2 and can select an SME2 kernel, resulting in SIGILL. Fix to (1ULL << 24) (and double-check the other fall-backs against arch/arm64/include/uapi/asm/hwcap.h). This bug was inherited from the old kleidiai.cpp #define HWCAP2_SME2 (1UL << 37), but promoting it into a shared header makes it worth fixing now.

This is incorrect, the correct bits are used for SME2 and other features in this code, e.g. https://github.com/torvalds/linux/blob/master/arch/arm64/include/uapi/asm/hwcap.h#L115

JonathanC-ARM · 2026-07-27T16:23:05Z

SMIDR_EL1.NSMC interpretation looks wrong - ggml/src/ggml-cpu/kleidiai/kleidiai.cpp:148-175
Per the Arm ARM, SMIDR_EL1.NSMC (bits [63:56]) is "number of SMCUs implemented" (system/domain count), not "number of PEs sharing this SMCU". The new code computes shared_count = (nsmc==0xF) ? 1 : nsmc+1 and then adds that value for each distinct shared id: total = num_private + sum(shared_count_per_id) (kleidiai.cpp:213-218). On a part with several distinct shared SMCUs, this multiplies a global count across each id and can wildly overcount SMCUs, inflating sme_thread_cap. The old code used shared_ids.size() (one SMCU per distinct shared id).
Either (a) revert to counting one SMCU per distinct shared id (matching the old behaviour), or (b) cite the exact Arm ARM wording for NSMC and SH that justifies nsmc+1 as the per-SMCU multiplicity, and add that citation to the comment at kleidiai.cpp:149-151. As written a reviewer cannot confirm it.

This is also incorrect, The defines for NSMC as bits [59:56], not [63:56], and as the number of SMCUs minus one associated with a particular {Affinity2, Affinity} group. It is neither a global system count nor a PE count.
Arm SMIDR_EL1 documentation (https://developer.arm.com/documentation/ddi0601/2025-12/AArch64-Registers/SMIDR-EL1--Streaming-Mode-Identification-Register)

Our code correctly:

Stores NSMC + 1 once per affinity ID at ggml/src/ggml-cpu/kleidiai/kleidiai.cpp:163.
Deduplicates repeated PE reports using the map.
Sums each distinct SMCU group once.

For example, eight PEs reporting affinity A with NSMC=3 produce four SMCUs, not 32

JonathanC-ARM · 2026-07-27T16:25:23Z

Reuse vs new infrastructure - ggml/src/ggml-aarch64.h (new file)
The codebase already has runtime aarch64 capability state in ggml/src/ggml-cpu/ggml-cpu.c (ggml_arm_arch_features, ggml_cpu_has_, ggml_cpu_get_sve_cnt). It currently only covers SVE length and the _ARM_FEATURE compile-time checks. Before landing a parallel, self-contained runtime detector, consider populating ggml_arm_arch_features with the runtime fields (dotprod/fp16/sve/sve2/i8mm/sme/sme2/sve_cnt) once and having both kleidiai.cpp and cpu-feats.cpp read from it, so there is a single source of truth for aarch64 capability detection. If the feats target must stay self-contained to avoid arch-instruction leakage (the LTO comment in CMakeLists.txt:1-13), say so in the header comment so the design choice is explicit. Also: placement at ggml/src/ggml-aarch64.h (top of the ggml tree) is unusual for an arch-private helper; ggml/src/ggml-cpu/arch/arm/ would be a more natural home.

So this code was moved from cpu_feats.cpp file (and added to). The existing ggml_cpu_has code, while used in places as a runtime feature detect are all entirely compile time detection, so I in theory if using ggml_cpu_has_sme2 function could compile with a compiler that emits the correct sme define but then take that binary and run it on a non sme device causing a segfault. The intent here was to reuse and centralize something that was already in the code base and which enables us to do true runtime feature detection. I would be open to moving this to a more arm specific place, but it does feel like a useful and meaningful extension to existing code.

JonathanC-ARM · 2026-07-27T16:30:08Z

No tests / perf data for the new detection paths
The SME thread cap is a performance heuristic with three platform branches (Linux sysfs, Apple sysctl perf-levels, Windows unknown). There are no tests and no calibration numbers. Per the ggml/backend checklist, new capability-detection that drives scheduling is expected to come with concrete data on real hardware (the detected SMCU count vs. the actual SME throughput, and the resulting thread cap). At minimum, add the model/SoC the heuristic was calibrated on and the counts it produced, so reviewers can sanity-check the Apple performance/super name matching (kleidiai.cpp:136-146) and the Linux NSMC handling.

Updated PR description with information about the testing that was done.

JonathanC-ARM · 2026-07-27T16:31:09Z

Apple SMCU estimate is L2-cluster-count, not per-core - ggml/src/ggml-cpu/kleidiai.cpp:242-258 units += ceil(physical_cpus / cpus_per_l2) counts performance L2 clusters, but FEAT_SME/SME2 are per-core. If the intent is "conservative lower bound for the SME thread cap", say so explicitly and explain why cluster count is the right ceiling (it leaves SME threads on the table on parts where each P-core has its own SME unit). The previous hard-coded M4 table used per-SiC counts that do not obviously match this formula, so the replacement changes Apple results for already-listed chips; confirm on M4/M4 Pro/Max that the new generic formula returns the same values the old table did.

So FEAT_SME/SME2 is defined per core, but it is implemented via a shared co-processor within silicon, the number of units depends on the hardware and this smcu estimate is trying to move away from a maintained list of machines and values to something that is a bit less maintance. I tested this code on M4, M4 Pro, M5, M5 Pro, M5 Max and Macbook Neo. All of which returned the expected number of SMCU's (1,2,1,3,3,1) respectively.

JonathanC-ARM · 2026-07-27T16:39:13Z

has_sme2 flip changes aarch64 backend-variant scoring - ggml/src/ggml-cpu/arch/arm/cpu-feats.cpp:14-33
The score now gates on af.has_sve / af.has_sve2 (runtime) for GGML_USE_SVE/GGML_USE_SVE2. Previously feats did not check SVE/SVE2 at all, so an SVE/SVE2-built variant could win even on a CPU that disables SVE at runtime (e.g. via prctl PR_SVE_SET_VL_OFF). The new behaviour is safer (avoids selecting an SVE-built backend that would SIGILL), but it is a behaviour change to backend selection - call it out in the commit/PR description and ideally exercise it (e.g. run with SVE disabled and confirm the SVE variant scores out).

This change doesn't impact the scoring mechanism it only changes how we construct the data type from which we get the runtime information to generate a score but the behavior and gating are the same.

JonathanC-ARM · 2026-07-28T09:55:22Z

Hi @taronaeo I left comments yesterday evening addressing the AI reviewer comments. Thanks for taking time to look at the PR.

taronaeo · 2026-07-28T09:57:36Z

Hi @taronaeo I left comments yesterday evening addressing the AI reviewer comments. Thanks for taking time to look at the PR.

Yep I saw it, thanks for responding to the bot comments. I don't know this area too well so I'll be leaving it to @chaxu01 and you. Feel free to ping me for a general review and/or approval once the PR is finalized :)

chaxu01 · 2026-07-30T08:11:06Z

@ggerganov could you please approve the workflow runs? Thanks.

bill88t · 2026-07-31T00:53:52Z

Merged PR1 and PR2 in my local copy and built it on an ARMv9 Orion O6. It works alright. I don't see any noticeable changes or issues.

Complied with -DGGML_CPU_KLEIDIAI=ON but I also have a GPU which does the bulk of everything. I'll test more on a secondary Orion O6N (that won't be getting a GPU) in a few days.

ggerganov · 2026-07-31T06:47:45Z

+// SPDX-FileCopyrightText: Copyright 2026 Arm Limited and/or its affiliates <open-source-office@arm.com>
+// SPDX-License-Identifier: MIT
+//


This is not needed - remove

ggerganov · 2026-07-31T06:48:45Z

 #include <assert.h>
 #include <stdio.h>
+#include <stdlib.h>


Suggested change

#include <assert.h>

#include <stdio.h>

#include <stdlib.h>

#include <cassert>

#include <cstdio>

#include <cstdlib>

Applied the changes suggested

taronaeo · 2026-08-08T09:46:04Z

Looks like there is a conflict. Can you fix that?

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

JonathanC-ARM · 2026-08-10T11:10:43Z

Looks like there is a conflict. Can you fix that?

Hi @taronaeo, I've rebased and resolved conflicts (moved some new additions in cpu-feats, to the centralized file)

ggerganov

A couple of naming nits for consistency.

ggerganov · 2026-08-10T12:11:00Z

+
+#endif
+
+typedef struct ggml_aarch64_runtime_features {


Suggested change

typedef struct ggml_aarch64_runtime_features {

typedef struct ggml_feats_arch64_runtime {

Updated every usage of this as suggested

ggerganov · 2026-08-10T12:11:34Z

+    int sve_cnt;
+} ggml_aarch64_runtime_features_t;
+
+static inline ggml_aarch64_runtime_features_t ggml_get_aarch64_runtime_features(void) {


Suggested change

static inline ggml_aarch64_runtime_features_t ggml_get_aarch64_runtime_features(void) {

static inline ggml_feats_arch64_runtime_t ggml_feats_get_arch64_runtime(void) {

Agreed, I've renamed as suggested.

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

JonathanC-ARM · 2026-08-10T20:50:47Z

Thanks for all the review comments @taronaeo @ggerganov much appreciated!

taronaeo · 2026-08-11T13:56:00Z

Merging in a few hours if there are no further changes.

JonathanC-ARM · 2026-08-11T14:55:45Z

Merging in a few hours if there are no further changes.

Thanks @taronaeo

Signed-off-by: Gabe Goodhart <ghart@us.ibm.com> * origin/master: (383 commits) cmake : introduce semantic versioning (ggml-org#26839) gguf : harden loader against malformed tensor dims and metadata types (ggml-org#25596) kleidiai: Add runtime feature detection mechanism for aarch64/kleidiai (ggml-org#26076) model : disallow integer dflash sliding_window_pattern (ggml-org#26900) sync : ggml cmake : add config version support (ggml/1582) server : support slot save/restore with media inputs (ggml-org#26640) ui: add read_media tool (ggml-org#25877) opencl: default FA c8 cluster width to 16 on X1E (ggml-org#26433) tests : update speculative params (ggml-org#26925) vulkan: add TQ2_0 (ternary) support (ggml-org#25850) wavtokenizer-dec : bound posnet/convnext block_count against n_layer_all (ggml-org#26892) convert : handle per_layer_config in Gemma4 (transformers 5.15) (ggml-org#26882) opencl: use flat mv q5_k when weight exceeds image1d_buffer_t limit (ggml-org#26880) chat : fix muse-glimmer detection of tool calls after EOM (ggml-org#26879) ci : add missing release check (ggml-org#26923) CUDA: only disable CUDA graphs when mul_mat_id actually needs a stream sync (ggml-org#26802) cuda : add warp-per-row wkv7 kernel for single-token decode (ggml-org#26111) spec : update speculative-simple (ggml-org#26904) chat : tighten bare function parsing for Qwen models (ggml-org#26793) ...

ggml-org#26076) * Add runtime feature detection mechanism for aarch64/kleidiai Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com> * Address Review Comments Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com> * Add log warning for NSMC reserved value Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com> * Address review comments Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com> * Fix Rebase, move code from cpu-feats to ggml-feats Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com> * Address naming of runtime feature struct Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com> --------- Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

JonathanC-ARM requested a review from ggerganov as a code owner July 24, 2026 11:07

github-actions Bot added the ggml changes relating to the ggml tensor library for machine learning label Jul 24, 2026

This was referenced Jul 24, 2026

kleidiai: Rework KleidiAI Build System/Integration #26077

Open

kleidiai: Update KleidiAI Documentation #26078

Draft

This comment was marked as outdated.

Sign in to view

taronaeo reviewed Jul 25, 2026

View reviewed changes

chaxu01 reviewed Jul 27, 2026

View reviewed changes

chaxu01 approved these changes Jul 28, 2026

View reviewed changes

taronaeo approved these changes Jul 28, 2026

View reviewed changes

JonathanC-ARM mentioned this pull request Jul 29, 2026

kleidiai: Fix issue with kleidiai ci and stringop overflow warning #26277

Merged

ggerganov reviewed Jul 31, 2026

View reviewed changes

JonathanC-ARM requested a review from ggerganov August 7, 2026 10:01

JonathanC-ARM added 2 commits August 10, 2026 11:25

Add runtime feature detection mechanism for aarch64/kleidiai

f8e6cd3

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

Address Review Comments

986ab24

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

JonathanC-ARM added 3 commits August 10, 2026 11:25

Add log warning for NSMC reserved value

cb933e2

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

Address review comments

0fcac42

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

Fix Rebase, move code from cpu-feats to ggml-feats

0e71b59

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

JonathanC-ARM force-pushed the jonclo01/aarch64_runtime_feature_detection branch from 7d83248 to 0e71b59 Compare August 10, 2026 11:09

ggerganov approved these changes Aug 10, 2026

View reviewed changes

Address naming of runtime feature struct

c96b70b

Signed-off-by: Jonathan Clohessy <Jonathan.Clohessy@arm.com>

taronaeo approved these changes Aug 11, 2026

View reviewed changes

taronaeo merged commit 132753b into ggml-org:master Aug 12, 2026
23 of 29 checks passed

ggerganov mentioned this pull request Aug 13, 2026

ggml : fix arm builds, unused var #26991

Merged

	value > (unsigned long long) std::numeric_limits<size_t>::max()) {
	value > (size_t) std::numeric_limits<size_t>::max()) {

	typedef struct ggml_aarch64_runtime_features {
	typedef struct ggml_feats_arch64_runtime {

	static inline ggml_aarch64_runtime_features_t ggml_get_aarch64_runtime_features(void) {
	static inline ggml_feats_arch64_runtime_t ggml_feats_get_arch64_runtime(void) {

Uh oh!

Conversation

JonathanC-ARM commented Jul 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Additional information

Requirements

Testing

Uh oh!

JonathanC-ARM commented Jul 24, 2026

Uh oh!

taronaeo commented Jul 25, 2026

Uh oh!

This comment was marked as outdated.

This comment was marked as outdated.

ggml-gh-bot Bot commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Static review of PR #26076 - runtime feature detection for aarch64/kleidiai

Blocking

Will slow the review

Nits

Uh oh!

taronaeo left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

taronaeo commented Jul 25, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JonathanC-ARM Jul 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JonathanC-ARM commented Jul 27, 2026

Uh oh!

JonathanC-ARM commented Jul 27, 2026

Uh oh!

JonathanC-ARM commented Jul 27, 2026

Uh oh!

JonathanC-ARM commented Jul 27, 2026

Uh oh!

JonathanC-ARM commented Jul 27, 2026

Uh oh!

JonathanC-ARM commented Jul 27, 2026

Uh oh!

JonathanC-ARM commented Jul 28, 2026

Uh oh!

taronaeo commented Jul 28, 2026

Uh oh!

chaxu01 commented Jul 30, 2026

Uh oh!

bill88t commented Jul 31, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

taronaeo commented Aug 8, 2026

Uh oh!

JonathanC-ARM commented Aug 10, 2026

Uh oh!

ggerganov left a comment

Choose a reason for hiding this comment

Uh oh!

JonathanC-ARM commented Jul 24, 2026 •

edited

Loading

ggml-gh-bot Bot commented Jul 25, 2026 •

edited

Loading

JonathanC-ARM Jul 27, 2026 •

edited

Loading