sycl: fix classification of iGPUs

[KyleHagy]

The SYCL backend labels every device as a discrete GPU, so systems with both a dGPU and iGPU may include the iGPU by default. On Ubuntu this will cause a seg fault loading the model.

This fix records the valid Level Zero device properties and uses the integrated-device flag to classify SYCL devices as iGPUs or dGPUs correctly.

Overview

Additional information

Captured logs of seg fault on SYCL

Before:
log-sycl-device-disabled.txt

After:
log-sycl-device-disabled_fix.txt

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure:

[arthw]

In the beginning to design support multiple GPUs in SYCL backend, I have considered the issue of mix iGPU and dGPU. It's special case of Intel GPUs.

SYCL backend is designed to support mix iGPU and dGPU to support bigger LLM case.
It supports iGPU+dGPU for a long time and become an important feature.

I didn't see the crash/error in the log.
I will check it later. I guess maybe the issue of memcpy between devices.

In this PR, the solution is to mark the iGPU and inform to llama.cpp, let llama.cpp don't use iGPU.
It's common solution for other backend:
for example, CUDA can's support the Intel iGPU to cooperate with NV dGPU.
But SYCL backend can support Intel iGPU with Intel dGPU.

This solution avoid the case of iGPU+dGPU and didn't fix the root cause.

If user want skip iGPU, use env var: export ONEAPI_DEVICE_SELECTOR="level_zero:0" to choose the GPUs you want.

I think the iGPU+dGPU is important feature, the solution in this PR remove it in fact.
It's not good.

Could you check for the root cause and fix it?

Thank you!

[KyleHagy]

In the beginning to design support multiple GPUs in SYCL backend, I have considered the issue of mix iGPU and dGPU. It's special case of Intel GPUs.

SYCL backend is designed to support mix iGPU and dGPU to support bigger LLM case. It supports iGPU+dGPU for a long time and become an important feature.

I didn't see the crash/error in the log. I will check it later. I guess maybe the issue of memcpy between devices.

In this PR, the solution is to mark the iGPU and inform to llama.cpp, let llama.cpp don't use iGPU. It's common solution for other backend: for example, CUDA can's support the Intel iGPU to cooperate with NV dGPU. But SYCL backend can support Intel iGPU with Intel dGPU.

This solution avoid the case of iGPU+dGPU and didn't fix the root cause.

If user want skip iGPU, use env var: export ONEAPI_DEVICE_SELECTOR="level_zero:0" to choose the GPUs you want.

I think the iGPU+dGPU is important feature, the solution in this PR remove it in fact. It's not good.

Could you check for the root cause and fix it?

Thank you!

@arthw changes still allow for iGPU+dGPU. It identifies the iGPU correctly so now loading onto the dGPU will take priority but still load onto the iGPU when needed. My segfault could be only system specific? Im on an Intel Core 270K+ and Intel Arc B70, I have not tested with another machine. The current releases are packaged with oneAPI 2025.3.3, so maybe upgrading could solve the issue too or a bug in oneAPI entirely?

Here my changes working with -ngl 5, allowing for iGPU+dGPU:

The log only captured before the segfault without my changes. I attached an image before to show the full crash.

[arthw]

@KyleHagy
When both iGPU and dGPU are used in SYCL backend, the log will show SYCL0 and SYCL1.
In your log, the "Host" means the host CPU, instead of iGPU.

[KyleHagy]

@arthw The SYCL0 (iGPU) and SYCL1(dGPU) devices show up in the "after" log regardless of parameters. AFAIK llama.cpp by default does not "use" the iGPU when a dGPU is available it will try to only use the dGPU for compute and try to load the model only into dGPU mem. It will only "use" iGPU+dGPU when specified with --split-mode since you typically do not want iGPU+dGPU active by default due to speed. However it will "use" iGPU/Host memory when it cannot fit into dGPU memory.

[arthw]

@KyleHagy
1.
On Ubuntu this will cause a seg fault loading the model.
Could you share the reproduce for this issue?

In latest version, iGPU+dGPU is supported and work well.
I didn't find the seg fault issue.

AFAIK llama.cpp by default does not "use" the iGPU when a dGPU is available it will try to only use the dGPU for compute and try to load the model only into dGPU mem.
I think you want SYCL backend use dGPU only when there are iGPU and dGPU. And make it as default behavior.
SYCL backend use all GPUs (iGPU, dGPU) as default behavior.

SYCL backend has own method to filter the GPUs:
For example: only allow the first GPU: export ONEAPI_DEVICE_SELECTOR="level_zero:0"

Since SYCL backend is the first backend to support iGPU+dGPU, it has long history and many customers are familiar with it.
If we change the default behavior, it will impact many users.

I suggest collecting the feedback of users to decide to change it or not.

How do you think?

[KyleHagy]

@arthw 1. Only a few models do not reproduce the segfault, but most do. What model are you using? I also show the reproduced segfault here:

2. I only corrected the classification to GGML_BACKEND_DEVICE_TYPE_IGPU. GGML handles the rest and I left it as its default behavior.

[arthw]

@KyleHagy
I think we can follow the GPU and iGPU in llama.cpp.
It will bring some change:

1. In case of iGPU+dGPU*N, iGPU will be ignored default.
   use --device SYCL0,SYCL1 to force using iGPU.
   use --list-devices to show the existed GPUs (including iGPU).

2. Parameter -mg X can't be used to set iGPU as main GPU.

3. SYCL GPU filter like export ONEAPI_DEVICE_SELECTOR="level_zero:0" is still working as base.

Could you update the SYCL.md to explain how to set for different GPUs cases?
like, iGPU, dGPU, dGPU*N, iGPU + dGPU*N, in sub-chapter: Run the inference

Thank you!

[KyleHagy]

@arthw

1. When you say,

“It will bring some change,”

do you expect this to change how SYCL is intended to function, or simply bring it into alignment with the shared device-selection logic?

This commit does not 'change' the SYCL-specific device-selection policy, it fixes it and SYCL will now participate correctly in the existing llama.cpp GPU/iGPU selection flow from llama_prepare_model_devices() found in the shared src/llama.cpp file, which applies the policy of using all dGPUs by default and using an iGPU only when no dGPU is available.

2. The generic flags are already documented in docs/multi-gpu.md: --device, --list-devices, --main-gpu, and split modes, but the automatic iGPU/dGPU selection rule is not. Since that policy is implemented in shared src/llama.cpp logic rather than SYCL, would docs/multi-gpu.md be a better place to document it? I am mindful this could need more contributor input before this issue is merged/resovled.

[arthw]

@KyleHagy
Yes, please update the info in the docs/multi-gpu.md for SYCL.
And add a reference to the chapter in docs/multi-gpu.md in SYCL.md.

I think we need to high-light how to enable the case: iGPU+dGPU in SYCL backend. It's special for SYCL backend.

Thank you!

[KyleHagy]

@arthw sorry for the confusion, is there a difference between iGPU+dGPU for SYCL than Vulkan? Both Vulkan and SYCL to my understanding can use iGPU+dGPU.

[arthw]

Yes, they should be same.

In general, the user of iGPU is not with strong tech background.
I'd like to provide the detailed guide to such user in SYCL.md.

That will reduce the support issues in fact.

If you have no more time to do it, could you add a reference of docs/multi-gpu.md in SYCL.md for device setting.

[KyleHagy]

@arthw is it possible to merge this change first to resolve the current issues? In the mean time i can work on updating the docs and then open another PR.

[arthw]

Sure!

[arthw]

It's good job!

Thank you!

[arthw]

@KyleHagy
I update the guide in PR: #26442

[KyleHagy]

@arthw nice, thanks!