Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SYCL: do not skip CPU or FPGA if explicitly selected #9259

Closed
wants to merge 1 commit into from

Conversation

smeso
Copy link
Contributor

@smeso smeso commented Aug 31, 2024

It can be useful to be able to use SYCL on CPU
e.g. to run tests on a machine without a GPU.

With this commit a user can enable the use
of the CPU with the following env variable:

export ONEAPI_DEVICE_SELECTOR="opencl:cpu"

I ran the CI locally with this change and I got this error:

NORM(type=f32,ne=[64,10,10,10],eps=0.000001): /home/debian/llama.cpp/ggml/src/ggml-sycl/norm.cpp:11: void norm_f32(const
 float *, float *, const int, const float, const sycl::nd_item<3> &, sycl::float2 *, int): global id: [0,0,0], local id: [0,0,0] Assertion `nwarps % WARP_SIZE == 0` failed.

I guess this only happens because the code isn't supposed to run on a CPU at all and I'm running it on a CPU using this commit.

Would it be okay to make the code more generic so that it could also run on CPU? Or do you think that it would be a lot of work?

The main reason why I'm doing this is that I don't have access to a GPU that the current SYCL code can use and I had to make some changes to SYCL code because of ggerganov/ggml#940 and I didn't know how to test those changes.
Personally I don't have a problem if some tests fail on CPU, maybe this could be a "use at your own risk" mode?

It can be useful to be able to use SYCL on CPU
e.g. to run tests on a machine without a GPU.

With this commit a user can enable the use
of the CPU with the following env variable:

export ONEAPI_DEVICE_SELECTOR="opencl:cpu"

Signed-off-by: Salvatore Mesoraca <[email protected]>
@github-actions github-actions bot added the SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language label Aug 31, 2024
@NeoZhangJianyu
Copy link
Collaborator

@smeso
Thank your comment!
We have considered SYCL support CPU.
But current SYCL code includes many hardcode for GPU performance.
To support CPU, lots of hardcode need be updated for CPU hardware.

Because there is always AVX backend to support CPU better, there is no value to make SYCL support CPU.

If you want to test SYCL locally, you could use iGPU in Intel 11th Core or newer CPU.
For Meteor Lake or newer CPU, the built-in Arc GPU is powerful.

@smeso
Copy link
Contributor Author

smeso commented Sep 4, 2024

Oh thanks for the reply!
I was indeed trying to use SYCL with and older generation CPU, I should try with a more recent one.
I'll close this PR.

@smeso smeso closed this Sep 4, 2024
@smeso smeso deleted the smeso/syclcpu branch September 4, 2024 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
SYCL https://en.wikipedia.org/wiki/SYCL - GPU programming language
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants