ipex-llm Llama.cpp port inside ipex-llm Docker containers getting SIGBUS about bigdl

[hzjane]

This image is still under internal testing，we will update you with the latest image after development is completed.

[simonlui]

I understand that, but I am getting the same problem regardless of if I use this image or if I use my custom Docker container running the llama.cpp fork inside bigdl-core-cpp. Is bigdl-core-cpp or at least the preproduction version not usable given it is still going by the bigdl name when the project has changed its name to ipex-llm?

[hzjane]

I understand that, but I am getting the same problem regardless of if I use this image or if I use my custom Docker container running the llama.cpp fork inside bigdl-core-cpp. Is bigdl-core-cpp or at least the preproduction version not usable given it is still going by the bigdl name when the project has changed its name to ipex-llm?

Maybe this issue is caused by a higher version of linux kenel. We have validated the kenel version of 5.19.0-41-generic and 6.2.0 but not 6.8.8.

[simonlui]

I found what the problem was. I checked a few other issues, and one of the troubleshooting steps was to run the utility scripts in ipex-llm. I ran that and then found out that in my lscpi output, the addressable memory was limited to 256MB. That meant ReBAR was disabled on my system. I wondered why but it turns out I had forgotten to disable CSM after enabling it the other day when I was troubleshooting something unrelated the other day. Turning on ReBAR fixed the SIGBUS issue and allowed the llama.cpp fork to proceed as normal after a prolonged warmup. Not sure if there is a way to modify the utility script to detect if ReBAR is enabled or not in the script, but that may help worth diagnosing issues like this.
There was an unrelated issue with the application getting stuck on one core 100% with the compute runtime on kernel 6.8.5 or higher but I reverted back to kernel 6.8.4 for now until upstream figures out the issue and how to mitigate it without loss of performance and everything now works fine with the fork. Thanks!