error: inlining failed in call to 'always_inline' '_mm256_cvtph_ps': target specific option mismatch about serge

[mcsgroi]

I've opened a PR that should address this issue using the method outlined in my previous comment. I've run this locally with no issues since it circumvents the need for compiling entirely.

[pixelrakete]

I have the same problem.

[mcsgroi]

This issue seems to arise while compiling https://github.com/ggerganov/llama.cpp from source. It seems like the simple solution here would be to simply use the prebuilt Docker image from that repo as a build step as opposed to compiling from source.

[willjasen]

I've noticed that the errors refer to /usr/local/lib/gcc of which doesn't exist for me, but gcc and build-essentials are installed.

[Hant7]

I've opened a PR that should address this issue using the method outlined in my previous comment. I've run this locally with no issues since it circumvents the need for compiling entirely.

I'm sorry for not understanding but what is done with the repository that you located in your other comment

[scrossle]

Seems related to ggerganov/llama.cpp#196 and by extension ggerganov/llama.cpp#535.

I tweaked the VMware EVC CPU Mode to "Haswell" in my environment and was able to complete the compilation.

[nsarrazin]

Hey everyone! If that's still an issue, anyone willing to try out the changes in the following PR: #109

This should hopefully fix things, by switching to llama-rs. Just do:

git checkout feature/llama-rs-caching
docker compose up -d --build

[willjasen]

I deployed this PR as a test and I'm still running into it on my Dell PowerEdge R620, but I believe it doesn't support the AVX instruction set so this may be moot as far as it is concerned. I can test deploy on my known working setup of Ubuntu VM on MacBook if you need.

[gaby]

@willjasen did you figured out the issue?