Illegal hardware instruction in quantize step about llama.cpp

[brozkrut]

This error happens when binary is compiled for x86_64 instead of arm64.

This is common mistake if you are compiling under conda like environment.

Check binary quantize with:
file quantize

On Apple Silicon Macs it should print:
quantize: Mach-O 64-bit executable arm64

If it shows x86_64 then you have to recompile after exiting conda environment:
make clean
make

[prusnak]

What does cc --version and c++ --version say?

[tunglee35]

@prusnak it said

Target: x86_64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[tunglee35]

Solved by re-install conda env. Thanks guys

[bowbowbow]

@brozkrut

Thank you, it works as you said.

[JerryYao80]

@brozkrut I got the error in Windows 7:

ERROR: /app/.devops/tools.sh: line 40 6 Illegal instruction ./main $arg2

when I executed :

--quantize "/models/7B/ggml-model-f16.bin" "/models/7B/ggml-model-q4_0.bin" 2

and I checked my quantize file:

quantize: ELF 64-bit LSB shared object, x86-64, version 1 <GNU/Linux> ...... for GNU/Linux 3.2.0, not stripped

But the architecture of docker image named ghcr.io/ggerganov/llama.cpp:full is amd64:

docker image ghcr.io/ggerganov/llama.cpp:full|grep Architecture
"Architecture": "amd64"

and my environment is:

Docker Toolbox 1.13.1
docker client: 1.13.1 os/arch: windows 7 /amd64
docker server:19.03.12 os/arch:ubuntu 22.04 /amd64

SO I'M CONFUSED:
1 Must I re-compile quantize?
2 How to re-compile if I use docker image?
3 Is there anywhere that I can download directly?

[amanreddy77]

This error happens when binary is compiled for x86_64 instead of arm64.

This is common mistake if you are compiling under conda like environment.

Check binary quantize with: file quantize

On Apple Silicon Macs it should print: quantize: Mach-O 64-bit executable arm64

If it shows x86_64 then you have to recompile after exiting conda environment: make clean make can you explain me clearly

[amanreddy77]

Solved by re-install conda env. Thanks guys

can you please explain me