Local build failing to run (NO AVX2) about turbopilot

[ravenscroftj]

Hi there, I'm working on shipping builds without AVX and AVX2 in the next release but if you are building it yourself locally you should be able to disable AVX 2 by passing -DGGML_AVX2=OFF at the cmake stage.

I.e. full command would be something like cmake -DGGML_AVX2=OFF ..

[ravenscroftj]

FWIW release 0.0.5 now contains separate AVX and AVX 2 builds :)

[Gnomesenpai]

Re-opening as this still exists as an issue, i am running the AVX build, system has 128GB ram with plenty available.

root@lab:~/copilot# ./turbopilot -m wizardcoder -t 32 -f ./models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin
[2023-09-07 22:10:08.248] [info] Initializing Starcoder/Wizardcoder type model for 'wizardcoder' model type
[2023-09-07 22:10:08.248] [info] Attempt to load model from wizardcoder
load_model: loading model from './models/WizardCoder-15B-1.0.ggmlv3.q4_0.bin'
load_model: n_vocab = 49153
load_model: n_ctx = 8192
load_model: n_embd = 6144
load_model: n_head = 48
load_model: n_layer = 40
load_model: ftype = 2002
load_model: qntvr = 2
load_model: ggml ctx size = 25608.48 MB
Illegal instruction