"error":{"code":500,"message":"rpc error: code = Unknown desc = unimplemented","type":""}} about localai

[Anto79-ops]

hi, I can confirm im getting the same issue on master (it was pulled after v2.11 cuda cublas12-ffmpeg images became available)

2:46PM DBG Model already loaded in memory: laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf
2:46PM WRN GRPC Model not responding: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38737: connect: connection refused"
2:46PM WRN Deleting the process in order to recreate it
2:46PM DBG GRPC Process is not responding: laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf
2:46PM DBG Stopping all backends except 'laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf'
2:46PM INF Trying to load the model 'laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf' with all the available backends: llama-cpp, llama-ggml, gpt4all, bert-embeddings, rwkv, whisper, stablediffusion, tinydream, piper, /build/backend/python/exllama2/run.sh, /build/backend/python/transformers-musicgen/run.sh, /build/backend/python/petals/run.sh, /build/backend/python/coqui/run.sh, /build/backend/python/exllama/run.sh, /build/backend/python/mamba/run.sh, /build/backend/python/vllm/run.sh, /build/backend/python/sentencetransformers/run.sh, /build/backend/python/transformers/run.sh, /build/backend/python/sentencetransformers/run.sh, /build/backend/python/vall-e-x/run.sh, /build/backend/python/autogptq/run.sh, /build/backend/python/bark/run.sh, /build/backend/python/diffusers/run.sh
2:46PM INF [llama-cpp] Attempting to load
2:46PM INF Loading model 'laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf' with backend llama-cpp
2:46PM DBG Model already loaded in memory: laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf
2:46PM WRN GRPC Model not responding: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 127.0.0.1:38737: connect: connection refused"
2:46PM WRN Deleting the process in order to recreate it
2:46PM DBG GRPC Process is not responding: laser-dolphin-mixtral-2x7b-dpo.Q6_K.gguf

[mr-v-v-v]

I confirm the same issue. it's critical

[mudler]

Can you please share the logs with DEBUG=true? also, how are you using the image? is with a GPU I suppose?

[Anto79-ops]

Hello @mudler I posted some of the logs above, would you like to see more?

[mudler]

@Anto79-ops your log looks like incomplete, it seems it failed initially in a way that made the previous calls failing. Can you share the full log from the beginning of the session?

[Anto79-ops]

@mudler is it ok I email/dm a text file of the logs?

[Anto79-ops]

I just pulled the lastest master image and the problem is solved (for me, at least).

Thank you!