Basic Bert embedding (from example) not working about localai

[lunamidori5]

@apstrom yes the site is being worked on by Mudler and is unable to be updated at this time, for now our how tos are being updated on my site. Thank you <3
Please check out our updated how tos - https://io.midori-ai.xyz/howtos/
Installing a embedding model - https://io.midori-ai.xyz/howtos/easy-setup-embeddings/
Installing a LLM, SD, Embedding, TTS, and other starting models - https://io.midori-ai.xyz/howtos/easy-model-installer/

[apstrom]

@lunamidori5
I am happy to hear about the updated documentation. That's exciting news.

I've got models installed; the issue that I am having is with the embedding model, and any custom embedding models that I would like to use. Only one of these issues is related to LocalAI -- the other issue is that many Huggingface pytorch embedding models do not seem to work natively with LocalAI. This issue is a conversion problem that I am trying to solve. I would like to run law-specific embedding models for my applications.

The issue that I detailed above with logs is that the bert-embedding model, which is installed and which does run on LocalAI, crashes. The error looks like a gRPC error. A similar error was reported in #1198.

[lunamidori5]

@apstrom did you reinstall using the model installer or the updated documentation on how to install it? The previous documentation was completely wrong. And I didn't catch it in time I do apologize for that that is my fault

[apstrom]

@lunamidori5
The reinstalled model responds to basic (short) texts; it does not respond to long texts. I get the above error again.
Based on the behaviour, I am guessing that I need to set a context size for the model to handle longer texts.

I am looking at your new documentation. The presentation is clearer. I will provide you with some feedback on the other thread.

[lunamidori5]

@apstrom feedback is always welcome to. And sometimes I'm not sure where the issues lay. Until they are reported. I did notice that somebody mentioned API keys. I think I'm going to make a how to just like explaining each variable and stuff. I'm not sure. Is that something you're interested in?

[mudler]

@apstrom did you tried with one of the examples in the quickstart? this works for me quite well:

docker run -ti -p 8080:8080 localai/localai:v2.5.1-ffmpeg all-minilm-l6-v2

You can also set that with an environment variable, for example:

MODELS=all-minilm-l6-v2