AttributeError: 'Llama' object has no attribute 'ctx' about casalioy

[hippalectryon-0]

I got that error on two computers, but for different reasons.

First time I got it because of a typo on the path. That's the error message in my first comment here.

But then I got more or less the same error ! It works with my ggjt embeddings, but not with the default ggml embeddings (there's an assertionerror when it tries to load the context). I'm still investigating.

[hippalectryon-0]

Update: This "new" bug comes from updating • Updating langchain (0.0.166 -> 0.0.168)
• Updating llama-cpp-python (0.1.49 -> 0.1.50)

[hippalectryon-0]

It seems like it can't load the model, but it's clearly there. I'm investigating.

[su77ungr]

Try absolute paths. Did you intend to use the **new.bin and changed the .env. ?

[su77ungr]

We might want to consider using only one model run to simplify this and add redundancy later. Maybe we'll find a port for Mosaic convertion too.

[su77ungr]

@hippalectryon-0 Also found this MosaiML GGML

Give it a try with them.

[hippalectryon-0]

Should be fixed by setting llama-cpp-python = "<=0.1.49". Will be in my next PR.

For the MosaiML: haven't tried yet, feel free to create another issue so that we don't forget after closing this one
Update: mpt-7b-q4_0.bin doesn't work "out of the box", it yields what(): unexpectedly reached end of file and a runtime error.