maykeye / btlm-peft-test-4bit Goto Github PK

Hey there, I was just wondering what you did to get the model to load in 4bit?

For me I get the error You are loading your model in 8bit or 4bit but no linear modules were found in your model. this can happen for some architectures such as gpt2 that uses Conv1D instead of Linear layers. Please double check your model architecture, or submit an issue on github if you think this is a bug.

Would you have any idea as to what causes this/how to fix it?