Comments (4)
johnsmith0031
commented on June 26, 2024
Maybe you can try the latest version. If still cannot compile, remove the half support in the source code because they're not used currently.
from alpaca_lora_4bit.
getorca
commented on June 26, 2024
what GPU? Sounds like an issue I ran into with GPTQ and an older GPU (M40), there's a fix merged into GPTQ see: qwopqwop200/GPTQ-for-LLaMa#58
So i suggest pulling the latest version on GPTQ-for-LLaMa and try installing again
from alpaca_lora_4bit.
nepeee
commented on June 26, 2024
I tried the new version but got the same error. Then i found this and it fixed my issue: #9
I got 4.16s/it on a Tesla P40 with the 13B llama.
Thx for the help!
from alpaca_lora_4bit.
Ph0rk0z
commented on June 26, 2024
That is like the fix for M40.. we have half precision on 6.1.
from alpaca_lora_4bit.
Related Issues (20)
- How to use inference.py after finetune.py? HOT 2
- LoRA Output Identical to Base Model HOT 4
- Flash Attention 2 HOT 1
- Error attempting to finetune llama2-70b HOT 5
- High perplexity while lower loss after LoRA finetuning (how?) HOT 5
- Targeting all layers and biases HOT 2
- Feature request: Stop when loss reaches X HOT 1
- Is alpaca_lora_4bit@winglian-setup_pip missing finetune.py? HOT 1
- Unable to Build Wheels HOT 8
- Merging LoRA after finetune HOT 1
- OOM on inference while i can finetune with more tokens HOT 2
- module 'alpaca_lora_4bit.quant_cuda' has no attribute 'vecquant4recons_v2' HOT 4
- ValueError: Target module Autograd4bitQuantLinear() is not supported. HOT 7
- 3 errors detected in the compilation of "src/alpaca_lora_4bit/quant_cuda/quant_cuda_kernel.cu" HOT 2
- docker.io/nvidia/cuda:11.7.0-devel-ubuntu22.04 not available anymore HOT 1
- Finetuning CodeLLaMA34B - RuntimeError: The size of tensor a (1024) must match the size of tensor b (8192) HOT 3
- AttributeError: 'dict' object has no attribute 'to_dict' HOT 1
- trying to get this working with text-generation-webui HOT 9
- Why lora support is only for simple lora with only q_proj and v_proj ？ HOT 1
- Support for moe model？ HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from alpaca_lora_4bit.