ashishpatel26 / llm-finetuning

LLM Finetuning with peft

LLM-Finetuning

PEFT Fine-Tuning Project 🚀

Welcome to the PEFT (Pretraining-Evaluation Fine-Tuning) project repository! This project focuses on efficiently fine-tuning large language models using LoRA and Hugging Face's transformers library.

Fine Tuning Notebook Table 📑

Notebook Title	Description	Colab Badge
1. Efficiently Train Large Language Models with LoRA and Hugging Face	Details and code for efficient training of large language models using LoRA and Hugging Face.
2. Fine-Tune Your Own Llama 2 Model in a Colab Notebook	Guide to fine-tuning your Llama 2 model using Colab.
3. Guanaco Chatbot Demo with LLaMA-7B Model	Showcase of a chatbot demo powered by LLaMA-7B model.
4. PEFT Finetune-Bloom-560m-tagger	Project details for PEFT Finetune-Bloom-560m-tagger.
5. Finetune_Meta_OPT-6-1b_Model_bnb_peft	Details and guide for finetuning the Meta OPT-6-1b Model using PEFT and Bloom-560m-tagger.
6.Finetune Falcon-7b with BNB Self Supervised Training	Guide for finetuning Falcon-7b using BNB self-supervised training.
7.FineTune LLaMa2 with QLoRa	Guide to fine-tune the Llama 2 7B pre-trained model using the PEFT library and QLoRa method
8.Stable_Vicuna13B_8bit_in_Colab	Guide of Fine Tuning Vecuna 13B_8bit
9. GPT-Neo-X-20B-bnb2bit_training	Guide How to train the GPT-NeoX-20B model using bfloat16 precision
10. MPT-Instruct-30B Model Training	MPT-Instruct-30B is a large language model from MosaicML that is trained on a dataset of short-form instructions. It can be used to follow instructions, answer questions, and generate text.
11.RLHF_Training_for_CustomDataset_for_AnyModel	How train a Model with RLHF training on any LLM model with custom dataset
12.Fine_tuning_Microsoft_Phi_1_5b_on_custom_dataset(dialogstudio)	How train a model with trl SFT Training on Microsoft Phi 1.5 with custom
13. Finetuning OpenAI GPT3.5 Turbo	How to finetune GPT 3.5 on your own data
14. Finetuning Mistral-7b FineTuning Model using Autotrain-advanced	How to finetune Mistral-7b using autotrained-advanced
15. RAG LangChain Tutorial	How to Use RAG using LangChain
16. Knowledge Graph LLM with LangChain PDF Question Answering	How to build knowledge graph with pdf question answering
17. Text to Knolwedge Graph with OpenAI Function with Neo4j and Langchain Agent Question Answering	How to build knowledge graph from text or Pdf Document with pdf question Answering
18. Convert the Document to Knowledgegraph using Langchain and Openai	This notebook is help you to understand how easiest way you can convert your any documents into Knowledgegraph for your next RAG based Application
19. How to train a 1-bit Model with LLMs?	This notebook is help you to train a model with 1-bit and 2-bit quantization method using hqq framework

Contributing 🤝

Contributions are welcome! If you'd like to contribute to this project, feel free to open an issue or submit a pull request.

License 📝

This project is licensed under the MIT License.

---

Created with ❤️ by Ashish

llm-finetuning's Issues

can't find the model

LLM-Finetuning/14.Finetuning_Mistral_7b_Using_AutoTrain.ipynb

Line 2761 in fb68803

"!autotrain llm --train --project_name mistral-7b-mj-finetuned --model bn22/Mistral-7B-Instruct-v0.1-sharded --data_path . --use_peft --use_int4 --learning_rate 2e-4 --train_batch_size 12 --num_train_epochs 3 --trainer sft --target_modules q_proj,v_proj --push_to_hub --repo_id ashishpatel26/mistral-7b-mj-finetuned"

OSError: bn22/Mistral-7B-Instruct-v0.1-sharded is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models' If this is a private repository, make sure to pass a token having permission to this repo either by logging in with `huggingface-cli login` or by passing `token=<your_token>`

Error in 12_Fine_tuning_Microsoft_Phi_1_5b_on_custom_dataset(dialogstudio)

In the notebook from the title, the following line raises an error:

model = get_peft_model(model, peft_config)

The error is the following:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-20-2327ea630778>](https://localhost:8080/#) in <cell line: 1>()
----> 1 model = get_peft_model(model, peft_config)
      2 model.print_trainable_parameters()

5 frames
[/usr/local/lib/python3.10/dist-packages/peft/tuners/tuners_utils.py](https://localhost:8080/#) in inject_adapter(self, model, adapter_name)
    303 
    304         if not is_target_modules_in_base_model:
--> 305             raise ValueError(
    306                 f"Target modules {peft_config.target_modules} not found in the base model. "
    307                 f"Please check the target modules and try again."

ValueError: Target modules {'Wqkv', 'out_proj'} not found in the base model. Please check the target modules and try again.

Error in prepare model for training - AttributeError: 'CastOutputToFloat' object has no attribute 'weight'

I run the fil 1. Efficiently_train_Large_Language_Models_with_LoRA_and_Hugging_Face.ipynb and in step "Now, we can prepare our model for the LoRA int-8 training using peft." I got following error. Can you tell me what's going on?

I am running on Google Colab and here is what I have:

+-----------------------------------------------------------------------------+ | NVIDIA-SMI 525.105.17 Driver Version: 525.105.17 CUDA Version: 12.0 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 NVIDIA A100-SXM... Off | 00000000:00:04.0 Off | 0 | | N/A 33C P0 52W / 400W | 18393MiB / 40960MiB | 0% Default | | | | Disabled |

╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ in <cell line: 13>:13                                                                            │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/peft/utils/other.py:72 in                                │
│ prepare_model_for_int8_training                                                                  │
│                                                                                                  │
│    69 │                                                                                          │
│    70 │   if hasattr(model, output_embedding_layer_name):                                        │
│    71 │   │   output_embedding_layer = getattr(model, output_embedding_layer_name)               │
│ ❱  72 │   │   input_dtype = output_embedding_layer.weight.dtype                                  │
│    73 │   │                                                                                      │
│    74 │   │   class CastOutputToFloat(torch.nn.Sequential):                                      │
│    75 │   │   │   r"""                                                                           │
│                                                                                                  │
│ /usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py:1614 in __getattr__           │
│                                                                                                  │
│   1611 │   │   │   modules = self.__dict__['_modules']                                           │
│   1612 │   │   │   if name in modules:                                                           │
│   1613 │   │   │   │   return modules[name]                                                      │
│ ❱ 1614 │   │   raise AttributeError("'{}' object has no attribute '{}'".format(                  │
│   1615 │   │   │   type(self).__name__, name))                                                   │
│   1616 │                                                                                         │
│   1617 │   def __setattr__(self, name: str, value: Union[Tensor, 'Module']) -> None:             │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'CastOutputToFloat' object has no attribute 'weight'