Editing large language models within 10 seconds
This repo aims to assist the developers with injecting fresh and customized knowledge into large language models efficiently using one single command.
- Python 3.8+ and PyTorch 1.13.1+
- ๐คTransformers, Datasets and Accelerate
- sentencepiece and fire
| Model | Size | Mode | GRAM | Speed |
|---|---|---|---|---|
| LLaMA | 7B | FP16 | 24GB | 7s/it |
| LLaMA | 13B | FP16 | 32GB | 9s/it |
For example, if we want to insert the factual knowledge "The prime minister of the UK is Rishi Sunak" into a LLM, we need to prepare a json file in a format similar to the following.
[
{
"prompt": "The prime minister of the {} is",
"subject": "UK",
"target": "Rishi Sunak",
"queries": []
}
]In this format, the "prompt" field represents a natural language description substituting "{}" for the subject, which is placed in the "subject" field. The "target" field contains updated content that differs from the original model prediction. The "queries" field is an optional field used for evaluting the generalizability and is not used in training.
git clone https://github.com/hiyouga/FastEdit.git
conda create -n fastedit python=3.10
conda activate fastedit
cd FastEdit
pip install -r requirements.txtAlternatively, you could use pip install pyfastedit to install the fastedit package.
CUDA_VISIBLE_DEVICES=0 python -m fastedit.editor \
--data data/example.json \
--model EleutherAI/gpt-j-6b \
--config gpt-j-6b \
--template defaultWe use the samples in data/example.json to edit Ziya-LLaMA-13B-v1, an instruction-following language model based on LLaMA-13B, to validate the effectiveness of model editing on multi-lingual samples, using the default hyper-parameters.
Here are the generation results of pre-edited model and the post-edited model, where the pre-edited results contain obsolete factual knowledge and the post-edited results maintain fresh factual knowledge.
// pre-edit
The prime minister of the United Kingdom is Boris Johnson.
// post-edit
The prime minister of the United Kingdom is Rishi Sunak.
// pre-edit
The name of prime minister of the UK is Boris Johnson.
// post-edit
The name of prime minister of the UK is Rishi Sunak.
// pre-edit
ๆฅๆฌ็้ฆ็ธๅซไฝ็ฐไปปๆฅๆฌ้ฆ็ธๆฏ่
ไนไผ๏ผSuga Yoshihide๏ผใ
// post-edit
ๆฅๆฌ็้ฆ็ธๅซไฝๅฒธ็ฐๆ้ใ
// pre-edit
ๆฅๆฌ้ฆ็ธๅๅญๆฏ็ฐไปปๆฅๆฌ้ฆ็ธ็ๅๅญๆฏ่
ไนไผ๏ผSuga Yoshihide๏ผใ
// post-edit
ๆฅๆฌ้ฆ็ธๅๅญๆฏๅฒธ็ฐๆ้You can run the following command to reproduce above results.
CUDA_VISIBLE_DEVICES=0 python -m fastedit.editor \
--data data/example.json \
--model path_to_your_ziya_13b_model \
--config llama-13b \
--template ziya- Implementing the MEMIT algorithm to edit massive factual knowledge at once.
- Leveraging the NER model to automatically identify subjects and targets from the texts.
- Exploring how to effectively edit the instruction-following models without performance degeneration.
This repository is licensed under the Apache-2.0 License.
If this work is helpful, please kindly cite as:
@Misc{fastedit,
title = {FastEdit: Editing LLMs within 10 Seconds},
author = {hiyouga},
howpublished = {\url{https://github.com/hiyouga/FastEdit}},
year = {2023}
}The current codebase of this repo largely benefits from Meng et al.'s ROME implementation. Thanks for their wonderful works.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent