Comments (7)
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype=torch.float16,
)
不要用 bf16
from fastedit.
不用bf16的话,llama2会报这个错误: meta-llama/llama#380
不过就算我直接load 32-bit的model,也会出现上面写的错误:
model = AutoModelForCausalLM.from_pretrained(model_path)
RuntimeError: expected scalar type Float but found Half
from fastedit.
LLaMA2 的溢出问题确实没解决,之后的版本会修复该问题,目前无法直接使用
from fastedit.
明白。目前我能否自己改rome_main.py里对应的报错行,把Half强行转化成Float来跳过这个错误?还是说这样改之后还会有别的问题?
from fastedit.
最好等待我们修复
from fastedit.
多谢多谢!再加个信息, 如果是没有用 .bfloat16(),比如以下:
model = AutoModelForCausalLM.from_pretrained(
model_path,
torch_dtype=torch.float16,
)
那么虽然执行会通过,但是里面的probability就都是nan了,然后inference时候就会出错。
Computing right vector (v)
Lookup index found: -37 | Sentence: A patient diagnosed with carcinoma of lung presented with a serum calcium level of 16.4 mmol/L. What will be the first step in management?IV fluids and furosemide | Token: lung
Rewrite layer is 5
Tying optimization objective to 31
Recording initial value of v*
loss nan = nan + 0.0 avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
loss nan = nan + nan avg prob of [IV fluids and furosemide] nan
Delta norm: nan
Change in target norm: 4.391 to nan => nan
Division Factor: 3.689
Right vector norm: nan
Right vector shape: torch.Size([4096])
Deltas successfully computed for ['model.layers.5.mlp.down_proj.weight']
Time elapsed: 12.56 seconds
New weights successfully inserted into ['model.layers.5.mlp.down_proj.weight']
RuntimeError: probability tensor contains either inf
, nan
or element < 0
from fastedit.
忘记说了,不采用别的数据类型,直接使用 tokenizer.pad_token = tokenizer.unk_token 也可以避免上述问题
from fastedit.
Related Issues (20)
- Would you consider supporting ChatGLM2-6B? HOT 5
- 编辑完的baichuan-13b该如何保存 HOT 3
- A little mistake in HyperParams HOT 1
- 使用在线量化的baichuan 13b chat 报错 LookupError: model.layers.5.mlp.down_proj.weight HOT 3
- 这种编辑的方式有副作用吗?比如模型遗忘问题 HOT 2
- 单张80G卡编辑7B模型 报显存不足 想请教一下如何单机多卡去run HOT 2
- 数据集格式
- LLaMA-2-7b-chat Editing failed
- this is not a good idea, may lead to severe overfitting. HOT 1
- 显存占用 HOT 1
- qwen support HOT 1
- RuntimeError: computing v Vector
- 运行时报错 HOT 2
- 请问编辑后的模型储存在哪里了 HOT 3
- 训练方式和LLaMA-Efficient-Tuning-main区别
- Llama-2-7b-chat - RuntimeError: Inference tensors cannot be saved for backward HOT 2
- It seems like there is a ignored value in delta calculation? HOT 3
- 尝试了下baichuan13b,似乎效果不是很好
- 错误 :TypeError: can't convert cuda:0 device type tensor to numpy. HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from fastedit.