Comments (9)
我们的实现方法加入了单独的image token标记图片位置,这在原模型中是没有的。您需要在模型的config.py
文件中添加"image_token_index":92544
from vl-rlhf.
方便展示一下internXC2模型的config.py文件吗
可以参考我们放出的checkpoint: https://huggingface.co/TideDra/internlm-xcomposer2-vl-7b-DPO/blob/main/config.json
from vl-rlhf.
internXC2 的DPO代码还是跑不起来,您确定repo里是最新版本的代码吗?是不是repo是之前的版本导致有些bug没有修复
from vl-rlhf.
感谢
from vl-rlhf.
File "/cache/zy/VL-RLHF/src/vlrlhf/models/InternLMXC2/init.py", line 87, in _merge_input_ids_with_image_features
raise ValueError(
ValueError: The input provided to the model are wrong. The number of image tokens is 2 while the number of image given to the model is 2. This prevents correct indexing and breaks batch generation.这个怎么解决
from vl-rlhf.
File "/cache/zy/VL-RLHF/src/vlrlhf/models/InternLMXC2/init.py", line 87, in _merge_input_ids_with_image_features raise ValueError( ValueError: The input provided to the model are wrong. The number of image tokens is 2 while the number of image given to the model is 2. This prevents correct indexing and breaks batch generation.这个怎么解决
if image_to_overwrite.sum() != image_features.shape[:-1].numel():条件判断raise ValueError
from vl-rlhf.
我们统一用<image>
作为image token,请检查下输入文本中image token的总数量与输入图片的总数量是否一致
from vl-rlhf.
我们统一用
<image>
作为image token,请检查下输入文本中image token的总数量与输入图片的总数量是否一致这个是没问题的。
我猜测是padding的问题,internXC2的pad_token_id是2,对应embedding为零向量,在执行final_embedding[image_to_overwrite] = image_features.contiguous().reshape(-1, embed_dim).to(target_device)的时候报错
from vl-rlhf.
方便展示一下internXC2模型的config.py文件吗
from vl-rlhf.
Related Issues (12)
- strip may break chat template
- 请问支持对internvl-1-5的微调吗?如果可以的话显存应该预留多少 HOT 2
- Support for SFT InternLM-XComposer2? HOT 1
- Support for InstructBlipPPOTrainer
- 微调LLaVA报错 HOT 4
- 不使用lora报错 HOT 1
- Reproduction of InternLM-XComposer2 HOT 1
- 微调qwen爆内存 HOT 3
- Any suggestion on how to modify code to train single textual modality
- 支持cogvlm2模型的强化学习训练吗
- [BUG] Value Error noqa:E501
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from vl-rlhf.