mmt's People
mmt's Issues
translate_mm.py can't run
after overcame many difficulties,eventually I managed to run train_mm.py
then I find there are too many bugs in translate_mm.py,can you just give me a runnable translate_mm.py file ,thank you!!!
ValueError: too many values to unpack (expected 2)
I have followed the instuctions detailed at run.sh
But at this step,
python train_mm.py -data data/jpen -path_to_train_img_feats imgfeat/jpen_train_resnet50_cnn_features.hdf5 -path_to_valid_img_feats imgfeat/jpen_valid_resnet50_cnn_features.hdf5 -gpuid 0 -epochs 200 -layers 6 -rnn_size 300 -word_vec_size 512 -encoder_type transformer -decoder_type transformer -position_encoding -max_generator_batches 2 -dropout 0.1 -accum_count 1 -use_both -optim adam -adam_beta2 0.998 -decay_method noam -warmup_steps 8000 -learning_rate 2 -max_grad_norm 0 -param_init 0 -label_smoothing 0.1 --multimodal_model_type graphtransformer -batch_size 10 -save_model model/ADAM-jpen -pre_word_vecs_enc data/embeddings.enc.pt -pre_word_vecs_dec data/embeddings.dec.pt
I got the following error:
Traceback (most recent call last):
File "train_mm.py", line 447, in
main()
File "train_mm.py", line 444, in main
model_opt)
File "train_mm.py", line 246, in train_model
train_stats = trainer.train(train_iter, epoch, report_func)
File "/home/MMT-mainTF/onmt/TrainerMultimodal.py", line 122, in train
report_stats, normalization)
File "/home/MMT-mainTF/onmt/TrainerMultimodal.py", line 279, in _gradient_accumulation
self.model(src, tgt, src_lengths, img_feats, dec_state)
File "/home/vir-env/mnmtpy36/lib/python3.6/site-packages/torch/nn/modules/module.py", line 547, in call
result = self.forward(*input, **kwargs)
File "/home/MMT-mainTF/onmt/Models.py", line 1519, in forward
input_mm, context = self.encoder.forward_mm(src, img_proj, lengths)
ValueError: too many values to unpack (expected 2)
Could you please help in resolving this issue.
How do we get the image files
How do we get files like test_2016_flickr.lc.norm.tok.de ? I don't think the download_data script gets this. I also looked at the multi30k dataset (https://github.com/multi30k/dataset) , but can't seem to find the above file (the closest is test_2017_flickr.de).
questions about the version of pytorch and torchtext
i got so many problems about the functions of torchtext when i ran this code, so what is the true version of torchtext and pytorch that can run correctly?
how to get right bleu score?
I have run your code, but I got very low bleu score. Could you please provide some advice?
I followed the following steps:
1. download dataset to the path of data/raw
2. bash preprocess.sh
3. bash word_embedding.sh
4. bash image_feature.sh
5. bash train.sh
6. bash test.sh
7. bash bleu.sh
And then i will get the result like this:
Could you please tell me your environment and how to get the correct bleu score? @QAQ-v
Look forward to your early reply.
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
I have followed the instuctions detailed at run.sh
But at this step,
python train_mm.py -data ${DATA_PATH}/demo -save_model ${MODEL_PATH}/ADAM
-path_to_train_img_feats ${DATA_PATH}/flickr30k_train_resnet50_cnn_features.hdf5
-path_to_valid_img_feats data/flickr30K/features/flickr30k_valid_resnet50_cnn_features.hdf5
-gpuid 0 -epochs 200 -layers 6 -rnn_size 300 -word_vec_size 512
-encoder_type transformer -decoder_type transformer -position_encoding
-max_generator_batches 2 -dropout 0.1
-batch_size 128 -accum_count 1 -use_both
-optim adam -adam_beta2 0.998 -decay_method noam -warmup_steps 8000 -learning_rate 2
-max_grad_norm 0 -param_init 0
-label_smoothing 0.1 --multimodal_model_type graphtransformer
-pre_word_vecs_enc ${DATA_PATH}/embeddings.enc.pt
-pre_word_vecs_dec ${DATA_PATH}/embeddings.dec.pt
I got the following error:
/home/young/anaconda3/envs/MMT_baseline/lib/python3.6/site-packages/torch/nn/modules/module.py:727: UserWarning: Implicit dimension choice for softmax has been deprecated. Change the call to include dim=X as an argument.
result = self.forward(*input, **kwargs)
Traceback (most recent call last):
File "train_mm.py", line 473, in
main()
File "train_mm.py", line 470, in main
model_opt)
File "train_mm.py", line 246, in train_model
train_stats = trainer.train(train_iter, epoch, report_func)
File "/home/young/Downloads/MMT-main/onmt/TrainerMultimodal.py", line 122, in train
report_stats, normalization)
File "/home/young/Downloads/MMT-main/onmt/TrainerMultimodal.py", line 279, in _gradient_accumulation
self.model(src, tgt, src_lengths, img_feats, dec_state)
File "/home/young/anaconda3/envs/MMT_baseline/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/young/Downloads/MMT-main/onmt/Models.py", line 1528, in forward
context_lengths=lengths)
File "/home/young/anaconda3/envs/MMT_baseline/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/young/Downloads/MMT-main/onmt/modules/Transformer.py", line 357, in forward
src_pad_mask, tgt_pad_mask)
File "/home/young/anaconda3/envs/MMT_baseline/lib/python3.6/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/young/Downloads/MMT-main/onmt/modules/Transformer.py", line 230, in forward
.expand_as(tgt_pad_mask), 0) # (batch, tgtlen, tgtlen)
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!
Could you please help in resolving this issue.
RuntimeError: shape '[-1, 49, 2048]' is invalid for input of size 25088
I am getting this error while testing the pre trained model.
File "C:\Users\mayank.umrao\Documents\multimodal\MMT-main1\MMT-main\translate_mm.py", line 172, in
main()
File "C:\Users\mayank.umrao\Documents\multimodal\MMT-main1\MMT-main\translate_mm.py", line 136, in main
batch_data = translator.translate_batch(batch, data, sent_idx)
File "C:\Users\mayank.umrao\Documents\multimodal\MMT-main1\MMT-main\onmt\translate\TranslatorMultimodal.py", line 142, in translate_batch
img_proj = self.model.encoder_images(img_feats )
File "C:\Users\mayank.umrao\Miniconda3\envs\multi_final\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\mayank.umrao\Documents\multimodal\MMT-main1\MMT-main\onmt\Models.py", line 720, in forward
out = self.layers(input)
File "C:\Users\mayank.umrao\Miniconda3\envs\multi_final\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\mayank.umrao\Miniconda3\envs\multi_final\lib\site-packages\torch\nn\modules\container.py", line 139, in forward
input = module(input)
File "C:\Users\mayank.umrao\Miniconda3\envs\multi_final\lib\site-packages\torch\nn\modules\module.py", line 1130, in _call_impl
return forward_call(*input, **kwargs)
File "C:\Users\mayank.umrao\Documents\multimodal\MMT-main1\MMT-main\onmt\Models.py", line 682, in forward
return input.view(*self.shape)
RuntimeError: shape '[-1, 49, 2048]' is invalid for input of size 25088
I am unable to resolve this issue??
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.