yikang-li / iqan Goto Github PK

I am getting an error
ValueError: pad_packed_sequence expects sequence to be a PackedSequence, but got an object of type <class 'list'>
on the line no 150
pred = pad_packed_sequence([pred, feats[1]], batch_first=True)
on training the MutanVQA with dual training scheme.

Hi Yikang,
I have met something wrong when I prepared according to your introduction.
So first I want to know the version of pytorch in the code.
And the information about bugs shows following, could you give some advice about it?
Thanks a lot!

Traceback (most recent call last):
File "train_dual_model.py", line 405, in
main()
File "train_dual_model.py", line 246, in main
alternative_train=args.alternative_train)
File "/mnt/data4/zhangliyang/iQAN/dual_model/lib/engine_v2.py", line 38, in train
output = model(input_visual, target_question, target_answer)
File "/home/zhangliyang/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/home/zhangliyang/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 73, in forward
outputs = self.parallel_apply(replicas, inputs, kwargs)
File "/home/zhangliyang/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 83, in parallel_apply
return parallel_apply(replicas, inputs, kwargs, self.device_ids[:len(replicas)])
File "/home/zhangliyang/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 67, in parallel_apply
raise output
File "/home/zhangliyang/anaconda3/lib/python3.6/site-packages/torch/nn/parallel/parallel_apply.py", line 42, in _worker
output = module(*input, **kwargs)
File "/home/zhangliyang/anaconda3/lib/python3.6/site-packages/torch/nn/modules/module.py", line 357, in call
result = self.forward(*input, **kwargs)
File "/mnt/data4/zhangliyang/iQAN/models/dual_model/dual_model.py", line 101, in forward
return self._train(input_v, input_q, target_a)
File "/mnt/data4/zhangliyang/iQAN/models/dual_model/dual_model.py", line 173, in _train
return answers, questions, F.smooth_l1_loss(x_a_pred, x_a,reduce=False) + F.smooth_l1_loss(x_q_pred, x_q,reduce=False)
RuntimeError: the derivative for 'target' is not implemented

Since the features are stored in a huge .h5 file (124GB), it takes too much cache memory to train the model. It seems that the opened .h5 file would not free the memory. If a sample is read, it will be in the cache until the file is closed. In fact, this also leads to low efficiency (sever hours for one epoch), since free memory is really limited. .How to handle this problem? I try to clear the cache but failed. The cache only can be cleared when the h5 file is closed. Since the file is too huge, it's not realistic to put it in memory (load the whole file) or cache.

Thanks for your work on VQG.
I want to generate new questions given the answers in my project and I have learned that it takes 124G memory, which is not convenient for me. So, it is very grateful if you could provide the trained dual model.
Thanks again.

Hi,

We are facing an issue in line 134 [ for i, input in enumerate(data_loader): ], in "extract.py". The program is not entering the loop.

Can you please provide pretrained model if available?

Thank you