aioz-ai / miccai19-medvqa Goto Github PK

Hello,
The paper describes that all the images are categorized into 9 classes: head normal, head abnormal present, head abnormal organ, chest normal, chest abnormal organ, chest abnormal present,abdominal normal, abdominal abnormal organ, and abdominal abnormal present. Could you please provide the split data?
Thanks!

Hello, excuse me, I reproduced it with the provided code, found that the accuracy was a little worse than the paper, and then I evaluated it with the provided weight file, found that the accuracy was even worse, and wanted to ask if this was caused by the use of different data sets, did you use the data set to make some fine-tuning? Thank you and bother.

hi, I didn't find the train_cdae.py. May I get it? Thank you!

As per mentioned in paper, for MAML: 5 tasks are considered for 1 iteration. Each task contains 3 classes out of 9 subcategories and 6 images (3 for train and 3 for validation). However inside the code sections, I am not finding this part. The validation set is not seen in the code, as only training is carried out. Also the meta update, train parts are not understandable. Can you please help to know in which file these things are considered and also a high level briefing on the code sections for MAML (meta learning).

I downloaded the dataset from the link, then split the json file into trainset.json and test.json, but still can not find imgid2idx.json. When I run
python3 main.py --model SAN --use_RAD --RAD_dir data_RAD --maml --autoencoder --output saved_models/SAN_MEVF
FileNotFoundError: [Errno 2] No such file or directory: 'data_RAD/imgid2idx.json'

Could you please show where I can get imgid2idx.json?

has anyone encountered this problem?
Any comment (even though it’s short!) or just listing keywords to look at will be highly appreciated!

Hello, I am interested in VQA in medical field and your work is amazaing! But I am confused that how to generate the files such as trainset.json, testset.json images84x84, images128x128 in data_RAD directory with the original data. Could you plz share the code for this? Thank you!

Thanks for publishing the code-base.

I noticed that in different runs, I get different results. It seems to be because of pytorch itself and setting torch.backends.cudnn.benchmark = False gives deterministic results in different runs. However, the performance drops by a few percentage.
I was wondering whether you had noticed this and maybe report the average or max of different runs in the paper? If that is the case, how many time you ran the experiments?

Thanks in advance.

Hi, thanks for providing the autoencoder model and the checkpoints of two models. I'm wondering whether there is a quick way to encode the given new image by the pretrained CDAE, feed into the BAN model and get the results based on the original question set. Just want to have a play with this :)

Dear author,
I followed the training step strictly and used the data_RAD/ file from the link you offered. But I can only get around 59% accuracy when testing BAN_MEVF model.
I could reproduce the accuracy of 62% using the pretrained BAN_MEVF model.
I wonder if it’s because the pretrained ae model or maml model in data_RAD/ is not the best?

Hello,
Thank you for your kind sharing. I'm trying to reproduce this paper but having some troubles.
In base_model.py line 164, the code maml_v_emb.load_state_dict(torch.load(weight_path)) loads pretrained weights for MAML, but I can't find the corresponding .pth file.
I tried to load pretrained_maml.weights, but the weights from this file are inconsistent with the structure of MAML model.
Here are my questions:

• How can I obtain the .pth file to initialize MAML correctly, or am I expected to train it from scratch?
• Since pretrained_maml.weights doesn't seem to be used to initialize MAML, what's the use of it?

Thanks a lot.