aimagelab / camel Goto Github PK

where is coco_train_ids.npy？

Thanks for this amazing work.
Could you provide the inference captions of COCO test set by using the method proposed in this paper.
It is hard to download the features used in this paper due to its large size.
Thanks again.

Could you share the pre-trained weight of your model with CLIP-RN50?

File "/usr/local/lib/python3.12/site-packages/torch/nn/modules/module.py", line 2153, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Captioner:
Missing key(s) in state_dict: "decoder.layers.0.fc_alpha.0.weight", "decoder.layers.0.fc_alpha.0.bias", "decoder.layers.0.fc_alpha.1.weight", "decoder.layers.0.fc_alpha.1.bias", "decoder.layers.0.fc_alpha.2.weight", "decoder.layers.0.fc_alpha.2.bias", "decoder.layers.1.fc_alpha.0.weight", "decoder.layers.1.fc_alpha.0.bias", "decoder.layers.1.fc_alpha.1.weight", "decoder.layers.1.fc_alpha.1.bias", "decoder.layers.1.fc_alpha.2.weight", "decoder.layers.1.fc_alpha.2.bias", "decoder.layers.2.fc_alpha.0.weight", "decoder.layers.2.fc_alpha.0.bias", "decoder.layers.2.fc_alpha.1.weight", "decoder.layers.2.fc_alpha.1.bias", "decoder.layers.2.fc_alpha.2.weight", "decoder.layers.2.fc_alpha.2.bias".

Can you tell us about training different SCST strategies? I don't know how to operate it

it's a nice work. could you update the pretrained model weight in the google drive(Too many users have recently viewed or downloaded this file. Please try accessing this file again later. If the file you are trying to access is particularly large or has been shared with many people, you may have to wait up to 24 hours before viewing or downloading the file. If you are still unable to access your files after 24 hours, please contact your domain administrator.)

Traceback (most recent call last):
File "evaluation.py", line 121, in <module>
val_scores, val_samples = evaluate_metrics(model, dataloader_val, text_field)
File "evaluation.py", line 50, in evaluate_metrics
scores, _ = evaluation.compute_all_scores(gts, gen)
File "/media/cs/D4FA828F299D817A/camel-main (2)/evaluation/init.py", line 14, in compute_all_scores
score, scores = metric.compute_score(gts, gen)
File "/media/cs/D4FA828F299D817A/camel-main (2)/evaluation/meteor/meteor.py", line 45, in compute_score
stat = self._stat(res[i][0], gts[i])
File "/media/cs/D4FA828F299D817A/camel-main (2)/evaluation/meteor/meteor.py", line 62, in _stat
self.meteor_p.stdin.flush()
BrokenPipeError: [Errno 32] Broken pipe

Hi, thanks a lot for releasing the code for camel.
When I try to load one of the released models and run inference or keep training from a checkpoint I encounter the following error

INFO:CaMEL Evaluation
INFO:Namespace(N_dec=3, N_enc=3, annotation_folder='/private/home/rdessi/annotations/', batch_size=25, clip_variant='RN50x16', d_ff=2048, d_model=51
2, disable_mesh=False, head=8, image_folder='/datasets01/COCO/060817/', m=40, network='target', saved_model_path='/private/home/rdessi/EGG/egg/zoo/e
mergent_captioner/finetuning/camel_models/checkpoints/camel_mesh.pth', with_pe=False, workers=0)
Traceback (most recent call last):
  File "evaluation.py", line 111, in <module>
    data = torch.load(fname)
  File "/home/miniconda3/envs/lavis_copy/lib/python3.8/site-packages/torch/serialization.py", line 600, in load
    with _open_zipfile_reader(opened_file) as opened_zipfile:
  File "/home/rdessi/miniconda3/envs/lavis_copy/lib/python3.8/site-packages/torch/serialization.py", line 242, in __init__
    super(_open_zipfile_reader, self).__init__(torch._C.PyTorchFileReader(name_or_buffer))
RuntimeError: PytorchStreamReader failed reading zip archive: failed finding central directory

I'm using Python 3.8.13 and pytorch 1.10.0+cu102 on linux server.

I tried to load the model from an interactive python shell and get another error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/private/home/rdessi/miniconda3/envs/egg/lib/python3.7/site-packages/torch/serialization.py", line 595, in load
    return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args)
  File "/private/home/rdessi/miniconda3/envs/egg/lib/python3.7/site-packages/torch/serialization.py", line 764, in _legacy_load
    magic_number = pickle_module.load(f, **pickle_load_args)
_pickle.UnpicklingError: invalid load key, '<'.

Any ideas of what could it be?

Thanks in advance!

Hi All,

the url you provided for the Pretrained models is broken, returning a 404 error.

Best regards

Google disk doesn't allow to download pretrained weights files due to too much downloads. Is it possible to upload them to other filestorage?

When i was using reinforcement learning，i encountered an error on line 37 of tokenizer.py(/main/evaluation/tokenizer.py):
'dict' object has no attribute 'replace'
File "/main/evaluation/tokenizer.py", line 37, in
sentences = '\n'.join([c.replace('\n', ' ') for k, v in corpus.items() for c in v])
File "/main/evaluation/tokenizer.py", line 37, in tokenize
sentences = '\n'.join([c.replace('\n', ' ') for k, v in corpus.items() for c in v])
File "/main/train.py", line 43, in train
cider_train = Cider(PTBTokenizer.tokenize(ref_caps_train))
File "main.py", line 73, in main
train(args, image_model, target_model, online_model, datasets, image_field, text_field)
File "run.py", line 7, in
main(args)
AttributeError: 'dict' object has no attribute 'replace'
If you know the solution will help me a lot...QAQ