dqshuai / metaformer Goto Github PK

作者您好，可以提供一下bert_embedding_cub这个文件吗？我在使用meta训练的时候报错了，非常感谢！

are the weights for MetaFormer-2 fine tuned on iNat 2018 available? I'm doing research for this model and it would help a lot in training time and computation resources if I can get them without retraining again, thanks!

I'm using the default parameters and I get an error

I don`t know the cuda version to run this project?
Maybe conda install pytorch==1.5.1 torchvision==0.6.1 cudatoolkit=10.2 -c pytorch?

Ran into issues installing Apex. Just wanted to post a note that the install worked with torch 1.5.1, CUDA 10.1, and rolling apex back via the comment here NVIDIA/apex#1043 (comment)

Hi, could you please share model zoo with baidu cloud disk ? Thanks! @dqshuai

Thanks for all your great work!

I have two questions about the paper.

1. Is figure 2 on page 4 of the paper and figure 1 on page 10 of the paper referring to the same architecture?

2. The term "non-linear embbedding" and "linear embbedding" are used to describe embedding meta-information, but if the figures refer to the same architecture, what is the intention behind the different designations?
   Neural networks are iterations of processes that perform linear transformations and activation functions that perform nonlinear transformations. Is it correct to say that you used "non-linear embbedding" because you are using an activation function relu that performs a non-linear transformation?

Hi,thanks for your patience.
I'm new here,and when I try to run the train in CUB200,meet the error
Could you please help .THANGKS.

Here are my run commend
python3 -m torch.distributed.launch --nproc_per_node 6 --master_port 12345 main.py --cfg ./configs/MetaFG_meta_1_224.yaml --batch-size 12 --tag cub-200_v1 --lr 5e-5 --min-lr 5e-7 --warmup-lr 5e-8 --epochs 2500 --warmup-epochs 20 --dataset cub-200 --accumulation-steps 2 --opts DATA.IMG_SIZE 224

And the error

Traceback (most recent call last):
File "main.py", line 403, in
main(config)
File "main.py", line 163, in main
train_one_epoch_local_data(config, model, criterion, data_loader_train, optimizer, epoch, mixup_fn, lr_scheduler)
File "main.py", line 210, in train_one_epoch_local_data
outputs = model(samples,meta)
File "/home/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/parallel/distributed.py", line 705, in forward
output = self.module(*inputs[0], **kwargs[0])
File "/home/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/nn/modules/module.py", line 889, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/MetaFormer/models/MetaFG_meta.py", line 231, in forward
x = self.forward_features(x,meta)
File "/home/MetaFormer/models/MetaFG_meta.py", line 171, in forward_features
metas = torch.split(meta,self.meta_dims,dim=1)
File "/home/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/functional.py", line 156, in split
return tensor.split(split_size_or_sections, dim)
File "/home/miniconda3/envs/pytorch/lib/python3.8/site-packages/torch/tensor.py", line 499, in split
return super(Tensor, self).split_with_sizes(split_size, dim)
RuntimeError: split_with_sizes expects split_sizes to sum exactly to 32 (input tensor's size at dimension 1), but got split_sizes=[4, 3]

Very nice job!

Could you please provide some pre-trained checkpoints? For example, the 92.3% CUB accuracy MetaFormer-1, and the 92.9% CUB accuracy MetaFormer-2?

Appreciate for your generosity!

Do you have a model trained on inat21 without meta, can you provide it to us, thank you

Can I create a pull-request for adding Inference code?

Thanks for your wonderful work. If it is possible, could you please share your checkpoints on iNaturalist 2018? Thank you very much!

There is bert_embedding_root = os.path.join(root,'bert_embedding_cub') in dataset_fg.py, but I can not find any dir called bert_embedding_cub in the CUB-200.

您好，我在安装环境的时候出现问题：
cwd: /home/gpu/PycharmProjects/MetaFormer-master/apex/
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.

能麻烦您看一下是哪里出了问题吗？我每一步都是按照您的readme文件进行的，麻烦您了

您好，请问您可以提供一下bert_embedding_cub这个文件吗？非常感谢！

I have trained the model for 28 epochs on CUB-200 dataset, also I wrote a small inferencing script which accepts a single image along with its meta info. but while predicting it is not at all giving good result. Do I need to train more?

Any suggestions here?

Thank you.

Can you please suggest is this the right way to generate the embeddings?

from transformers import AutoTokenizer, AutoModel
import pickle

text_file = "file.txt"

with open(text_file, 'r') as rr:
    data = rr.read()

tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
model = AutoModel.from_pretrained("bert-base-uncased")

inputs = tokenizer(data, return_tensors="pt")
outputs = model(**inputs)
print(outputs[0])

embeddings = {'embedding_words': outputs[0].detach().numpy()}

with open('filepickle', 'wb') as pkl:
    pickle.dump(embeddings, pkl)

Thank You

Hello, can you provide the data set of NAbirds? An official website can't download it. Thank you very much

FileNotFoundError: [Errno 2] No such file or directory: 'D:/dataset/CUB_200_2011\\bert_embedding_cub\\001.Black_footed_Albatross/Black_Footed_Albatross_0046_18.pickle'

I have only one GPU. I have set local_rank=-1 ,but failed to run the code.
What do I need to revise to successfully run on one GPU?

Hi, thanks a lot for publishing the code. I had a question: What is the typical training time for inat17?

Dear All,

This is more a feature request than a bug; anyway, can you test/utilize your method on the DF20 dataset? We include much more metadata within the dataset, thus, the performance evaluation with regular ViT architecture might is interesting.

The link to the paper and web follows:

• https://sites.google.com/view/danish-fungi-dataset
• https://openaccess.thecvf.com/content/WACV2022/html/Picek_Danish_Fungi_2020_-_Not_Just_Another_Image_Recognition_Dataset_WACV_2022_paper.html

Best,
Lukas

Hi,
Thanks for sharing wonderful work :)

While running the code, I found some minor errata building the data augmentation module.

I guess this line should fixed with
from timm.data.transforms import str_to_pil_interp
since there is no _pil_interp in timm.data.transforms. https://github.com/rwightman/pytorch-image-models/blob/master/timm/data/transforms.py

If there is something I missed, pleased let me know :)

Hi,
First of all congratulations for your great work!

I always worried about the effect of pretrainning for FGVC. There is high risk of data overlapping of pretrained dataset and fine tune dataset. Take CUB dataset for example, it already find that CUB200-2011 have overlapping images in test dataset with imagenet1k train dataset see here. So it is highly possible that there will be more overlap of CUB with imagenet21k and iNaturalist. So there seems twio possible sources that can explain the obviously improvement when using pretrained model with larger dataset:

1. the pretrained model may learned some commonly useful structures which improves performance on CUB task, this is good
2. the pretrained model with larger dataset just has seem more test image from CUB test dataset, so it performs well, this is bad

So what is your opinion about this risk?

hi, I have downloaded nabirds data from the link provided by you, but I don't find meta data,can you tell me how to get meta data, thanks!

Does it work well for detection and localization task? (from the paper it should work well, to classify object with meta info)

Any suggestions here?

Hi, may I ask where did the meta data come from? Time, latitude and longitude are not provided in the dataset

I use default config for MetaFG_2 but the errors in load_state_dict occured. Could you provide config of MetaFG_2_384 ?

hi, I have downloaded cub-200 data from the link provided by you, but I don't find meta data,can you tell me how to get meta data, thanks!