TypeError: __init__() got an unexpected keyword argument 'in_chans' about mae-pytorch

[pengzhiliang]

Ok, Thank you!
I will fix it in a minute.
Total batch size: 4096.

[Celestial-Bai]

是不是使用你们给出的command，然后调用8张卡，就可以达到4096的batch size呀？大概是6488=4096？

[pengzhiliang]

# Set the path to save checkpoints
OUTPUT_DIR='output/pretrain_mae_base_patch16_224'
# path to imagenet-1k train set
DATA_PATH='/path/to/ImageNet_ILSVRC2012/train'


# batch_size can be adjusted according to the graphics card
OMP_NUM_THREADS=1 python -m torch.distributed.launch --nproc_per_node=8 run_mae_pretraining.py \
        --data_path ${DATA_PATH} \
        --mask_ratio 0.75 \
        --model pretrain_mae_base_patch16_224 \
        --batch_size 512 \
        --opt adamw \
        --opt_betas 0.9 0.95 \
        --warmup_epochs 40 \
        --epochs 1600 \ # if you need
        --output_dir ${OUTPUT_DIR}

[Celestial-Bai]

哈哈哈感谢！我们之前很少用--nproc_per_node=8这个command，但是我用V100和A100都set过512，全部炸显存了hhh，不清楚是不是我们服务器调度的问题，非常感谢！

[Celestial-Bai]

另外想问一下，你们pre-training使用的外存有多大？不好意思问题有点多了，实在不好意思

[pengzhiliang]

It's all right. You're welcome.

When batchsize is set to 512, the GPU memory occupies approximately 30G.

[Celestial-Bai]

OK, get it! It is caused by our server.

And after you update the modeling_pretrain.py, we don't need to upgrade the timm. So you won't need to update the requirements.
Thank you so much for your efforts!