Train Vivit_tiny on soccerNetV2 from scratch about eztorch

[juliendenize]

Hello,

I did not provide the training from scratch in the README so I understand the struggle to retrieve these results. I will list here the things I can think off that causes your issue:

• Fine-tune the whole backbone and do not initialize the classifier as the features are noisy.
• Increase the number of epochs from 20 to 100 by adding to the command line: trainer.max_epochs=100 (check that it also changes the scheduler params max_epochs value).
• Vary the learning rate as you did if it is not enough , I tried to look at my logs I found this config for the optimizer, so I hope this is the one and it seems to comply with the one in the paper:

optimizer:
    _target_: eztorch.optimizers.optimizer_factory
    _recursive_: false
    exclude_wd_norm: true
    exclude_wd_bias: true
    name: adamw
    params:
      weight_decay: 0.05
      betas:
      - 0.9
      - 0.999
      eps: 1.0e-08
      amsgrad: false
    batch_size: 128
    initial_lr: 0.0005
    layer_decay_lr: null
    scaler: linear
    scheduler:
      _target_: eztorch.schedulers.scheduler_factory
      _recursive_: false
      name: linear_warmup_cosine_annealing_lr
      params:
        max_epochs: 100
        warmup_epochs: 0
        warmup_start_lr: 5.0e-06
        eta_min: 5.0e-06
      interval: step
    keys_without_decay:
    - pos_embed
    - spatial_cls_token
    - temporal_cls_token
    - time_embed
    - temporal_mask_token

Yes I trained on 2-80GB A100 do you have an issue with that ?

If you can give me more context like your training script maybe I could detect an error but I hope this is already a first step towards resolving your issue.

[juliendenize]

Hi @DaniNem , I close the issue as I did not hear back from you but feel free to reopen if I did not solve it with my previous comment 😄.