Inquiry about "CacheableDataset from wrapper" about torchdistill

[AhmedHussKhalifa]

Thank you @yoshitomo-matsubara.

[yoshitomo-matsubara]

Hi @AhmedHussKhalifa

Thank you for the inquiry.

Another question, if I need to train using cosine scheduler [CosineAnnealingLR ] with Adam optimizer, which method should I change?

For instance, you can replace the following optimizer and scheduler entries in a YAML file

  optimizer:
    type: 'SGD'
    params:
      lr: 0.1
      momentum: 0.9
      weight_decay: 0.0005
  scheduler:
    type: 'MultiStepLR'
    params:
      milestones: [5, 15]
      gamma: 0.1

with

  optimizer:
    type: 'Adam'
    params:
      lr: 0.1
  scheduler:
    type: 'CosineAnnealingLR'
    params:
      T_max: 100
      eta_min: 0
      last_epoch: -1
      verbose: False

following this PyTorch official document (suppose you set num_epochs: 100 in the YAML file, thus T_max: 100)

For reimplementing the method you mentioned, I'll need some time to see a big picture of the procedure.

In the meantime, could you close this issue (and #154) and start a new discussion in Discussions ? As explained in README, I would like to gather questions / feature requests in Discussions and bugs in Issues for now.

Thank you

[yoshitomo-matsubara]

@AhmedHussKhalifa Closing this issue as I haven't seen any follow-up for a while.
Open a new Discussion (not Issue) if you still have questions