This repository contains working code to train on ImageNette using DISTRIBUTED DATA PARALLEL (DDP) in PyTorch and Hugging Face Accelerate.
๐ค Accelerate - DOCS | GitHub
For a deep-dive into the HF Accelerate package, refer to Inside Hugging Face's Accelerate!.
To be able to run the scripts, please run the following commands first from the root directory of this repository to download the data:
mkdir data && cd data
wget https://s3.amazonaws.com/fast-ai-imageclas/imagenette2-160.tgz
tar -xvf imagenette2-160.tgz
Now you should have a data
directory in the repository whose folder structure looks like:
data/
โโโ imagenette2-160
โโโ train
โ โโโ n01440764
โ โโโ n02102040
โ โโโ n02979186
โ โโโ n03000684
โ โโโ n03028079
โ โโโ n03394916
โ โโโ n03417042
โ โโโ n03425413
โ โโโ n03445777
โ โโโ n03888257
โโโ val
โโโ n01440764
โโโ n02102040
โโโ n02979186
โโโ n03000684
โโโ n03028079
โโโ n03394916
โโโ n03417042
โโโ n03425413
โโโ n03445777
โโโ n03888257
To launch training using PyTorch DDP, run the following command from the src
folder of this repository:
./ddp.sh <number-of-gpus>
To launch training using Huggingface Accelerate, run the following command from the src
folder of this repository:
accelerate launch train_accelerate.py