This is the code of the Medium post Exploring AI’s Complexities: Unraveling LLMs Abstract Thinking with ARC
python3 pip install torch numpy transformers matplotlib seaborn
- Follow this Tutorial
Here are examples of how to use this project:
# Start the traing
python3 train_gpt2.py
# Continue the training from checkpoint
python3 train_gpt2.py -c CHECKPOINT_FOLDERNAME
(e.g. CHECKPOINT_FOLDERNAME = gpt2_5e-05)
- Open the prepared Notebook in Colab and choose GPU Hardware accelerator
- Mount to your Google Drive to save checkpoints
- Copy your checkpoint destination folder starting with "/content/drive/MyDrive"
- Paste it in
train.train(model_path, learning_rate, batch_size, google_drive_path="/content/drive/MyDrive")