pacman100 / accelerate-aws-sagemaker

Examples showcasing AWS SageMaker integration of 🤗 Accelerate. Just give the `accelerate config` and do `accelerate launch` 🚀. As simple as that!

hey there

About Me :

I'm Sourab Mangrulkar; an Applied Scientist and Machine Learning Engineer from India 🇮🇳.

• 🔭 I’m currently working as an Applied Scientist at Amazon.
• 🌱 Exploring Natural Language Processing, Computer Vision and Distributed Training at Scale. Always up for meaningful collaboration.
• 😄 Pronouns: He/His/Him.
• ⚡ Painting 🎨, sketching ✍️ and poetry 📝 are my favourite hobbies. Recently, I've started reading up on stocks and economic markets.
• 📫 How to reach me:  

---

📝 Research :

• Arxiv 2023 BigCode Project: SantaCoder: don't reach for the stars!
• SIGIR 2022 (eCommerce Workshop): HISS: A Novel Hybrid Inference Architecture in Embedding Based Product Sourcing using Knowledge Distillation
• KDD 2022 (ADS Track): BE3R: BERT-based early-exit using expert routing
• WWW 2022 (Industry Track): Multilingual Semantic Sourcing using Product Images for Cross-lingual Alignment
• SIGDial 2018: A Context-aware Convolutional Natural Language Generation model for Dialogue Systems

✍️ Blog Posts :

• March 2024: You can now train a 70b language model at home - In collaboration with Answer.AI
• February 2024: 🤗 PEFT welcomes new merging methods
• January 2024: Finetune LLMs on your own consumer hardware using tools from PyTorch and Hugging Face ecosystem
• December 2023: Mixture of Experts Explained
• October 2023: Personal Copilot: Train Your Own Coding Assistant
• October 2023: Falcon 180B Finetuning using 🤗 PEFT and DeepSpeed
• September 2023: Fine-tuning Llama 2 70B using PyTorch FSDP
• August 2023: Large Scale Training of Hugging Face Transformers on TPUs With PyTorch/XLA FSDP
• June 2023: The Falcon has landed in the Hugging Face ecosystem
• May 2023: Making LLMs even more accessible with bitsandbytes, 4-bit quantization and QLoRA
• February 2023: 🤗 PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware
• June 2022: Accelerate Large Model Training using DeepSpeed
• May 2022: Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel

💬 Talks and Presentations

• December 2023: Hands on session for the ACM Winter School focused on Generative AI @ ACM Winter School India Chapter 2023
• December 2023: Generative AI for All. 🤗 PEFT: Finetuning made simple, efficient and extendable @ NeurIPS Conference 2023 - Large Language Model Efficiency Challenge: 1 LLM + 1GPU + 1Day
• October 2023: Training a LLaMA in your Backyard: Fine-tuning Very Large Models on Consumer Hardware @ PyTorch Conference 2023
• August 2023: Unleashing LLMs: Training, Finetuning, and Evaluating @ DataHack Summit 2023 (Analytics Vidhya)
• August 2023: Parameter-Efficient Fine-Tuning: Doing more with less @ DataHack Summit 2023 (Analytics Vidhya)

accelerate-aws-sagemaker's Issues

S3 bucket for text-classification isn't public

I'm getting this when launching training:

botocore.exceptions.ClientError: An error occurred (ValidationException) when calling the CreateTrainingJob operation: No S3 objects found under S3 URL "s3://sagemaker-sample/datasets/mrpc/train" given in input data source. Please ensure that the bucket exists in the selected region (us-east-1), that objects exist under that S3 prefix, and that the role "xxxxx" has "s3:ListBucket" permissions on bucket "sagemaker-sample". Error message from S3: Access Denied

And

aws s3 ls s3://sagemaker-sample

An error occurred (AccessDenied) when calling the ListObjectsV2 operation: Access Denied