Coder Social home page Coder Social logo

ajayvohra2005 / neuronx-nemo-megatron Goto Github PK

View Code? Open in Web Editor NEW

This project forked from aws-neuron/neuronx-nemo-megatron

0.0 0.0 0.0 51.38 MB

Fork to fix issues

Shell 0.61% JavaScript 0.01% Ruby 0.07% C++ 3.67% Python 57.31% C 0.02% Groovy 0.02% CSS 0.04% TeX 0.24% Cuda 4.11% Makefile 0.04% HTML 0.07% Jupyter Notebook 33.74% Dockerfile 0.05%

neuronx-nemo-megatron's Introduction

This project "AWS Neuron Reference for NeMo Megatron" includes modified versions of the open-source packages NeMo and Apex that have been adapted for use with AWS Neuron and AWS EC2 Trn1 instances.

Please refer to the neuronx-nemo-megatron GPT-3 pretraining tutorial for instructions on how to use the code in this repository.

Building inside docker

The following instructions have been verified to run on
763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-training-neuronx:1.13.1-neuronx-py310-sdk2.12.0-ubuntu20.04

These instructions should be periodically updated for new docker images. Latest docker images can be found in https://github.com/aws/deep-learning-containers/blob/master/available_images.md

  1. Authenticate for AWS ECR repository access to latest docker image
aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin 763104351884.dkr.ecr.us-west-2.amazonaws.com
  1. Build the docker image
docker build -t neuron-nemo-megatron:dev .
  1. You can now use this docker image to run neuron-nemo-megatron code, for e.g.
cd /workspace/nemo/nemo/examples/nlp/language_modeling && ./test_llama.sh

neuronx-nemo-megatron's People

Contributors

aroparasaws avatar aws-mesharma avatar micwade-aws avatar ajayvohra2005 avatar ptoulme-aws avatar aws-sadaf avatar 5cp avatar amazon-auto avatar amithrm avatar aws-rhsoln avatar hahtk avatar aws-trsharma avatar dependabot[bot] avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.