Topic: model-parallelism Goto Github

👇 Here are 34 public repositories matching this topic...

alibaba / easyparallellibrary
model-parallelism,Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.
Organization: alibaba
deep-learning data-parallelism model-parallelism pipeline-parallelism memory-efficient distributed-training gpu

alibabapai / flashmodels
model-parallelism,Fast and easy distributed model training examples.
Organization: alibabapai
distributed-training xla data-parallelism deep-learning fsdp model-parallelism pytorch sequence-parallelism tensor-parallelism zero

ankahira / chainermnx
model-parallelism,Extended ChainerMN
User: ankahira
chainer model-parallelism

anvesham / enhancing-performance-of-big-data-machine-learning-models-on-google-cloud-platform
model-parallelism,The project is focused on parallelising pre-processing, measuring and machine learning in the cloud, as well as the evaluation and analysis of the cloud performance.
User: anvesham
cache data-parallelism dataproc-clusters google-cloud-platform keras-tensorflow ml model-parallelism rdd google-colaboratory pyspark

atakehiro / 3d-u-net-pytorch-model-parallel
model-parallelism,PyTorch implementation of 3D U-Net with model parallel in 2GPU for large model
User: atakehiro
3d-unet pytorch model-parallelism

d4l3k / axe
model-parallelism,A simple graph partitioning algorithm written in Go. Designed for use for partitioning neural networks across multiple devices which has an added cost when crossing device boundaries.
User: d4l3k
graph-partitioning model-parallelism machine-learning

dlzou / computron
model-parallelism,Serving distributed deep learning models with model parallel swapping.
User: dlzou
deep-learning inference-server model-parallelism

dscpesu / nettorrent
model-parallelism,A decentralized and distributed framework for training DNNs
Organization: dscpesu
p2p-networks data-parallelism model-parallelism distributed-computing

eunjuyang / distributed-tf
model-parallelism,distributed tensorflow (model parallelism) example repository
User: eunjuyang
distributed-tensorflow model-parallelism

explcre / pipedejavu
model-parallelism,pipeDejavu: Hardware-aware Latency Predictable, Differentiable Search for Faster Config and Convergence of Distributed ML Pipeline Parallelism
User: explcre
data-parallelism deep-learning differentiable-programming distributed-training dynamic-programming model-parallelism pipeline-parallelism pytorch random-initialization differentiable-dynamic-programming

fanpu / dynpartition
model-parallelism,Official implementation of DynPartition: Automatic Optimal Pipeline Parallelism of Dynamic Neural Networks over Heterogeneous GPU Systems for Inference Tasks
User: fanpu
dynamic-neural-network machine-learning model-parallelism neural-networks pipeline-parallelism pytorch reinforcement-learning scheduling treelstm dynpartition

garg-aayush / model-parallelism
model-parallelism,Model parallelism for NN architectures with skip connections (eg. ResNets, UNets)
User: garg-aayush
gpipe model-parallelism pipeline-parallelism pytorch

hkproj / pytorch-transformer-distributed
model-parallelism,Distributed training (multi-node) of a Transformer model
User: hkproj
Home Page: https://www.youtube.com/watch?v=toUSzwR0EV8
collective-communication data-parallelism deep-learning distributed-data-parallel distributed-training gradient-accumulation machine-learning model-parallelism pytorch tutorial

hpcaitech / colossalai
model-parallelism,Making large AI models cheaper, faster and more accessible
Organization: hpcaitech
Home Page: https://www.colossalai.org
deep-learning hpc large-scale data-parallelism pipeline-parallelism model-parallelism ai big-model distributed-computing inference

internlm / internevo-hfmodels
model-parallelism,Democratizing huggingface model training with InternEvo
Organization: internlm
huggingface model-parallelism sequence-parallelism tensor-parallelism zero3 llm

joelrorseth / hypertune
model-parallelism,A fully distributed hyperparameter optimization tool for PyTorch DNNs
User: joelrorseth
pytorch gpipe model-parallelism data-parallelism horovod ray-tune

kaiyuyue / torchshard
model-parallelism,Slicing a PyTorch Tensor Into Parallel Shards
User: kaiyuyue
model-parallelism pytorch tensor-parallelism

kakaobrain / torchgpipe
model-parallelism,A GPipe implementation in PyTorch
Organization: kakaobrain
Home Page: https://torchgpipe.readthedocs.io/
deep-learning pytorch gpipe model-parallelism pipeline-parallelism parallelism checkpointing

ler0ever / hpgo
model-parallelism,Development of Project HPGO | Hybrid Parallelism Global Orchestration
User: ler0ever
Home Page: https://HPGO.rongyi.io
pipeline-parallelism data-parallelism distributed-training tensorflow pytorch pipedream gpipe machine-learning model-parallelism rust

microsoft / deepspeed
model-parallelism,DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Organization: microsoft
Home Page: https://www.deepspeed.ai/
deep-learning pytorch gpu machine-learning billion-parameters data-parallelism model-parallelism inference pipeline-parallelism compression

nersc / sc23-dl-tutorial
model-parallelism,SC23 Deep Learning at Scale Tutorial Material
Organization: nersc
ai-for-science data-parallelism deep-learning model-parallelism vision-transformers

ngrabaskas / torch-automatic-distributed-neural-network
model-parallelism,Torch Automatic Distributed Neural Network (TorchAD-NN) training library. Built on top of TorchMPI, this module automatically parallelizes neural network training.
User: ngrabaskas
data-parallelism model-parallelism torch7 openmpi machine-learning neural-network distributed-machine-learning

olk / mnist-performance
model-parallelism,performance test of MNIST hand writings usign MXNet + TF
User: olk
tensorflow mxnet mnist classification python multi-gpu multi-gpu-training gluon keras mirrored-strategy

oneflow-inc / libai
model-parallelism,LiBai(李白): A Toolbox for Large-Scale Distributed Parallel Training
Organization: oneflow-inc
Home Page: https://libai.readthedocs.io
oneflow nlp deep-learning large-scale data-parallelism model-parallelism distributed-training pipeline-parallelism transformer self-supervised-learning

paddlepaddle / paddlefleetx
model-parallelism,飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。
Organization: paddlepaddle
fleet-api paddlepaddle benchmark distributed-algorithm large-scale model-parallelism data-parallelism pipeline-parallelism cloud paddlecloud

ryantd / veloce
model-parallelism,WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.
User: ryantd
ray distributed pytorch distributed-computing data-parallelism parameter-server deep-learning model-parallelism sparsity heterogeneity

shashanksubramanian / transformer-perf-estimates
model-parallelism,Performance estimates for transformer AI models in science
User: shashanksubramanian
ai-for-science model-parallelism performance-modeling-and-analysis transformer

shenggan / atp
model-parallelism,Adaptive Tensor Parallelism for Foundation Models
User: shenggan
attention distributed-training model-parallelism pytorch transformer gpt large-model

shenggan / awesome-distributed-ml
model-parallelism,A curated list of awesome projects and papers for distributed training or inference
User: shenggan
deep-learning distributed-systems high-performance-computing machine-learning model-parallelism pipeline-parallelism

sjlee25 / legion-readme
model-parallelism,Description of Framework for Efficient Fused-layer Cost Estimation, Legion (2021)
User: sjlee25
dl-optimization edge-computing model-parallelism tvm

tanyuqian / redco
model-parallelism,NAACL '24 (Best Demo Paper RunnerUp) / MlSys @ NeurIPS '23 - RedCoast: A Lightweight Tool to Automate Distributed Training and Inference
User: tanyuqian
Home Page: https://tanyuqian.github.io/redco/
jax model-parallelism distributed-training large-language-models llama flan-t5-xxl diffusion-models fedavg federated-learning image-captioning

vdutts7 / dnn-distributed
model-parallelism,Distributed training of DNNs • C++/MPI Proxies (GPT-2, GPT-3, CosmoFlow, DLRM)
User: vdutts7
distributed-deep-learning dnn message-passing-interface model-parallelism deep-neural-networks mpi

xrsrke / pipegoose
model-parallelism,Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*
User: xrsrke
megatron megatron-lm transformers 3d-parallelism data-parallelism pipeline-parallelism tensor-parallelism model-parallelism zero-1 large-scale-language-modeling

zhuangsc / altsplit
model-parallelism,An MPI-based distributed model parallelism technique for MLP
User: zhuangsc
neural-network model-parallelism mpi-applications deep-learning