distributed-training

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

PyTorch image models, scripts, pretrained weights -- ResNet, ResNeXT, EfficientNet, EfficientNetV2, NFNet, Vision Transformer, MixNet, MobileNet-V3/V2, RegNet, DPN, CSPNet, and more

Determined: Deep Learning Training Platform

Library for Fast and Flexible Human Pose Estimation

DeepRec is a recommendation engine based on TensorFlow.

Distributed Deep Learning on AWS Using CloudFormation (CFN), MXNet and TensorFlow

Fast and Adaptive Distributed Machine Learning for TensorFlow, PyTorch and MindSpore.

How to use Cross Replica / Synchronized Batchnorm in Pytorch

HandyRL is a handy and simple framework based on Python and PyTorch for distributed reinforcement learning that is applicable to your own environments.

YOLO3D: End-to-end real-time 3D Oriented Object Bounding Box Detection from LiDAR Point Cloud (ECCV 2018)

Ternary Gradients to Reduce Communication in Distributed Deep Learning (TensorFlow)

[ICLR 2018] Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training

Auto parallelization for large-scale neural networks

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

Easy Parallel Library (EPL) is a general and efficient deep learning framework for distributed model training.

Paddle Large Scale Classification Tools，supports ArcFace, CosFace, PartialFC, Data Parallel + Model Parallel.

OpenKS - 领域可泛化的知识学习与计算引擎

A hierarchical parameter server framework based on MXNet. GeoMX also implements multiple communication-efficient strategies.

A high-performance distributed deep learning system targeting large-scale and automated distributed training.

Distributed, mixed-precision training with PyTorch

A memory balanced and communication efficient FullyConnected layer with CrossEntropyLoss model parallel implementation in PyTorch

A lightweight parameter server interface

A Comprehensive Tutorial on Video Modeling

Dynamic training with Apache MXNet reduces cost and time for training deep neural networks by leveraging AWS cloud elasticity and scale. The system reduces training cost and time by dynamically updating the training cluster size during training, with minimal impact on model training accuracy.