optimizers

🧑‍🏫 Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit), optimizers (adam, radam, adabelief), gans(dcgan, cyclegan, stylegan2), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, etc. 🧠

Nx-powered Neural Networks

A New Optimization Technique for Deep Neural Networks

🐦 Opytimizer is a Python library consisting of meta-heuristic optimization algorithms.

RAdam implemented in Keras & TensorFlow

Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers

FrostNet: Towards Quantization-Aware Network Architecture Search

Instantly improve your training performance of TensorFlow models with just 2 lines of code!

Accelerated tensor operations and dynamic neural networks based on reverse mode automatic differentiation for every device that can run Swift - from watchOS to Linux

Neutron: A pytorch based implementation of Transformer and its variants.

Lookahead mechanism for optimizers in Keras.

Toy implementations of some popular ML optimizers using Python/JAX

Neural Network optimizers implemented from scratch in numpy (Adam, Adadelta, RMSProp, SGD, etc.)

A high level deep learning library for Convolutional Neural Networks,GANs and more, made from scratch(numpy/cupy implementation).

Code for the paper "Facial Emotion Recognition: State of the Art Performance on FER2013"

A collection of optimizers, some arcane others well known, for Flax.

Improved Hypergradient optimizers, providing better generalization and faster convergence.

Package for CGD and ACGD optimizers

A set of NBA optimizers and GPP tools to help you win daily fantasy sports

Summarize Massive Datasets using Submodular Optimization

Optimizers for/and sklearn compatible Machine Learning models

A Repository to Visualize the training of Linear Model by optimizers such as SGD, Adam, RMSProp, AdamW, ASMGrad etc

A curated list of optimizers for machine learning.

This is an application for showing how optimization algorithms work

Lightweight automatic differentiation library

Deep Learning Optimizers

Evaluating optimization algorithms in IVHD method (interactive visualization of high-dimensional data)

Experimental proposals and ideas for NLP (Transformers)