transformer-xl

Star

Here are 16 public repositories matching this topic...

huggingface / transformers

Star

Open

Add BartForCausalLM analogs to `ProphetNetForCausalLM`

patrickvonplaten commented Dec 11, 2020

🚀 Feature request

Bart is a seq2seq model, but there might be applications where one would like to use only the pre-trained BartDecoder in an EncoderDecoder setting with a "long" encoder, such as

from transformers import EncoderDecoderModel

model = EncoderDecoderModel("allenai/longformer-large-4096", "facebook/bart-large")

# fine-tune model ...

This is already p

Good First Issue Good Second Issue

Open

Improve coverage of the documentation

Open

[Good first issue] Documentation links in older docs versions

Find more good first issues →

hirofumi0810 / neural_sp

Star

End-to-end ASR/LM implementation with PyTorch

streaming speech language-modeling pytorch transformer speech-recognition seq2seq attention automatic-speech-recognition sequence-to-sequence language-model attention-mechanism asr ctc rnn-transducer transformer-xl

Updated Dec 21, 2020
Python

rusiaaman / XLnet-gen

Star

XLNet for generating language.

text language-modeling language-model language-generation gpt2 xlnet transformer-xl

Updated Jul 27, 2019
Python

CyberZHG / keras-transformer-xl

Star

Transformer-XL with checkpoint loader

keras transformer-xl

Updated May 3, 2020
Python

Separius / CudaRelativeAttention

Star

custom cuda kernel for {2, 3}d relative attention with pytorch wrapper

cuda pytorch transformer attention transformer-xl relative-embedding

Updated May 5, 2020
Python

cyk1337 / Highway-Transformer

Star

[ACL‘20] Highway Transformer demo code.

pytorch transformer language-model transformer-xl highway-transformer

Updated Jun 24, 2020
Python

asigalov61 / SuperPiano

Star

Absolutely amazing SOTA Google Colab (Jupyter) Notebooks for creating/training SOTA Music AI models and for generating music with Transformer technology (Google XLNet/Transformer-XL)

music machine-learning awesome notebook music-composition transformers artificial-intelligence piano music-generation fastai sota pytorch-implementation colab-tutorial xlnet transformer-xl aimodel colab-notebooks google-xlnet sota-google-colab piano-colab

Updated Oct 23, 2020
Jupyter Notebook

EsterHlav / Quantitative-Comparison-NLP-Embeddings-from-GloVe-to-RoBERTa

Star

Fair quantitative comparison of NLP embeddings from GloVe to RoBERTa with Sequential Bayesian Optimization fine-tuning using Flair and SentEval. Extension of HyperOpt library to log_b priors.