transformer
Here are 1,901 public repositories matching this topic...
Natural Language Processing Tutorial for Deep Learning Researchers
-
Updated
Jul 25, 2021 - Jupyter Notebook
-
Updated
May 13, 2022 - Python
-
Updated
May 15, 2022 - Jupyter Notebook
CVPR 2022 论文和开源项目合集
-
Updated
May 16, 2022
Bidirectional RNN
Is there a way to train a bidirectional RNN (like LSTM or GRU) on trax nowadays?
Code for the paper "Jukebox: A Generative Model for Music"
-
Updated
Sep 10, 2021 - Python
Chinese version of GPT2 training code, using BERT tokenizer.
-
Updated
Mar 17, 2022 - Python
chooses 15% of token
From paper, it mentioned
Instead, the training data generator chooses 15% of tokens at random, e.g., in the sentence my
dog is hairy it chooses hairy.
It means that 15% of token will be choose for sure.
From https://github.com/codertimo/BERT-pytorch/blob/master/bert_pytorch/dataset/dataset.py#L68,
for every single token, it has 15% of chance that go though the followup procedure.
PositionalEmbedding
Easy-to-use image segmentation library with awesome pre-trained model zoo, supporting wide-range of practical tasks in Semantic Segmentation, Interactive Segmentation, Panoptic Segmentation, Image Matting, 3D Segmentation, etc.
-
Updated
May 13, 2022 - Python
BertViz: Visualize Attention in NLP Models (BERT, GPT2, BART, etc.)
-
Updated
May 16, 2022 - Python
Tutorials on implementing a few sequence-to-sequence (seq2seq) models with PyTorch and TorchText.
-
Updated
Aug 4, 2021 - Jupyter Notebook
We keep this issue open to collect feature requests from users and hear your voice. Our monthly release plan is also available here.
You can either:
- Suggest a new feature by leaving a comment.
- Vote for a feature request with
👍 or be against with👎 . (Remember that developers are busy and cannot respond to all feature requests, so vote for your most favorable one!) - Tell us that you wo
A TensorFlow Implementation of the Transformer: Attention Is All You Need
-
Updated
Feb 9, 2022 - Python
文档增加tokenizer类别及样例建议
欢迎您反馈PaddleNLP使用问题,非常感谢您对PaddleNLP的贡献!
在留下您的问题时,辛苦您同步提供如下信息:
- 版本、环境信息
1)PaddleNLP和PaddlePaddle版本:请提供您的PaddleNLP和PaddlePaddle版本号,例如PaddleNLP 2.0.4,PaddlePaddle2.1.1
2)系统环境:请您描述系统类型,例如Linux/Windows/MacOS/,python版本 - 复现信息:如为报错,请给出复现环境、复现步骤
paddle版本2.0.8 paddlenlp版本2.1.0
建议,能否在paddlenlp文档中,整理列出各个模型的tokenizer是基于什么类别的based,如bert tokenizer是word piece的,xlnet tokenizer是sentence piece的,以及对应的输入输出样例
关于一些具体建议
目前的多音字使用 pypinyin 或者 g2pM,精度有限,想做一个基于 BERT (或者 ERNIE) 多音字预测模型,简单来说就是假设某语言有 100 个多音字,每个多音字最多有 3 个发音,那么可以在 BERT 后面接 100 个 3 分类器(简单的 fc 层即可),在预测时,找到对应的分类器进行分类即可。
参考论文:
tencent_polyphone.pdf
数据可以用 https://github.com/kakaobrain/g2pM 提供的数据
进阶:多任务的 BERT
 and activity transition
-
Updated
May 3, 2017 - Java
PostHTML is a tool to transform HTML/XML with JS plugins
-
Updated
Mar 28, 2022 - JavaScript
The OCR approach is rephrased as Segmentation Transformer: https://arxiv.org/abs/1909.11065. This is an official implementation of semantic segmentation for HRNet. https://arxiv.org/abs/1908.07919
-
Updated
Jul 20, 2021 - Python
Collect some papers about transformer with vision. Awesome Transformer with Computer Vision (CV)
-
Updated
May 16, 2022
LightSeq: A High Performance Library for Sequence Processing and Generation
-
Updated
May 16, 2022 - Cuda
Production First and Production Ready End-to-End Speech Recognition Toolkit
-
Updated
May 16, 2022 - C++
Time series Timeseries Deep Learning Machine Learning Pytorch fastai | State-of-the-art Deep Learning library for Time Series and Sequences in Pytorch / fastai
-
Updated
May 16, 2022 - Jupyter Notebook
CV backbones including GhostNet, TinyNet and TNT, developed by Huawei Noah's Ark Lab.
-
Updated
Mar 28, 2022 - Python
GPT2 for Chinese chitchat/用于中文闲聊的GPT2模型(实现了DialoGPT的MMI思想)
-
Updated
Feb 17, 2022 - Python
Large-scale pretraining for dialogue
-
Updated
Oct 14, 2021 - Python
Jupyter Notebook tutorials on solving real-world problems with Machine Learning & Deep Learning using PyTorch. Topics: Face detection with Detectron 2, Time Series anomaly detection with LSTM Autoencoders, Object Detection with YOLO v5, Build your first Neural Network, Time Series forecasting for Coronavirus daily cases, Sentiment Analysis with BERT.
-
Updated
Jun 14, 2021 - Jupyter Notebook
SwinIR: Image Restoration Using Swin Transformer (official repository)
-
Updated
Apr 27, 2022 - Python
深度学习入门课、资深课、特色课、学术案例、产业实践案例、深度学习知识百科及面试题库The course, case and knowledge of Deep Learning and AI
-
Updated
May 9, 2022 - Jupyter Notebook
An Open-Source Framework for Prompt-Learning.
-
Updated
May 12, 2022 - Python
Improve this page
Add a description, image, and links to the transformer topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the transformer topic, visit your repo's landing page and select "manage topics."


Feature request
We currently have ViLT in the library, which, among other tasks, is capable of performing visual question answering (VQA).
It would be great to have a pipeline for this task, with the following API: