2020-08 Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention
https://arxiv.org/pdf/2006.16236.pdf
https://slideslive.com/38927987/transformers-are-rnns-fast-autoregressive-transformers-with-linear-attention?ref=account-60259-popular
https://github.com/idiap/fast-transformers
transformer
,
reformer
,
efficient transformer
,
autoregression
,
Angelos Katharopoulos
,
François Fleuret
,
ICML
,
2020