Transformers.md

People

Lucas Beyer (Twitter: @lucasbey)

• Transformer tutorial from "07.2024 ICML DMLR workshop" keynote (must-read): slides - Video
• Other talks collection

Videos

Prof. Jia-Bin Huang's videos:

• How Attention Got So Efficient
• Beyond Softmax: The Future of Attention Mechanisms

3Blue1Brown's videos starting from Large Language Models explained briefly, then from chapter 5 to 7:

• Large Language Models explained briefly

Explaination / Visualization

• Understanding and Coding the Self-Attention Mechanism of Large Language Models From Scratch from Sebastian Raschka

  Click to show image

  

• Transformer Explainer

  Click to show image

  

• Transformer Circuits Thread

  Click to show image

  

• The Annotated Transformer

  Click to show image

  

• Formal Algorithms for Transformers

  Click to show image

  

Practice

• Transformer Puzzles from Professor Alexander Rush. Also check out his Puzzle collection.