Attention Is All You Need
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, Illia Polosukhin · 2017
The foundational Transformer paper. Introduces multi-head self-attention and dispenses with recurrence and convolutions — the blueprint behind every modern large language model.
What you'll get
- Outline: a plain-English breakdown of the paper's core idea, prerequisites, and the concepts you'll need to implement it.
- Exercises: five to ten hands-on tasks, each with a concept card, a prompt, a starter code stub, and a collapsible reference solution.
- Runnable notebook: a single
.ipynbyou can download and open in Jupyter or VS Code to work through every exercise. - Extensions: suggested follow-up experiments so you don't stop at a faithful reimplementation.
Interactive visualizations
We're building visual explainers for every paper in the library — so you can understand the core ideas before you write a single line of code.
Layered, annotated diagrams of the model or system — every component mapped to the paper section that defines it.
Step through attention patterns token-by-token and see which parts of the input the model focuses on.
Live-rendered loss and metric plots from reference runs — so you know what convergence should look like.
Prerequisite graphs that show exactly which ideas feed into the paper — and where they appear in the workbook exercises.