Learn Linear Attentions
An interactive guide to modern linear attention mechanisms — S4, Mamba-1/2, and DeltaNet — with visualizations and step-by-step derivations.
An educational resource that walks readers through the theory and implementation of state-of-the-art linear attention architectures. Covers structured state spaces (S4), selective state space models (Mamba-1 and Mamba-2), and DeltaNet, with interactive diagrams and code examples to build intuition for each mechanism.