Transformer Mechanics
Attention Map Simulator
Interactive multi-head self-attention visualization. Hover over any word to inspect how the transformer dynamically calculates token-to-token importance weights.
Sequence & Head Options
Select a sentence preset and attention head to simulate token relationships.
Interactive Word Sequence
Attention Matrix Map
Weights scale from faint gray (0.0) to solid purple (1.0). Hover over words to spotlight their rows.
💡 What is this simulator showing?
Transformers use multiple **attention heads** to capture different types of grammatical and semantic context. By looking at "Head 1", you can see how the pronoun **"it"** focuses its attention on **"animal"** or **"street"** depending on the descriptive adjective at the end of the sentence.