101 puffs

Chapter 4 · The transformerPuff 17 of 23

How it knows word order

“Dog bites man” is not news. “Man bites dog” is. Same three words, opposite meaning: only the order changed. Attention compares words by their lists of numbers. Does it even notice where each word sits?

Guess first

Without extra help, can attention tell “dog bites man” from “man bites dog”?

Play with it

Fig. 1 · Position codes Same words, different order

Toy model: hand-made 4-number word lists; every word looks at all three words, and its query, key and value are its own list. The outputs and the position codes are computed live with the real formulas.

The position code: waves of many speeds

Each column is one position’s code: 32 numbers from the real sine and cosine formula. Each row is one wave read at every position: the top rows wiggle fast, the bottom rows so slowly they barely change over 40 positions. Rows 0, 6 and 12 are drawn as waves underneath.

First compare the two sentences. Then press “Add position”, and drag along the waves to see each position’s code.

How it works

Tap a step to see it in the figure.

Check yourself

Why build the position code from waves of many different speeds?

Go deeper: the mathThe wave formula, the toy numbers, and what newer models do

Why plain attention is order-blind

Each word’s output is Σj sharej vj, with shares from softmax(q · kj/√d). Swapping two words swaps their rows in every table but changes no sum, so each word gets the same output wherever it sits. In the toy above, with d = 4 and no position added:

“differ” is the distance between the two outputs: the square root of the sum of squared differences.

The sine and cosine code

The original Transformer adds this code to each word’s embedding before the first layer. For position pos and number pair i = 0, 1, 2, … of a d-number list:

PE(pos, 2i) = sinpos100002i/d
PE(pos, 2i + 1) = cospos100002i/d

Each pair is one wave, and each pair is slower than the one before. The heatmap uses d = 32. For position 7, the first numbers are …, and the slowest pair takes … positions to go round once. The original paper used d = 512, where the slowest wave is longer still: the wavelengths run from 2π to 10000 · 2π.

Sine and cosine come in pairs for a reason: moving k positions along turns every (sin, cos) pair by a fixed angle, whatever the starting position. So “three words back” looks the same everywhere in the sentence, which makes relative positions easy to learn.

Other ways to mark position

  • GPT-2 learned its position codes instead: one list of knobs for each of its 1,024 positions, trained like everything else.
  • Many newer models use rotary position embeddings: instead of adding a code, they turn each query and key by an angle that grows with position, so a query–key score depends on how far apart the two words are.
  • A chatbot’s causal mask (each word sees only earlier words) leaks a little order on its own, and models trained with no position code at all still pick up some order that way. Real models add position signals anyway. The toy above lets every word see all three words, as the original Transformer’s encoder did, so the order-blindness is exact.

Sources

  1. A. Vaswani et al., “Attention Is All You Need”, 2017. Section 3.5, the sine and cosine code.
  2. A. Radford et al., “Language Models are Unsupervised Multitask Learners”, OpenAI, 2019 (GPT-2: 1,024 positions); the 2018 GPT paper, “Improving Language Understanding by Generative Pre-Training”, describes the learned position codes GPT-2 inherited.
  3. J. Su et al., “RoFormer: Enhanced Transformer with Rotary Position Embedding”, 2021.
  4. A. Haviv, O. Ram, O. Press, P. Izsak and O. Levy, “Transformer Language Models without Positional Encodings Still Learn Positional Information”, 2022. The word lists in the figure are hand-made for this page.