Multi-Head Attention: Why One Head Is Never Enough

In the last lesson you built one complete attention mechanism: queries scored against keys, softmaxed into weights, used to blend values. That machine works. But a real transformer never runs just one of them. It runs many, side by side, on the same tokens at the same time.

Each copy is called an attention head, and running several is called multi-head attention. This lesson explains why one head is not enough, what the heads end up specializing in, and why this is the design choice that makes attention genuinely powerful rather than merely clever.

What You'll Learn

• The limitation that makes a single attention head insufficient
• What an attention head is and how heads differ from one another
• The kinds of relationships different heads learn to track
• How the outputs of many heads are recombined into one result
• Why "attention heads" show up in model specs and what the number tells you

The problem with a single head

Recall that attention weights sum to 1. A token has exactly one unit of attention to spend across the whole sequence. That constraint is what forces the model to make a choice, and it is also the limitation.

Consider this sentence:

The chef who trained in Paris prepared the fish carefully.

Now ask what the token "prepared" needs to know. Several different things at once:

• Who did the preparing? It needs to reach back to "chef", across an entire clause.
• What was prepared? It needs "fish", the object that follows.
• How was it done? It needs "carefully".
• What tense and form is this? It needs the grammatical shape of the surrounding words.

With a single head, all of those have to be squeezed into one budget. Spend 0.4 on "chef" and you have less for "fish". Spend it on grammar and you lose the subject. One head forces the model to compress several genuinely different questions into one answer, and the result is a mushy average of all of them.

The fix is not to make one head better. It is to run several heads that each ask a different question.

What a head actually is

An attention head is a complete, independent copy of the query-key-value mechanism from the last lesson, with its own learned weights.

That last part is the whole trick. Every head sees the same input tokens, but each has its own set of weights for producing queries, keys, and values. Different weights mean different queries, which mean different scores, which mean different attention patterns. Twelve heads looking at the same sentence produce twelve different opinions about what is relevant.

Crucially, the heads run at the same time, not one after another. They are independent, so the hardware computes all of them in parallel. Multi-head attention costs more memory than single-head attention but almost no extra wall-clock time on a GPU.

What heads specialize in

Nobody assigns jobs to heads. Each head's weights start random and get nudged during training toward whatever reduces prediction error. What researchers consistently find when they inspect trained models is that heads drift into rough specializations on their own.

Interpretability work has found heads that behave like this:

• Syntactic heads that link verbs to their subjects, or adjectives to the nouns they modify
• Coreference heads that connect pronouns to the nouns they refer to, the "it means the cat" job
• Positional heads that mostly attend to the token immediately before or after, tracking local order
• Delimiter heads that attend to punctuation, quotes, or brackets, tracking structure
• Rare-token heads that latch onto unusual or highly informative words in the input

Two honest caveats. These specializations are tendencies, not clean assignments, and a head that looks like a "pronoun head" on one sentence often does something else entirely on another. And many heads have no clean human interpretation at all. The specialization story is a useful mental model, not a literal blueprint.

How the heads come back together

Each head produces its own output for every token: its own context-aware blend of values. So after multi-head attention, a single token has not one new representation but N of them, one per head.

Two steps merge them.

Concatenate. The outputs are laid end to end into one long list of numbers. Note that each head works with a smaller slice of the model's dimensions, so twelve heads stitched together come back to the size the model started with. Heads split the work rather than multiplying it.

Project. That concatenated list is passed through one more set of learned weights, called the output projection. This is the step where the model learns how to weigh and mix what the different heads found: how much of the coreference head's answer versus the syntax head's answer to keep for this particular token.

The result is a single representation per token, now informed by many different views of the sentence at once. That is what gets handed to the rest of the transformer block, which is the subject of the next lesson.

Reading head counts in model specs

When you read a model card or a technical blog post, you will see numbers like "16 heads" or "32 attention heads per layer". Now you know what that counts: how many independent attention patterns the model computes at each layer.

A few things worth knowing about the number:

• Head count is paired with the model's dimension size. More heads means each head works with a thinner slice, so there is a trade-off between having many views and having each view be rich.
• More heads is not automatically better. Pruning research has repeatedly found that a trained model can lose a good fraction of its heads with little damage, which suggests real models carry redundancy.
• Head count multiplies across layers. A model with 32 layers and 32 heads per layer is running over a thousand attention patterns on your prompt. This is a large part of why inference needs serious hardware.

For a practical user, the takeaway is not to shop by head count. It is to understand that when a model handles a sentence with several overlapping relationships correctly, that competence comes from many heads tracking different things at once rather than one very smart lookup.

Key Takeaways

• A single attention head has one budget of weights summing to 1, which forces unrelated relationships to compete and blend into a mushy average.
• An attention head is a full copy of the query-key-value mechanism with its own learned weights, so each head produces a different attention pattern from the same input.
• Heads run in parallel and tend to specialize on their own during training into roles like syntax, coreference, position, and structure, though the specializations are tendencies rather than clean assignments.
• Head outputs are concatenated and projected through learned output weights, producing one richer representation per token; heads split the model's dimensions rather than multiplying them.
• Head counts in model specs tell you how many attention patterns run per layer, but more heads is not automatically better and trained models carry real redundancy.