The old shape was a chain

RNNs process text as a chain.

t1 -> state -> t2 -> state -> t3 -> state -> t4

That means token 4 cannot be fully processed until token 3 has updated the state.

Token 3 waits for token 2.

Token 2 waits for token 1.

The model is always carrying one running summary forward.

The chain creates an order dependency: later work waits for earlier work.

Self-attention changes that shape.

Self-attention starts with all token vectors

Before the attention step, the model already has a vector for each token.

For a short input:

the cache key changed

the model has:

the     -> vector
cache   -> vector
key     -> vector
changed -> vector

Those vectors exist at the same layer before attention runs.

That matters.

Because every token vector is already available, the model can compare token pairs directly.

the     compared with cache
the     compared with key
cache   compared with changed
changed compared with key

It does not have to move through the sentence one token at a time to make those comparisons.

The comparison becomes a table

Self-attention can be pictured as a table.

Rows are the tokens asking for context.

Columns are the tokens being considered.

          the  cache  key  changed
the        .     .     .      .
cache      .     .     .      .
key        .     *     .      .
changed    .     *     *      .

The table is the important idea.

Each row can score the columns.

For changed, the useful columns might be:

cache
key
changed

For key, the useful column might be:

cache

self-attentiontokens score other tokens from the same input

Why this can run in parallel

Computers, especially GPUs, are good at doing large batches of similar math.

Self-attention turns token comparison into matrix operations.

Roughly:

all token vectors -> all queries
all token vectors -> all keys
all queries x all keys -> score table

That score table can be computed as one large operation.

The model is not asking:

finish token 1, then token 2, then token 3

It is asking:

compute many token-pair scores together

That is why self-attention fit GPUs so well.

Look at every token does not mean free

There is a tradeoff.

If there are 4 tokens, the score table has 16 possible token pairs.

4 tokens -> 4 x 4 scores

If there are 1,000 tokens, the full table has about 1,000,000 possible pairs.

1,000 tokens -> 1,000 x 1,000 scores

So self-attention gives a direct path between tokens, but that path costs compute and memory.

Self-attention is easier to parallelize than an RNN chain, but full attention grows with the number of token pairs.

That is one reason long-context transformers can become expensive.

Causal models still use a mask

There is one detail that can be confusing.

Decoder language models should not look into the future when predicting the next token.

So during language modeling, self-attention uses a causal mask.

For token 4:

can look at: token 1, token 2, token 3, token 4
cannot look at: token 5, token 6, ...

The architecture still computes attention inside the same sequence.

The mask just hides future positions.

allowed past/current tokens -> visible
future tokens               -> blocked

So when people say self-attention can look at every token, read it carefully:

every allowed token in the attention window

For bidirectional models, that can mean left and right context.

For decoder LMs, it means the past and current position.

Why this helped transformers replace RNNs

The old recurrent shape was:

state must travel through every step

The self-attention shape is:

tokens compare through a matrix

That helped in two ways.

First, information had shorter routes.

If token 100 needs token 7, attention can give it a direct score.

Second, training could use parallel hardware better.

Many token representations and token-pair scores can be computed together.

That does not make generation fully parallel. Decoder LMs still generate one new token at a time.

But it made training and prompt processing much more parallel than recurrent sequence processing.

The mental model

Think of an RNN as a relay race.

Each token passes one note to the next token.

Think of self-attention as a shared table.

Every token places a card on the table.

Then each token can scan the table and decide which cards matter.

The table scan can happen with broad matrix math instead of one handoff at a time.

That is the core reason self-attention can look across the sequence.

Quick check

Why can self-attention compare many tokens at once?

A. Each token vector is available before attention, so token-pair scores can be computed as a matrix
B. The model deletes all earlier tokens
C. The model stores only one hidden state
D. The model uses hand-written grammar rules

The best answer is A.

Self-attention changes the work from a step-by-step chain into a table of token comparisons.

Next step

Self-attention gives tokens direct routes to each other.

Next: What Is a Transformer?

That article puts self-attention inside the larger block: attention, feed-forward layers, residual connections, normalization, and stacking.