How AI Predicts the Next Word

Riya opens an AI tool and types her request. In a few seconds, she will have her email. But the AI does not write the whole email at once. It predicts one word at a time.

How does AI read your prompt?

First, the prompt is cut into small pieces called tokens. AI only understands numbers, so each token becomes a long list of numbers that captures its meaning. This is called an embedding. Similar words get similar numbers. That is how the AI knows Monday is close to Friday, and nowhere near banana.

But a word alone can mean many things. So each word looks at the other words to work out its meaning, and its numbers get updated. This is called attention. It repeats many times, and each round builds a deeper understanding of what Riya wants.

How does it pick the next word?

The AI scores every token it knows for the next word. These scores are called probabilities, and Monday scores the highest.

But it does not always pick the top word, or it would sound robotic. It picks from the top options with a little chance. This is called sampling, and it is why the same prompt can give different answers. The chosen word is added, and everything runs again for the next word. That is why replies appear word by word.

What drives all of this?

Numbers inside the model called weights, or parameters. During training, the AI guessed the next word billions of times, and every wrong guess adjusted these weights a little. Over time, the weights learned patterns, like which words usually follow which and how an email is usually written. So when Riya types her request, the AI is matching it against these patterns.

What to remember

Riya's email was written one prediction at a time. This is what every LLM does, whether it is ChatGPT, Claude or Gemini. It is not magic. It is prediction, at a scale that feels like magic.