Token
The small unit of text an AI language model actually reads and writes. Rather than whole words, models break text into tokens — often a word, part of a word, or a punctuation mark — and process one at a time. Token counts are how usage is measured and priced.
When you type a sentence into a language model, it does not see letters or tidy words. It first chops the text into tokens: chunks that are usually a short word, a fragment of a longer word, or a mark of punctuation. The word “understanding” might become two or three tokens, while “cat” is just one. This step is called tokenisation, and it is how the model turns messy human text into a sequence it can handle.
Tokens matter for practical reasons as much as technical ones. A model generates its answer one token at a time, and everything it can hold in mind at once — your question plus its reply — is measured in tokens, not words. That limit is called the context window. When an AI service quotes a price or a usage cap, it is almost always counting tokens.
A rough rule of thumb for English is that a token is about three-quarters of a word, so a thousand tokens is roughly 750 words. It is a small idea, but it sits underneath almost everything a language model does.