The unit a language model actually reads and writes: not a word, not a letter, but a chunk of text chosen by a compression algorithm, so 'cat' is one token and 'unbelievable' might be three. English averages about three-quarters of a word per token. Models are priced, limited and benchmarked by the token, and their strangest failures, counting letters for instance, come from it.
I remember the first time I pasted text into a tokeniser demo and watched it colour the fragments. My surname was three pieces, none of them pronounceable. It explained a great deal about how the models spelled it.
Sighted on every AI pricing page since 2020: dollars per million tokens, input and output priced differently, with a calculator for people who cannot picture a million of anything. The record notes that the token became a currency before most users could define one.
The tokeniser is a dictionary of common fragments built before training, by repeatedly merging the most frequent pair of characters until the vocabulary hits a target, around fifty thousand for GPT-2 and two hundred thousand for later models. The model never sees letters. It sees numbers that stand for fragments.
The correction the record makes most often: tokens are not words. A number can be several tokens, a space can belong to the next word, and other languages pay more tokens for the same sentence than English does. When a company says a model read ten trillion tokens, divide by about 1.3 for words.
What does it mean? Write it the way you would say it out loud.
