💬Token (AI)
Also known as Token, LLM Token
A small piece of text, often part of a word, that a model reads.
Think of it like
Think of it like a Lego brick of text. Words are built from one or more of them.
Example
The words 'unbelievable cats' might become four tokens: 'un', 'belie', 'vable' and ' cats'. A 1,000-word English email is about 1,300 tokens.
Why it matters
Cost, speed and how much a model can read at once are all measured in tokens, not words.
Definition
Common mistake
Key takeaways
- 1A piece of text. Word, part of a word, or sign.
- 2About four letters. In English, on average.
- 3The unit of cost. Limits and prices use it.
Related terms
Learn it properly
Tokens and TokenizersLLMs · Lesson 2 of 15Prompt Engineering BasicsPrompt Engineering · Lesson 1 of 16Claude Code and Its SurfacesClaude Code · Lesson 1 of 16AI Model Names and SizesAI Models and Providers · Lesson 6 of 13Chat with Your DocumentsAI Assistants and Chatbots · Lesson 4 of 13Context, Recap and CopyClaude Code Commands · Lesson 5 of 17