What is a token in AI? Why tokens affect cost and limits
2026-11-10
Definition
A token is a small chunk of text (the idea comes from tokenisation), often a short word or part of a longer word, that an AI model uses as its basic unit for reading your input and writing its output.
Why tokens matter
Tokens decide two practical things. First, cost: most AI services that you reach through an API charge by the number of tokens in and out, so long prompts and long answers cost more. Check current pricing with the provider, since it changes. Second, limits: a model can only handle so many tokens at once, which is its context window.
As a rough guide, a page of ordinary English is a few hundred tokens. Other languages, tables and code can use more. Trimming what you send, such as removing signatures and repeated email threads, saves money and often improves answers from the large language model. For measuring spend against results, see how to measure AI ROI.
A hypothetical example
Imagine a recruitment firm that sends every full CV, plus a long instruction, to a model for each applicant. They trim the instruction and send only the relevant CV sections. The monthly usage bill falls because fewer tokens go in each time.
Budgeting for usage like this is part of AI strategy.