Token
Also called: tokens
A token is the unit AI models actually read and write — a chunk of a word, roughly three-quarters of an English word on average.
A token is the unit AI models actually read and write — a chunk of a word, roughly three-quarters of an English word on average. It matters for two practical reasons: models are priced per token, with output typically costing several times more than input, and every model has a hard ceiling on how many tokens it can consider at once — the context window.
Why it matters: Per-call costs look adorable and compound brutally. A fraction of a cent per call is real money at ten thousand calls a month, and a multi-step agent task can quietly burn tokens through retries you never see. The classic overspend: pointing a frontier-priced model at high-volume lookup work — reading a known field off a known form — that a model a tenth the price, or a plain rule, handles perfectly. That’s how a forty-dollar idea becomes a six-hundred-dollar monthly bill.