AI is billed per token
Most AI services don't bill per request but per token – the small text building blocks from module 2. And in both directions: for the tokens you put in (the prompt) and the ones the model puts out (the answer). So longer prompts and longer answers cost more. A single request often costs only fractions of a cent – the lever is in the volume.
The four big cost levers
Four factors determine the cost: the length of the input (how much context you send along), the length of the output (how verbose the model answers), the model choice (a larger or "reasoning" model costs more per token, see module 18), and the volume (requests per day times tokens per request). Multiplied together, that's the real bill.
Why a cheaper model can win at scale
At low volume, the price difference between models barely matters. At scale it flips: for anyone processing hundreds of thousands of similar requests a day, every cent of difference per request multiplies. Then the smallest model that reliably solves the task often wins – not the most powerful one (related to the fine-tuning consideration in module 16).
Levers to lower it
Concretely, costs can be lowered through shorter, more precise prompts, the right model per task (not the biggest everywhere), caching recurring content, and skipping a reasoning model where the task doesn't need one (module 18).
Why this matters for you as a decision-maker
A per-request price that looks tiny becomes a meaningful line item at scale. The real cost levers aren't a side note but design decisions: model choice and prompt design drive the bill more than the list price per token. That's exactly why it's worth factoring them in early – not only when the first big invoice arrives.