Token Cost Analysis for LLMs That Holds Up

A monthly model invoice rarely explains where an AI programme is becoming economically unsound. It reports consumption after the fact, often aggregated across products, teams and workloads with radically different value profiles. Token cost analysis for LLMs is the discipline of connecting that consumption to workload design, model behaviour and business output before spend becomes an architectural constraint.

Free tool: use our LLM cost calculator to compare monthly API costs across current models using your own numbers.

For enterprise operators, the relevant question is not whether a model is cheap per million tokens. It is whether a given execution path produces an acceptable outcome at a predictable unit cost, under real traffic and governance conditions. A lower-priced model that requires repeated retries, excessive context or human correction can be materially more expensive than a premium model used selectively.

Why provider price cards are an inadequate cost model

Published input and output token rates are necessary reference points, but they are not a cost model. They omit the factors that determine the effective cost of an autonomous execution layer: prompt expansion, retrieval payloads, conversation history, structured output overhead, tool loops, retries and fallback calls.

The basic calculation is straightforward:

`Request cost = (input tokens x input rate) + (output tokens x output rate) + auxiliary execution costs`

The difficulty lies in estimating each term honestly. Input volume is not simply the user prompt. In a retrieval-augmented system, it includes system instructions, policy text, retrieved passages, metadata, prior turns, tool schemas and formatting tokens. Output volume can increase sharply when the model is asked to expose reasoning-like intermediate artefacts, return verbose JSON or generate citations that are not operationally required.

Auxiliary costs matter most in agentic systems. A single user request may invoke a classifier, a retrieval model, a primary reasoning model, several tools and a verifier. If a tool response is then injected back into context, its contents become billable input on the next model call. The apparent cost of one interaction may therefore be the product of several dependent decisions rather than one inference event.

Price-card comparisons also disguise an asymmetry: output tokens often carry a higher rate than input tokens. Teams that optimise only prompt compression can miss the larger leakage source. A response constrained from 1,500 tokens to 400 may create more savings than removing several hundred tokens of static instructions, provided answer quality remains intact.

Token cost analysis for LLMs starts with workload distributions

Averages are useful for finance reporting and dangerous for system design. Most production workloads have a long tail. The median customer-support interaction might be inexpensive, while a small percentage of complex cases consume expanded retrieval context, generate several tool calls and trigger retries. That tail can dominate the monthly bill.

Build the analysis around a workload taxonomy rather than a model catalogue. Separate high-frequency transactional tasks from research workflows, document extraction, coding assistance, internal knowledge retrieval and autonomous process execution. Each category has a different quality threshold, token distribution and tolerance for latency.

For every workload, record the request count, median and p95 input tokens, median and p95 output tokens, model route, retry rate, tool-call count and human escalation rate. The most useful derived metric is cost per successful business outcome. Depending on the workflow, this may be cost per resolved case, qualified lead, completed document review or accepted software change.

This reframes optimisation. A model route costing £0.12 per request may be preferable to one costing £0.04 if it reduces escalation from 18 per cent to 4 per cent. The comparison must include the fully loaded cost of human intervention, delay and error exposure. It depends on the business process, but treating every token as equally valuable is a reliable route to false savings.

Measure the context tax

The context tax is the recurring token burden that accompanies the actual task. It includes permanent system prompts, duplicated policy instructions, large tool definitions and retrieved material that is present but unused. In many enterprise deployments, this tax exceeds the user-provided content by a wide margin.

Instrument the prompt as components, not as a single token total. A trace should show how much context came from instructions, memory, retrieval, tools and the user. This makes waste diagnosable. If retrieval contributes 70 per cent of input volume but has little effect on answer acceptance, the problem may be chunking, ranking or the decision to retrieve at all.

Cache-aware architecture changes this calculation. Where provider semantics permit it, stable prompt prefixes and repeated document context can reduce effective input cost. But cache savings should be treated as conditional, not assumed. They depend on request similarity, cache lifetime, provider implementation and the degree to which dynamic metadata disrupts prefix reuse.

Price the full execution path, not the model call

A useful operating model separates direct inference costs from orchestration costs and failure costs. Direct inference is the provider charge for tokens. Orchestration covers vector search, reranking, tool infrastructure, observability, queues and any supporting model calls. Failure cost captures retries, fallbacks, manual review and downstream remediation.

This distinction is especially important for multi-model architectures. Routing a simple request to a smaller model can lower direct inference spend, but a poor router can create expensive recoveries. Conversely, a deliberately conservative router may send more traffic to an expensive model while reducing aggregate cost by preventing failed completions.

Scenario analysis should test at least four conditions: baseline traffic, peak concurrency, long-context outliers and degraded-provider operation. The last condition is regularly excluded from budget models despite being commercially relevant. If a primary provider is unavailable or throttled, the fallback route may have a different tokeniser, output behaviour and price structure. Resilience has a token budget.

Model tokenisers also complicate direct comparisons. The same source document can produce different token counts across model families, particularly where it contains code, tables, legal text or non-English material. Benchmarking should use representative production samples, not a generic character-to-token assumption. A pricing comparison without measured tokenisation is an estimate of an estimate.

Currency exposure belongs in the analysis for UK operators. Many providers invoice in US dollars, while business-unit budgets and value realisation are assessed in pounds sterling. At meaningful scale, foreign-exchange movement can obscure whether an optimisation has genuinely improved the unit economics. Track native provider costs and a consistent internal reporting currency separately.

Optimise allocation before compression

Prompt compression is often the first response to rising spend because it is visible and relatively easy to implement. It should not be the first question. The highest-return intervention is usually better allocation: deciding when to retrieve, which model should act, how much context is justified and when a workflow should stop.

A disciplined sequence is to remove unnecessary calls, reduce repeated context, constrain output formats, route by task complexity and then tune prompt wording. This ordering protects quality. Aggressive compression can remove policy constraints or evidence needed for grounded answers, raising the more costly risk of incorrect execution.

Output constraints deserve particular scrutiny. Define schemas that contain only fields consumed by downstream systems, set credible maximum lengths and avoid generating explanatory prose that no user or process reads. However, hard caps can cause partial outputs or brittle structured responses. Evaluate task completion rates alongside token savings rather than treating shorter as inherently better.

For retrieval systems, improve selection rather than indiscriminately shrinking chunks. A compact set of highly relevant passages can outperform a larger context window filled with marginal evidence. The objective is not minimum token count. It is maximum decision-relevant information per token.

Establish token governance as a product control

Token budgets should exist at three levels: per request, per workflow and per business unit. Request limits prevent runaway loops. Workflow limits reveal whether an ostensibly useful capability is consuming disproportionate resources. Business-unit allocation creates accountability without forcing teams to abandon experimentation.

Governance should also distinguish experimentation from production. Early pilots often tolerate generous contexts and frequent model retries because the priority is learning. Production services need explicit cost envelopes, version-controlled prompts, release gates for material token changes and alerts when p95 consumption moves beyond its expected band.

The executive dashboard should not lead with total spend. It should show cost per successful outcome, cost variance by workflow, model-route mix, retry and escalation rates, and the share of tokens attributable to static context. These measures identify whether a rise in cost reflects healthy adoption, degraded system behaviour or architectural drift.

The strategic value of token cost analysis is not austerity. It is the ability to direct compute token budgets towards decisions and processes where model capability changes the economics of the business. Teams that can make that distinction will retain room to scale, even as autonomous systems make their execution paths more complex.