An LLM bill can rise even when user traffic looks stable. The reason is that request count is only one variable. Model choice, prompt size, output length, retries, caching behavior and agent loops can all change the cost of each unit of traffic.
1. The model mix changed
Moving a workload from a smaller model to a more capable model may improve quality, but it can change unit cost immediately. If routing happens automatically or model names are configured in multiple applications, the change may be hard to see from the provider invoice alone.
Track cost and request volume by model so a mix shift becomes visible as soon as it happens.
2. Prompts or context became larger
Input tokens grow when system prompts expand, more conversation history is retained or retrieval adds more documents. That can increase cost without changing the number of requests.
Measure input tokens per request over time. A gradual increase often points to context accumulation; a sudden step change often points to a prompt or application release.
3. Outputs became longer
Longer responses create more output tokens and can also increase user-facing latency. Changes in max-output settings, prompting style or model behavior can alter output length.
Track output tokens independently from input tokens. Combining them into one total hides which side of the request changed.
4. Retries and fallback multiplied calls
Retries improve resilience, but every retry is another provider call. A degraded provider can therefore produce both a reliability incident and a spend incident.
Correlate retry count, failed requests and provider latency with cost. If spend rises at the same time as error rate, reliability may be the root cause.
5. Cache behavior changed
Prompt caching can materially change the economics of repeated prefixes. OpenAI exposes prompt-cache controls and Anthropic prices cache writes and cache hits differently from normal input tokens.
A cache miss is not automatically a problem, but a falling cache-hit rate can explain why unit cost changes for workloads with repeated context. Track cached and uncached token behavior where the provider exposes it.
6. Agent loops created hidden work
An agent can execute multiple model calls for one user-visible task. A planner may retry, call tools, reflect on results and invoke another model.
For agentic workloads, cost per end-user request is not enough. Trace the full workflow and measure calls per run, tokens per run and cost per successful outcome.
The investigation pattern
Start with cost per request, not total spend. Segment by application, model and provider. Then compare tokens, retries and latency for the period where cost changed.
The goal is to turn a billing surprise into an operational diff: what changed, where it changed and what the expected trade-off is if you modify it.
CLYVEL
Put the operating model into practice.
Clyvel connects production AI traffic, cost, reliability and governance in one operations layer.
See Clyvel cost visibilitySources and further reading
Clyvel Research uses primary technical and vendor references wherever a claim benefits from external context.
Read the research methodology