Total AI spend tells finance how large the bill is. Cost per request tells operators how efficiently a workload is behaving. The metric is simple in concept, but production calculation needs model-specific pricing, token categories, retries and workload attribution. Used correctly, unit cost makes prompt, routing and reliability changes visible long before the monthly invoice explains them.
Key takeaways
- Calculate provider cost at the individual call first, then aggregate to the logical request, agent run or business task.
- Separate input, output and cached token categories when pricing treats them differently.
- Compare unit cost with success and business outcome; the cheapest request is not automatically the most efficient request.
The basic unit-cost model
At a high level, provider cost is the sum of each billable usage category multiplied by that model's rate. For token-priced models, that normally means calculating input and output usage separately and including cached or other categories when the provider exposes different rates.
Store the pricing version or effective rate used by the calculation. Provider pricing changes over time, so a financial report should be reproducible rather than silently recalculating historical traffic against whatever rate happens to be current today.
One user request may contain many model calls
Agentic workflows break the assumption that one application request equals one provider request. A single task can create planning calls, tool-selection calls, retries and a final generation. Measuring each provider call is necessary, but unit economics often needs the sum across the logical task.
Keep a stable run or trace identifier so provider-call costs can roll up into cost per agent run, cost per successful task or another unit that matches the product's actual value.
Retries and fallback belong in the unit cost
If a provider timeout causes two retries before a successful response, the success did not cost the same as a clean first attempt. Reliability overhead should remain attached to the original logical operation.
This is where cost and reliability data become more valuable together. A sudden unit-cost increase may be a prompt problem, a routing change or a provider degradation that multiplied attempts.
Segment cost by ownership
Aggregate unit cost by application, agent, environment, team or customer workflow. A global average can hide one runaway workload while making every team look slightly more expensive. Ownership dimensions turn the metric into something an operator can act on.
Pair cost per request with volume. A very expensive low-volume workflow may be acceptable; a tiny unit-cost regression on a high-volume path can create the larger financial problem.
Move from cheapest call to best economics
Model selection should not optimize for token price in isolation. A cheaper model can create more retries, longer workflows or lower task success. The useful comparison is cost for the outcome the system needs.
Track cost per successful request or task alongside latency, reliability and quality signals when they exist. That shifts AI FinOps from cutting spend to improving the economics of production AI.
FAQ
Common questions
How do you calculate LLM cost per request?
Calculate each model call from the provider's applicable usage categories and rates, then roll up all calls that belong to the same logical request or task. Include retries and fallback attempts rather than counting only the successful call.
Why is cost per successful request useful?
It connects spend to reliability. A cheaper model or route may not be more efficient if it increases failures, retries or the number of calls required to complete a task.
CLYVEL
Put the operating model into practice.
Clyvel connects production AI traffic, cost, reliability and governance in one operations layer.
Explore Clyvel FinOpsSources and further reading
Clyvel Research uses primary technical and vendor references wherever a claim benefits from external context.
Read the research methodology