Estimate LLM cost and reconcile usage

Separate an estimate from a charge

A provider’s published list price, a public platform reference and your account’s effective price are different facts. Check your current account pricing and model contract before budgeting. Do not treat a documentation example as a binding quote or assume every model bills every token category in the same way.

For a simple rate expressed per million tokens, an estimate is:

estimated cost = input_tokens / 1,000,000 × input_rate
               + output_tokens / 1,000,000 × output_rate

This is a conceptual formula, not a platform billing implementation. Use the actual unit, currency and price rules for the route. If cache reads, cache writes or reasoning have separate treatment, apply the documented categories without double counting overlapping usage. Never assume that missing fields mean free tokens.

Plan the workload budget

Measure input sizes and output needs on a small representative sample. An average alone can hide long-tail requests; include a high-end case. Set model-specific output limits only when supported, an application request limit, and a concurrency bound. More retries, longer histories and multi-step tool loops all add work.

Conversation history is usually sent again by the client when using a stateless request pattern. Trimming irrelevant history can reduce input size, but validate that the task still has necessary context. Do not assume automatic caching or a discount without evidence from the selected route and account pricing.

Keep an audit trail

Field	Suggested use
Application operation ID	Group attempts belonging to one user task
Native request ID, if returned	Correlate a support investigation
Public model ID, endpoint and timestamp	Identify the contract used
Native usage object	Preserve categories for later reconciliation
Completion state and retry count	Include failed or interrupted attempts
Observed final charge and price basis	Compare actual spend with the estimate

Keep credentials and private content out of this record. If a stream disconnects before usage arrives, mark the usage unknown and reconcile with account records; do not create a zero-cost entry. The same applies to a client timeout whose server outcome is uncertain.

Compare accepted work

For a fixed evaluation sample, divide total observed experiment spend by the number of tasks that pass the predeclared rubric. Include retries and failures in spend. If no task passes, cost per accepted task is undefined, not zero. Report incomplete receipts separately rather than hiding them in an average.

Use the evaluation guide to define acceptance, streaming guide to retain usage safely, and retry guide to avoid uncontrolled experiments. No prices or savings percentages are asserted here.