Capture evidence before retrying
Section titled “Capture evidence before retrying”Record UTC time, endpoint, public model ID, HTTP status, request ID if returned, and whether any output arrived. Keep a sanitized error summary. Do not record API keys, authorization headers, private prompts or full user payloads in shared logs.
Use model discovery to confirm the route and protocol selection to check its request shape. A media task status and an LLM HTTP error require different recovery paths.
| Observation | First check | Next action |
|---|---|---|
| 400 or validation error | Required fields, model-specific parameters, context/output limits | Correct the request before another attempt |
| 401 | Missing or invalid credential and required auth header | Fix credentials; do not retry unchanged |
| 403 | Key permissions or access policy | Confirm access with the account owner |
| 404 | Public model ID, protocol endpoint and final SDK URL | Check discovery and avoid duplicated /v1 |
| 429 | Rate limit, account limits and returned error details | Reduce concurrency; honor Retry-After when present |
| 5xx | Sanitized error and request ID | Investigate service/upstream failure; use bounded recovery |
| Timeout or broken stream | Whether output or usage was already produced | Treat completion as uncertain, not automatically failed |
The table is a diagnostic starting point. Read the actual native error body; the same status can have several causes. A 429 is not proof that buying credit resolves the problem.
Bound retries at the application boundary
Section titled “Bound retries at the application boundary”Retry only failures your application has classified as transient. Use exponential backoff with jitter, a small maximum attempt count, an overall deadline and a total cost budget. Stop at whichever bound is reached first. Respect Retry-After when supplied; do not promise that every response contains it.
Check SDK retry defaults before adding an outer retry loop. Nested retries multiply requests. Do not silently switch models on failure: changed capabilities and output behavior can invalidate the user’s task or your evaluation.
A timed-out generation may still have run. This guide does not assume idempotency support for LLM POST endpoints, or that resending the same body deduplicates usage. After partial output, preserve the partial result and ask the application/user to choose recovery rather than automatically replaying it.
Prepare an actionable support report
Section titled “Prepare an actionable support report”Include a minimal sanitized request shape, endpoint, model ID, timestamps with timezone, status, request ID, and whether the issue reproduces. Exclude credentials and customer content. Report usage as observed, missing or pending; do not mark absent receipts as zero cost.
For a newly integrated application, return to the minimal first request. For errors only during streaming, check event parsing and completion. Review cost accounting before approving a larger retry experiment.