> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Idempotent retries
> Retry a call, streaming or not, without starting duplicate inference work or a second charge.
Send an `Idempotency-Key` header when a lost response or a network failure might make you retry. It is optional, it works on streaming and non-streaming requests alike, and it guarantees one generation and one charge. The same header and replay rules apply to `/v1/messages`.
Use a non-blank printable ASCII value of 255 characters or fewer after trimming. One fresh key per logical request, reused only when you are retrying that same request. A UUID generated at the call site is a convenient source.
```python Python theme={"dark"}
response = client.chat.completions.create(
model="nemotron-3-5-lightning-30b",
messages=[{"role": "user", "content": "Write one sentence."}],
max_tokens=64,
extra_headers={"Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY"},
)
```
```typescript TypeScript theme={"dark"}
const response = await client.chat.completions.create(
{
model: "nemotron-3-5-lightning-30b",
messages: [{ role: "user", content: "Write one sentence." }],
max_tokens: 64,
},
{ headers: { "Idempotency-Key": "YOUR_STABLE_IDEMPOTENCY_KEY" } },
);
```
```bash cURL theme={"dark"}
curl https://api.runinfra.ai/v1/chat/completions \
-H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY" \
-H "Content-Type: application/json" \
-H "Idempotency-Key: YOUR_STABLE_IDEMPOTENCY_KEY" \
-d '{"model":"nemotron-3-5-lightning-30b","messages":[{"role":"user","content":"Write one sentence."}],"max_tokens":64}'
```
## What a reused key does
Reuse the same key only with the same HTTP method, route, and request body. A key sent with a different body is a different logical request, and it is refused with `400` `idempotency_mismatch` rather than answered with the wrong reply. If you would rather opt out entirely, omit the header: a request without an `Idempotency-Key` runs with no deduplication at all.
## Retrying a stream
A stream is not replayed byte for byte. We do not store the tokens we deliver, so there is nothing to send again, and a live event stream is never stored for replay. What the key protects on a streaming request is the part that costs money.
| Situation | What you get |
| - | - |
| The original stream is still open | `409 idempotency_conflict` with `Retry-After: 1`. No second generation starts. |
| The original stream finished and settled | `200` with `X-RunInfra-Idempotent-Replay: true` and a non-streaming JSON receipt, or a short event stream on a streaming `/v1/messages` retry. Nothing is regenerated and nothing is billed again. |
| The original delivered nothing and was billed nothing | The key is released, so a retry runs normally, exactly as a request without a key would. |
The receipt is an ordinary chat completion envelope, so an OpenAI-compatible client can parse it. `content` is `null` because the delivered text was never stored. `finish_reason` is `length` or `content_filter` when the original reply was cut short, and `null` otherwise. The replay facts ride in `idempotent_replay`.
```json theme={"dark"}
{
"id": "chatcmpl-8f0a1c2e-...",
"object": "chat.completion",
"created": 1786800000,
"model": "qwen3-8-27b",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": null },
"finish_reason": null
}
],
"usage": {
"prompt_tokens": 1000,
"completion_tokens": 500,
"total_tokens": 1500,
"prompt_tokens_details": { "cached_tokens": 200 },
"cost": 0.000282,
"runinfra": { "cost_microcents": 28200, "cached_input_tokens": 200 }
},
"idempotent_replay": {
"replayed": true,
"streamed": true,
"content_replayed": false,
"regenerated": false,
"billed_again": false,
"original_status": 200,
"stream_outcome": "completed",
"usage_basis": "provider_reported",
"cost_microcents": 28200,
"detail": "The original request with this Idempotency-Key streamed to completion and settled once. Streamed tokens are not stored, so this reply carries the terminal usage and cost instead of the generated text. Nothing was regenerated and nothing was billed a second time."
}
}
```
`X-RunInfra-Idempotent-Replay` is the signal on every route: read it before handing a response to a streaming parser. `/v1/chat/completions` and `/v1/messages` return the `idempotent_replay` object; on `/v1/responses` the reply is converted to a Responses object and only the header survives. A reply the original cut short keeps `status: "incomplete"` and its `incomplete_details.reason` there, as the original stream did. A streaming `/v1/messages` retry receives the receipt as a short event stream with no text.
If you need the generated text after a dropped stream, send a new request with a new key. That is new paid inference. Reusing the old key returns the receipt, not the text.
## Only one charge per key
A retried request is charged once, streamed or not: a duplicate settlement for the same key is refused rather than billed twice. The key holds for the whole time a request can run, so a retry sent while a slow non-streaming request is still generating gets `409 idempotency_conflict` and never starts a second generation.
When a key cannot be checked, a keyed request is refused rather than run without protection. Every endpoint, chat completions, responses and messages included, returns `503 idempotency_unavailable` with `Retry-After` before any inference starts or any charge is made; retry with the same key. On `/v1/messages` the same refusal arrives as Anthropic's `overloaded_error`, which the Anthropic SDKs retry. A request without an `Idempotency-Key` is never affected.
## Client request ids
`X-Client-Request-Id` is an optional correlation header. The gateway echoes it back on the response. It is not a substitute for `Idempotency-Key`. Use a non-blank printable ASCII value of 512 characters or fewer. Any other value is refused with `400` `invalid_trace_header`.
## Related
The frame shapes a retried stream is not replayed as.
The 24 hour replay window is the one place a response body is stored.