List models

Discover the model ids a workspace key can reach, and the limits the gateway enforces on them.

GET https://api.runinfra.ai/v1/models

This is how an application discovers, at boot, which model ids its key can call and what binds them. Never hard-code a list you guessed.

Request

Python

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://api.runinfra.ai/v1",
    api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)
for model in client.models.list().data:
    print(model.id)

Response

{
  "object": "list",
  "data": [
    {
      "id": "nemotron-3-5-lightning-30b",
      "object": "model",
      "owned_by": "runinfra",
      "created": 1786619239,
      "availability": "available",
      "max_request_bytes": 3670016,
      "context_window": 262144,
      "context_length": 262144,
      "max_output_tokens": 262144,
      "pricing": { "input": 0.05, "output": 0.15 },
      "cached_input_price_usd_per_mtok": 0.01,
      "cached_input_price": 0.01,
      "cache_isolation": "isolated",
      "cache_tiers": ["gpu"],
      "cache_retention": "best_effort",
      "max_concurrent_requests_per_api_key": 16,
      "max_tokens_per_minute_per_workspace": 4000000
    }
  ]
}

The numeric values above are an example. Read them from your own response, because they move with capacity.

The example uses Nemotron 3.5 Lightning 30B’s published 262,144-token context and maximum generated output.

Field	Type	Value
object	string	list at the top level, model for each item.
data[].id	string	The id to send in the model field of an inference request.
data[].owned_by	string	runinfra.
data[].created	integer	Unix seconds, the creation time of the underlying record.
data[].display_name	string	The model’s display name, such as Nemotron 3.5 Lightning 30B.
data[].supported_endpoints	string[]	Endpoints that serve this id: chat_completions, responses and messages for a chat model.
data[].modality	string	llm for a chat model.
data[].tool_calling	boolean	true when the model accepts tools.
data[].json_mode	boolean	true when the model accepts a JSON response_format.
data[].input_modalities	string[]	["text"], or ["text", "image"] when the model accepts images.
data[].reasoning_efforts	string[]	reasoning_effort values measured to behave distinctly on this model. Empty when none are declared.
data[].default_reasoning_effort	string or null	The measured default when you omit reasoning_effort. null means none is declared, not that reasoning is off.
data[].max_completion_tokens	integer or null	The same value as max_output_tokens, under the OpenAI request field’s name. null when no output limit is declared.
data[].availability	string	available or paused.
data[].max_request_bytes	integer	Largest request body the gateway accepts. Enforced at the gateway, so it is the same for every model.
data[].context_window	integer	Tokens the model can hold in one request. Present only where a served window has been reported.
data[].context_length	integer	The same served window under the name OpenAI-compatible catalog readers commonly use.
data[].max_output_tokens	integer	Generated tokens allowed in one request. Oversized output budgets are clamped to the model’s remaining context and reported with X-RunInfra-Output-Clamped: true and X-RunInfra-Output-Token-Ceiling.
data[].response_time_ceiling_seconds	integer	Wall-clock seconds one response may run. See Output limits.
data[].pricing	object	Hosted input and output prices in USD per one million tokens.
data[].cached_input_price_usd_per_mtok	number	Hosted cached-input price in USD per one million tokens, when the effective cached tier is active.
data[].cached_input_price	number	The same cached-input price in USD per one million tokens under the short discovery key.
data[].cache_isolation	string	isolated when the active request path separates cache entries by tenant, otherwise shared.
data[].cache_tiers	string[]	Proven storage tiers, such as gpu or host_ram.
data[].cache_retention	string	best_effort when cached prefixes are kept as long as capacity allows and may be evicted under memory pressure, or tiered_host_memory when the proven cache also spans host memory. Both are best effort and no TTL is implied. The human-readable statement is on the model’s page.
data[].max_concurrent_requests_per_api_key	integer	Requests one key may have in flight against this model once the model is busy. A workspace resolves to the same number. While the model has capacity to spare, requests beyond it are admitted; see rate limits.
data[].max_concurrent_plan_requests_per_workspace	integer	Present only while the workspace’s coding plan serves a model it covers. Requests the plan funds, on the plan or on credits after its limits, may run this many at once across the workspace once the model is busy. While the model has capacity to spare the gateway may admit more, so this understates and never overstates the allowance.
data[].max_concurrent_standby_requests_per_workspace	integer	Present beside the field above on a chat model. Standby requests the workspace may run at once, one allowance shared across every model it calls on Standby. 0 means Standby is closed on this model.
data[].max_tokens_per_minute_per_workspace	integer	Tokens per minute one workspace may spend on this model, over a rolling 60 second window.
data[].session_affinity_idle_ttl_seconds	integer	Idle seconds before a session’s replica placement is forgotten. See Session affinity.
data[].paused_until	string	ISO 8601 time availability is next rechecked, not a promised return. Present only when availability is paused.

The fields are additive.

The limit fields exist so you never learn a limit from a 429

They are read from the code that enforces them, so a published limit and an applied limit cannot drift apart. All of them are additive to and disjoint from the fields the OpenAI Model object declares, so a client typed against the OpenAI SDK keeps parsing this response and ignores what it does not recognize.

Absent means undeclared, never zero and never unlimited

A field appears only where the limit it names is actually enforced for that model. A model that has not reported a served window carries no context_window rather than borrowing another model’s number.

Bytes can bind before tokens

max_request_bytes is a byte ceiling, not a token ceiling, and the two bind independently. For a very long prompt the byte ceiling is often the one that binds first, whatever the context window says.

Prices and cache behavior

Hosted model prices are returned here for discovery and repeated on each model’s page in the Model Library. The page also carries the human-readable cache eviction and retention statement, served precision when verified, capabilities, and the recommended output budget.

Available and paused

A paused model keeps its place in the list. It is a real configured model that is temporarily not serving, so poll this endpoint or retry the call itself to detect it coming back. The id does not change across a pause. While paused, the row omits pricing, the cached-input price fields, max_concurrent_requests_per_api_key and max_tokens_per_minute_per_workspace.

Retrieve one model

curl https://api.runinfra.ai/v1/models/nemotron-3-5-lightning-30b \
  -H "Authorization: Bearer $RUNINFRA_GATEWAY_KEY"

Returns a single object with the same fields and no list wrapper. An id your key cannot reach returns 404 model_not_found. URL-encode any id containing a slash.

Which models appear

A workspace key lists every public hosted model, available or paused.

Authentication and rate limits

The endpoint needs a workspace API key, sent as Authorization: Bearer or x-api-key. It uses that key’s requests-per-minute limit and returns X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset and X-RateLimit-Tier. Listing models reserves no inference credit, but it is still an authenticated request against your per-minute budget, so do not call it in a hot loop.

Use the returned id exactly as it appears. It is the supported name for the model field.

Related

Chat completions

Send one of these ids as the model field.

Authentication

The key that decides which ids you see.

Rate limits

The four layers that can refuse you.