> ## Documentation Index
> Fetch the complete documentation index at: https://runinfra.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# Tool calling
> The model picks a tool and generates typed arguments, you run it, you feed the result back.
The model decides which of your registered tools to invoke and generates typed arguments for it. You execute the tool and hand the result back, and the model continues. Use it whenever the model needs to act on the world: call an API, query a database, run a calculation, fetch a document.
Tool calling is a per-model capability. Check the capability listing on the model's page before sending `tools`. A model that does not list tool calling refuses the request with `400` `hosted_capability_not_supported`.
## Minimal code
```python Python theme={"dark"}
import json
import os
from openai import OpenAI
client = OpenAI(
base_url="https://api.runinfra.ai/v1",
api_key=os.environ["RUNINFRA_GATEWAY_KEY"],
)
tools = [{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the current weather for a city",
"parameters": {
"type": "object",
"properties": {"city": {"type": "string"}},
"required": ["city"],
},
},
}]
def get_weather(city: str) -> dict:
return {"city": city, "temp_c": 21, "conditions": "partly cloudy"}
messages = [{"role": "user", "content": "What's the weather in Paris?"}]
for _ in range(10):
response = client.chat.completions.create(
model="nemotron-3-5-lightning-30b",
messages=messages,
tools=tools,
max_tokens=16384,
)
msg = response.choices[0].message
messages.append(msg)
if not msg.tool_calls:
print(msg.content)
break
for call in msg.tool_calls:
args = json.loads(call.function.arguments)
result = get_weather(**args)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": json.dumps(result),
})
```
```typescript TypeScript theme={"dark"}
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://api.runinfra.ai/v1",
apiKey: process.env.RUNINFRA_GATEWAY_KEY,
});
const tools = [{
type: "function" as const,
function: {
name: "get_weather",
description: "Get the current weather for a city",
parameters: {
type: "object",
properties: { city: { type: "string" } },
required: ["city"],
},
},
}];
function getWeather(city: string) {
return { city, temp_c: 21, conditions: "partly cloudy" };
}
const messages: any[] = [
{ role: "user", content: "What's the weather in Paris?" },
];
for (let turn = 0; turn < 10; turn += 1) {
const response = await client.chat.completions.create({
model: "nemotron-3-5-lightning-30b",
messages,
tools,
max_tokens: 16384,
});
const msg = response.choices[0].message;
messages.push(msg);
if (!msg.tool_calls?.length) { console.log(msg.content); break; }
for (const call of msg.tool_calls) {
if (call.type !== "function") continue;
const args = JSON.parse(call.function.arguments);
const result = getWeather(args.city);
messages.push({
role: "tool",
tool_call_id: call.id,
content: JSON.stringify(result),
});
}
}
```
## What one turn actually looks like
## What to tune
| Parameter | Effect |
| - | - |
| `tool_choice: "auto"` | The model chooses when to call a tool. This is the default. |
| `tool_choice: "required"` | Force a tool call every turn, no free-form answer. Refused with `400` on `nemotron-3-5-lightning-30b`; force a named tool there instead. |
| `tool_choice: {type:"function", function:{name:"..."}}` | Force one specific tool. |
| `parallel_tool_calls: false` | Ask for one call at a time. Whether it is honored depends on the model. |
## Common mistakes
* **Dropping the assistant turn.** Push the whole assistant message, `tool_calls` and all, before you push any tool result. Without it the tool message has nothing to attach to.
* **Returning a non-string tool result.** The `content` on a tool-role message must be a string. Always serialize it.
* **A vague `description`.** Write it as though the model has never seen your API: what the tool does, what each argument means, what a good input looks like. This is the single biggest lever on whether a tool gets called at all.
* **No turn limit.** Bound the loop. Ten turns covers almost every pattern, and an unbounded loop can spend real money.
* **Trusting the arguments.** The model can invent a field or omit a required one. Validate with Pydantic or Zod before you execute.
## Next steps
When you want JSON back without a tool loop.
Stream the assistant turn as it arrives.
The full request contract.