A JSON-first reference across nine provider APIs — OpenAI (Chat Completions and Responses), Anthropic, Google Gemini, Ollama, Mistral, Cohere, AWS Bedrock, Groq, and xAI. It first draws the practical line between what your agent runtime does and what the model provider does, then shows the literal request and response shapes each provider sends and receives. The examples and claims are cited to the provider's own documentation.
A language model can only do one thing: read a sequence of tokens and emit a sequence of tokens. It cannot query a database, send an email, or read a file. "Tool calling" (also called "function calling") is the convention by which a model emits a structured request to run a named function with named arguments, and the application — not the model — runs it. The model never executes anything. It only ever asks, in JSON, and then waits for the answer to come back in the next request.
Sources for the above: Anthropic states it directly — “The model never executes anything on its own. It emits a structured request, your code (or Anthropic's servers) runs the operation, and the result flows back into the conversation” (platform.claude.com · How tool use works · The tool-use contract). OpenAI: a function definition “allows the model to pass data to your application, where your code can access data or take actions,” and “when the model calls a function, you must execute it and return the result” (platform.openai.com · Function calling). Google: “Instead of generating text responses, the model determines when to call specific functions and provides the necessary parameters to execute real-world actions” (ai.google.dev · Function calling).
Every provider in this paper implements the same three-part contract, even though the field names differ. First the application declares the available tools (a name, a description, and a JSON Schema for the arguments). Then, when the model decides a tool is needed, it emits a tool call in its response — and signals, via a stop/finish reason, that it is pausing to wait for a result. Finally the application executes the function and returns the result as a new message, after which the model either calls another tool or writes the final answer. The differences between providers are entirely in the shape of those three JSON payloads and in who is expected to drive the loop.
Sources: OpenAI · Function calling · Anthropic · Tool use overview · Google · Function calling
The single most important idea in tool calling is the line between the two halves of the system. On the left is your agent runtime — your process, holding the conversation history, the tool code, and the loop. On the right is the model provider — a model provider endpoint that, given the current conversation context and tool catalogue, returns the next step. Most chat-style APIs expect your runtime to send the relevant history each turn; some Responses-style APIs can instead reference provider-stored previous responses. Either way, the model decides what to call; your runtime decides whether and how to run client tools. Secrets, side effects, and local tool execution stay on your side of the line.
messages + tools for this one request.
Sources: OpenAI · Handling function calls
· Anthropic · Handle tool calls
(“the Claude API integrates tools directly into the user and assistant message structure”)
· AWS · Bedrock client-side tool use.
On server-side response storage (the “may store response context” note):
OpenAI · Conversation state
(Responses API defaults to store: true and continues a turn via previous_response_id)
· xAI · Function calling (continues via previous_response_id).
Tool calling is a loop, not a single round trip. The runtime keeps re-sending the growing conversation, or a provider-supported previous-response reference, until the model returns an answer with no further tool calls. The sequence below is the common shape; later sections show the provider-documented JSON each provider uses at each numbered step.
The runtime POSTs the conversation plus the tool catalogue, or a provider-supported reference to prior response context plus any new input.
The model either writes text or emits a structured tool call — a tool name plus JSON arguments — and sets a stop/finish reason that means “I'm waiting for a result” (e.g. OpenAI tool_calls, Anthropic tool_use, Bedrock tool_use).
The runtime parses the arguments and runs the actual function. For developer-defined (client) tools this is the only step that touches your systems, and it runs on your side of the line; some providers also offer built-in tools that they execute server-side.
The runtime sends the result back as a new message or input item (a tool role, a tool_result block, a functionResponse part, a function_call_output item, …).
The model either produces the final answer (stop reason “end”) or requests another tool, in which case the runtime returns to step 3. The loop is owned by the runtime, not the provider.
Sources: OpenAI · worked example loop · Anthropic · How tool use works · Gemini · How function calling works · Mistral · Function calling steps
finish_reason: "tool_calls", Anthropic and
Bedrock use stop_reason/stopReason: "tool_use", Gemini surfaces a
functionCall part, and Ollama simply populates message.tool_calls. The
sections below give the provider-documented JSON for each.
Each card below walks the same three steps from Figure 2 — declare, model emits a call, return the result — using JSON taken from the provider's own documentation. A source line under every block links the source page it came from.
All nine APIs share that high-level contract, but the concrete JSON is not interchangeable: none are wire-compatible end to end, though several cluster around the OpenAI Chat Completions shape. They differ along five axes (the comparison table near the end gives the per-provider detail):
A function wrapper, a flat object, or Bedrock's nested toolSpec. Schema field: parameters, input_schema, or inputSchema.json.
A finish/stop reason, a typed functionCall / function_call item, or just the presence of tool_calls.
Arguments arrive either as a JSON string you parse, or as an already-parsed object.
A tool-role message, a tool_result block in a user message, a function_call_output item, or a functionResponse part.
The same intent is spelled required, any, ANY, or REQUIRED.
In short: one provider means one shape; several providers means translating between these — typically with a thin per-provider adapter over a shared internal representation, or a framework that abstracts them behind one interface.
/v1/chat/completions and the newer /v1/responses./v1/chat/completions)Each tool is wrapped in a function object inside the tools array. The arguments are a JSON Schema; strict: true enforces the schema exactly.
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Retrieves current weather for the given location.",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "City and country e.g. Bogotá, Colombia"
}
},
"required": ["location"],
"additionalProperties": false
},
"strict": true
}
}
]The assistant message carries a tool_calls array (each with an id, a function.name, and a string arguments), and the response sets finish_reason: "tool_calls". Multiple entries appear when the model calls tools in parallel.
"tool_calls": [
{
"id": "call_12345xyz",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\":\"Paris, France\"}"
}
},
{
"id": "call_67890abc",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\":\"Bogotá, Colombia\"}"
}
}
]The result goes back as a message with role: "tool", referencing the call by tool_call_id. Per the docs, a tool message must follow a preceding message that had tool_calls.
{
"role": "tool",
"tool_call_id": "call_12345xyz",
"content": "{\"temperature\": \"15 C\", \"conditions\": \"cloudy\"}"
}/v1/responses)The Responses API flattens the tool object — name, description, parameters, and strict sit at the top level, with no nested function wrapper.
{
"type": "function",
"name": "get_weather",
"description": "Retrieves current weather for the given location.",
"parameters": {
"type": "object",
"properties": {
"location": { "type": "string", "description": "City and country e.g. Bogotá, Colombia" },
"units": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location", "units"],
"additionalProperties": false
},
"strict": true
}The response output array contains items of type: "function_call". Note the separate call_id (used to submit the result) distinct from the item's own id.
[
{
"id": "fc_12345xyz",
"call_id": "call_12345xyz",
"type": "function_call",
"name": "get_weather",
"arguments": "{\"location\":\"Paris, France\"}"
}
]The result is a function_call_output item that references call_id (not id). The output is typically a string in any format you choose.
{
"type": "function_call_output",
"call_id": "call_12345xyz",
"output": "70 degrees and sunny."
}"auto", "required", "none", or a named function. Chat uses {"type":"function","function":{"name":...}}; Responses uses the flat {"type":"function","name":...} and adds an allowed_tools subset form.
On by default; set parallel_tool_calls: false to allow at most one. Not available with built-in tools.
Chat streams delta.tool_calls chunks; Responses emits semantic events like response.function_call_arguments.delta.
user/assistant message structure — there is no special tool or function role.Each tool in the tools array uses name, description, and input_schema (a JSON Schema object). Note the field is input_schema, not parameters.
{
"name": "get_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The unit of temperature"
}
},
"required": ["location"]
}
}The response sets stop_reason: "tool_use" and includes one or more tool_use content blocks, each with an id, a name, and the parsed input object (already JSON, not a string).
{
"role": "assistant",
"content": [
{
"type": "text",
"text": "I'll check the current weather in San Francisco."
},
{
"type": "tool_use",
"id": "toolu_01A09q90qw90lq917835lq9",
"name": "get_weather",
"input": { "location": "San Francisco, CA" }
}
]
}The result is sent back in a new user message whose content begins with a tool_result block referencing tool_use_id. Tool-result blocks must come first in the array; is_error: true reports a failure.
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": "toolu_01A09q90qw90lq917835lq9",
"content": "15 degrees and cloudy"
}
]
}auto (default with tools), any (must use some tool), tool with a name (forces one), and none. With any/tool no leading prose is emitted.
On by default — one assistant turn can hold several tool_use blocks; all their tool_result blocks must return in a single user message. Disable with disable_parallel_tool_use.
Supported, including fine-grained streaming of tool-call input as it is generated.
functionDeclarations; turns are parts inside contents. Endpoint: POST …/v1beta/models/{model}:generateContent.Tools go in tools[].functionDeclarations[]; each declaration has name, description, and an OpenAPI-subset parameters schema.
"tools": [
{
"functionDeclarations": [
{
"name": "schedule_meeting",
"description": "Schedules a meeting with specified attendees at a given time and date.",
"parameters": {
"type": "object",
"properties": {
"attendees": {
"type": "array",
"items": { "type": "string" },
"description": "List of people attending the meeting."
},
"date": { "type": "string", "description": "Date of the meeting (e.g., '2024-07-29')" },
"time": { "type": "string", "description": "Time of the meeting (e.g., '15:00')" },
"topic": { "type": "string", "description": "The subject or topic of the meeting." }
},
"required": ["attendees", "date", "time", "topic"]
}
}
]
}
]The model returns a functionCall part inside candidates[].content.parts[], with the function name, the args object, and (for Gemini 3 models) a unique id. Always scan the parts array rather than assuming position.
{
"functionCall": {
"name": "schedule_meeting",
"args": {
"attendees": ["Bob", "Alice"],
"date": "2025-03-27",
"time": "10:00",
"topic": "Q3 planning"
},
"id": "UNIQUE_CALL_ID"
}
}The runtime appends a turn containing a functionResponse part with the matching name, the response payload, and the same id for Gemini 3 models.
{
"functionResponse": {
"name": "schedule_meeting",
"response": { "result": "Meeting scheduled for 2025-03-27 at 10:00." },
"id": "UNIQUE_CALL_ID"
}
}function_calling_config.mode: AUTO (model decides), ANY (must call a function, optionally limited by allowed_function_names), NONE, and VALIDATED.
Supports parallel calls (several functionCall parts in one turn), sequential/compositional calling, and mixing with built-in tools.
Available via streamGenerateContent; the Live API offers tool use for earlier model families.
tools/function/parameters shape, but arguments come back as a parsed object, not a string.The tools array uses the OpenAI-style { "type": "function", "function": { … } } shape against the local /api/chat endpoint.
{
"model": "llama3.2",
"messages": [
{ "role": "user", "content": "what is the weather in tokyo?" }
],
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get the weather in a given city",
"parameters": {
"type": "object",
"properties": {
"city": { "type": "string", "description": "The city to get the weather for" }
},
"required": ["city"]
}
}
}
],
"stream": false
}The assistant message carries a tool_calls array. Unlike OpenAI, the arguments field is a JSON object, already parsed.
{
"model": "llama3.2",
"message": {
"role": "assistant",
"content": "",
"tool_calls": [
{
"function": {
"name": "get_weather",
"arguments": { "city": "Tokyo" }
}
}
]
}
}The result is appended as a message with role: "tool", carrying the content string and the tool_name it answers.
{
"role": "tool",
"content": "11 degrees celsius",
"tool_name": "get_weather"
}The native /api/chat docs do not list a tool_choice/mode parameter. The OpenAI-compatible /v1/chat/completions endpoint now lists tool_choice as a supported request field.
Supported — multiple entries can appear in tool_calls.
With "stream": true, the tool call arrives in a chunk with "done": false, then a final "done": true / "done_reason": "stop" object.
tools with a function wrapper, tool_calls in the response, results via a tool role.Tools use the familiar { "type": "function", "function": { name, description, parameters } } shape.
"tools": [
{
"type": "function",
"function": {
"name": "retrieve_payment_status",
"description": "Get payment status of a transaction",
"parameters": {
"type": "object",
"properties": {
"transaction_id": { "type": "string", "description": "The transaction id." }
},
"required": ["transaction_id"]
}
}
}
]The model returns a tool_calls array; each entry has an id, a type, and a function with a string arguments field you parse yourself.
[
{
"function": {
"name": "retrieve_payment_status",
"arguments": "{\"transaction_id\": \"T1001\"}"
},
"id": "D681PevKs",
"type": "function"
}
]The result is a tool message with the function name, the content result, and the matching tool_call_id.
{
"role": "tool",
"name": "retrieve_payment_status",
"content": "{\"status\": \"Paid\"}",
"tool_call_id": "D681PevKs"
}"auto" (default), "any" (forces tool use), "none" (prevents it). Mistral's force-equivalent is "any".
Controlled by parallel_tool_calls (default true); set false to force single tool calling.
Supported by the chat endpoint generally; the function-calling guide itself shows the non-streaming flow.
tool_plan, and results return as document blocks that feed built-in citations. Endpoint: POST /v2/chat.Tools use the { "type": "function", "function": { … } } shape with a JSON-Schema parameters block.
"tools": [
{
"type": "function",
"function": {
"name": "search_docs",
"description": "Search documentation and return relevant snippets as documents.",
"parameters": {
"type": "object",
"properties": {
"query": { "type": "string", "description": "The search query to look up in the docs." },
"top_k": { "type": "integer", "description": "How many documents to return." }
},
"required": ["query"]
}
}
}
]The assistant message returns a tool_plan (the model's reasoning) alongside tool_calls (each with an auto-generated id and a string arguments).
{
"role": "assistant",
"tool_plan": "I will search the docs for how tool use works in Cohere.",
"tool_calls": [
{
"id": "search_docs_1byjy32y4hvq",
"type": "function",
"function": {
"name": "search_docs",
"arguments": "{\"query\":\"tool use Cohere\",\"top_k\":3}"
}
}
]
}The result is a tool message referencing tool_call_id, whose content is a list of document blocks — a Cohere-specific shape that drives its citation generation.
{
"role": "tool",
"tool_call_id": "search_docs_1byjy32y4hvq",
"content": [
{
"type": "document",
"document": {
"data": "{\"title\": \"Tool use overview\", \"text\": \"Tool use connects models to external tools.\"}"
}
}
]
}"REQUIRED" forces tool call(s); "NONE" forces a direct answer; unset lets the model decide. Compatible with Command R7B and newer only.
Documented — the model can return several tool_calls; you append one tool message per call.
Supported via a dedicated tool-use streaming guide.
toolConfig.tools[].toolSpec with inputSchema.json; calls and results are content blocks; stopReason is "tool_use".Tools live under toolConfig.tools[].toolSpec; the schema sits inside inputSchema.json and its top-level type must be object.
"toolConfig": {
"tools": [
{
"toolSpec": {
"name": "top_song",
"description": "Get the most popular song played on a radio station.",
"inputSchema": {
"json": {
"type": "object",
"properties": {
"sign": {
"type": "string",
"description": "The call sign for the radio station, e.g. WZPZ."
}
},
"required": ["sign"]
}
}
}
}
]
}The response sets stopReason: "tool_use"; a toolUse content block carries a toolUseId, a name, and the parsed input object.
{
"stopReason": "tool_use",
"output": {
"message": {
"role": "assistant",
"content": [
{
"toolUse": {
"toolUseId": "tooluse_kZJMlvQmRJ6eAyJE5GIl7Q",
"name": "top_song",
"input": { "sign": "WZPZ" }
}
}
]
}
}
}The result goes back in a user message containing a toolResult block that references the same toolUseId; status may be success or error.
{
"role": "user",
"content": [
{
"toolResult": {
"toolUseId": "tooluse_kZJMlvQmRJ6eAyJE5GIl7Q",
"content": [
{ "json": { "song": "Elemental Hotel", "artist": "8 Storey Hike" } }
]
}
}
]
}A union: auto (default), any (must call some tool, no text), or tool (a named tool; Claude 3 & Nova only). No separate none/required members.
The response content array can hold multiple toolUse blocks; there is no parallel_tool_calls toggle.
Use ConverseStream, which emits toolUse deltas.
Groq documents itself as “mostly compatible with OpenAI's client libraries”: point the SDK at https://api.groq.com/openai/v1. xAI also accepts OpenAI clients at https://api.x.ai/v1, but its current function-calling guide is written around the Responses API. The two surfaces are shown separately below.
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Get current weather for a location",
"parameters": {
"type": "object",
"properties": {
"location": { "type": "string", "description": "City and state, e.g. San Francisco, CA" },
"unit": { "type": "string", "enum": ["celsius", "fahrenheit"] }
},
"required": ["location"]
}
}
}
]{
"role": "assistant",
"tool_calls": [
{
"id": "call_abc123",
"type": "function",
"function": {
"name": "get_weather",
"arguments": "{\"location\": \"San Francisco, CA\", \"unit\": \"fahrenheit\"}"
}
}
]
}{
"role": "tool",
"tool_call_id": "call_abc123",
"name": "get_weather",
"content": "{\"temperature\": 72, \"condition\": \"sunny\", \"unit\": \"fahrenheit\"}"
}xAI's function-calling guide uses the Responses API, so the tool object is flat (type, name, description, parameters) — the same shape as OpenAI's Responses surface.
"tools": [
{
"type": "function",
"name": "get_temperature",
"description": "Get current temperature for a location",
"parameters": {
"type": "object",
"properties": {
"location": { "type": "string", "description": "City name" },
"unit": { "type": "string", "enum": ["celsius", "fahrenheit"], "default": "fahrenheit" }
},
"required": ["location"]
}
}
]The response output array carries items of type: "function_call", each with a call_id, a name, and a string arguments.
{
"type": "function_call",
"call_id": "call_12345xyz",
"name": "get_temperature",
"arguments": "{\"location\":\"San Francisco\"}"
}The result is a function_call_output item referencing call_id; the turn continues with previous_response_id.
{
"type": "function_call_output",
"call_id": "call_12345xyz",
"output": "{\"temperature\": 59, \"unit\": \"fahrenheit\"}"
}xAI documents auto, required, none, and a named function form. Groq's OpenAI-compatible chat endpoint lists tool_choice as a supported request field.
Groq supports parallel tool use on a per-model basis; xAI enables parallel_tool_calls by default and lets you set it false.
Groq streams tool-call deltas OpenAI-style; xAI notes that its function call is returned whole in a single chunk rather than streamed across chunks.
Read across a row to see how each API answers the same question. The client-tool contract is the same; field names, nesting, and state-handling conventions differ.
| OpenAI (Chat) | OpenAI (Responses) | Anthropic | Gemini | Ollama | Mistral | Cohere v2 | Bedrock | Groq / xAI | |
|---|---|---|---|---|---|---|---|---|---|
| Tools declared under | tools[].function |
tools[] (flat) |
tools[] |
tools[].functionDeclarations[] |
tools[].function |
tools[].function |
tools[].function |
toolConfig.tools[].toolSpec |
Groq Chat: tools[].functionxAI Responses: flat tools[] |
| Schema field | parameters |
parameters |
input_schema |
parameters |
parameters |
parameters |
parameters |
inputSchema.json |
parameters |
| "Wants a tool" signal | finish_reason: "tool_calls" |
output[] function_call item |
stop_reason: "tool_use" |
functionCall part |
message.tool_calls present |
tool_calls present |
tool_calls (+ tool_plan) |
stopReason: "tool_use" |
Groq Chat: tool_callsxAI Responses: function_call item |
| Call carries | id, name, arguments (str) |
call_id, name, arguments (str) |
id, name, input (obj) |
name, args (obj), id |
name, arguments (obj) |
id, name, arguments (str) |
id, name, arguments (str) |
toolUseId, name, input (obj) |
Groq: id, name, arguments (str)xAI: call_id, name, arguments (str) |
| Result returned as | role: "tool" + tool_call_id |
function_call_output + call_id |
user msg, tool_result block |
functionResponse part |
role: "tool" + tool_name |
role: "tool" + tool_call_id |
role: "tool", document blocks |
user msg, toolResult block |
Groq Chat: role: "tool" + tool_call_idxAI Responses: function_call_output + call_id |
| Force / restrict | tool_choice: auto/required/none/named |
tool_choice: + allowed_tools |
tool_choice: auto/any/tool/none |
mode: AUTO/ANY/NONE/VALIDATED |
native: not documented; compat: tool_choice supported |
tool_choice: auto/any/none |
tool_choice: REQUIRED/NONE |
toolChoice: auto/any/tool |
auto/required/none/named |
| Parallel calls | parallel_tool_calls |
parallel_tool_calls |
default; disable_parallel_tool_use |
parallel + compositional | multiple tool_calls |
parallel_tool_calls |
multiple tool_calls |
multiple toolUse blocks |
per-model / parallel_tool_calls |
Every cell is sourced to the provider docs cited in that provider's section above. Scroll the bar above (or the table) to see all columns.
Six factual observations drawn from the per-provider JSON above.
Every API implements the same declare → model-emits-call → return-result loop for client tools. The model emits a structured request; your runtime executes client-side tools and returns results. The differences are field names, nesting, and state-handling conventions.
The OpenAI Chat tools[].function wrapper (also Ollama, Mistral, Cohere, and Groq Chat); flat variants (OpenAI Responses, xAI Responses, Anthropic's input_schema, Gemini's functionDeclarations); and Bedrock's nested toolSpec.inputSchema.json.
OpenAI, Mistral, Cohere, Groq, and xAI return arguments as a JSON string you must parse. Anthropic (input), Gemini (args), Ollama, and Bedrock (input) hand back an already-parsed object.
A finish reason (finish_reason: "tool_calls"), a stop reason (stop_reason/stopReason: "tool_use"), a typed content block/part (tool_use, functionCall), or simply the presence of a tool_calls field (Ollama).
Several chat-style APIs add a dedicated tool role. Anthropic and Bedrock fold the result into a normal user message as a tool_result / toolResult block, while Responses-style APIs use a function_call_output item.
The same intent is required (OpenAI, xAI), any (Anthropic, Mistral, Bedrock), ANY (Gemini), or REQUIRED (Cohere). Ollama's native API docs do not document an equivalent, while its OpenAI-compatible endpoint lists tool_choice as supported.
Terms that mean the same thing but are spelled differently across providers.
parameters, input_schema, or inputSchema.json).tool_calls / tool_use).contents.