Function Calling on EXAIV: Defining Tools, Handling Tool Calls, and Closing the Loop

Function calling is the mechanism that lets a model request an action rather than just produce text: you declare the tools your application can perform, the model decides which one to call and with what arguments, your code executes it, and you send the result back so the model can continue. That loop is what turns a chat endpoint into something that can look up a record, run a query, or operate a file.

The protocol is straightforward. The hard parts are the ones where the model is confidently wrong: arguments that are plausible but invalid, tools that should have been called but were not, results that come back in a shape the model cannot use, and loops that never terminate.

Declaring a tool

A tool is a name, a description, and a parameter schema:

{
  "name": "get_order_status",
  "description": "Look up the current status of an order by its order ID. Returns the status, carrier, and expected delivery date.",
  "parameters": {
    "type": "object",
    "properties": {
      "order_id": {
        "type": "string",
        "description": "The order ID, in the form ORD-123456."
      }
    },
    "required": ["order_id"]
  }
}

Descriptions matter more than they look. The model chooses between tools using the names and descriptions, so "Look up the current status of an order by its order ID" tells it when to use the tool, while "order stuff" does not. The same applies to parameter descriptions: an un-described string parameter produces guesses.

Send the tools with the request:

{
  "model": "gpt-6-sol",
  "messages": [{"role": "user", "content": "Where is order ORD-987321 right now?"}],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_order_status",
        "description": "Look up the current status of an order by its order ID. Returns the status, carrier, and expected delivery date.",
        "parameters": {
          "type": "object",
          "properties": {
            "order_id": {"type": "string", "description": "The order ID, in the form ORD-123456."}
          },
          "required": ["order_id"]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

The response contains a call, not an answer

When the model wants a tool, the response has no content — it has tool_calls:

{
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call_abc123",
        "type": "function",
        "function": {
          "name": "get_order_status",
          "arguments": "{\"order_id\":\"ORD-987321\"}"
        }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

Two things surprise people on the first encounter. content is null — there is no text to display. And arguments is a string containing JSON, not an object. Parse it before use.

Closing the loop

You execute the tool and send the result back as a tool message whose tool_call_id matches the request:

import json

response = client.chat.completions.create(
    model="gpt-6-sol",
    messages=messages,
    tools=tools,
    tool_choice="auto",
)

message = response.choices[0].message

if message.tool_calls:
    messages.append(message.model_dump(exclude_none=True))   # the assistant turn with the calls

    for call in message.tool_calls:
        args = json.loads(call.function.arguments)            # parse the JSON string
        result = run_tool(call.function.name, args)            # your code
        messages.append({
            "role": "tool",
            "tool_call_id": call.id,
            "content": json.dumps(result, ensure_ascii=False),
        })

    final = client.chat.completions.create(
        model="gpt-6-sol",
        messages=messages,
        tools=tools,
    )
    print(final.choices[0].message.content)

Two rules here are not optional:

• Append the assistant turn with the tool calls before appending the tool results. The API expects the call to be part of the conversation, and skipping it produces a malformed request.
• Return a result for every requested call, matched by tool_call_id. A call without a result leaves the conversation inconsistent, and the next request usually fails.

Validate the arguments

The model produced those arguments. They are untrusted input, exactly like anything else arriving from outside your process.

from pydantic import BaseModel, ValidationError

class OrderArgs(BaseModel):
    order_id: str

def run_tool(name, raw_args):
    if name == "get_order_status":
        try:
            args = OrderArgs(**raw_args)
        except ValidationError as e:
            return {"error": "invalid_arguments", "detail": e.errors()}
        return lookup_order(args.order_id)
    return {"error": "unknown_tool", "tool": name}

Beyond the schema, apply your own business rules: does the order exist, does the signed-in user own it, is the amount within a permitted range. Schema validation proves the shape is right; it does not prove the action is allowed.

tool_choice

Use "required" when the workflow cannot proceed without a tool — for example, a lookup step that must happen before an answer. Use a specific function when exactly one tool is valid. "auto" is right for most applications.

Parallel tool calls

The model can request several calls in one turn. Two things to handle:

Execution. Independent calls can run concurrently. Calls with side effects, or calls that depend on each other, should run in sequence — a tool that deletes something and a tool that reads it must not race.

Ordering of results. Return every result, each with its own tool_call_id. The order of the tool messages does not need to match the order of the calls, as long as the IDs are right.

If your application cannot tolerate parallel calls, request one at a time. It is simpler to reason about and the difference in latency is often smaller than it appears.

Returning errors

When a tool fails, return the failure as a result. Do not raise an exception out of the loop and do not return an empty string:

result = {"error": "not_found", "order_id": args.order_id, "message": "No order with that ID."}
messages.append({"role": "tool", "tool_call_id": call.id, "content": json.dumps(result)})

A structured error tells the model what happened and lets it adjust — ask for a different ID, explain the problem to the user, or try a different tool. A silent failure or a thrown exception usually ends the conversation.

Keeping the loop under control

An unbounded tool loop is a real failure mode: the model calls a tool, gets an unhelpful result, calls it again with the same arguments, and repeats until you run out of budget.

• Cap the number of rounds. Ten tool rounds is generous for most workflows. When the cap is hit, return whatever the model has and let the user continue.
• Cap the total tokens. A tool that returns a large payload grows every subsequent request.
• Make tool results small. Return the fields the model needs, not a full record. A 10,000-character tool result costs 10,000 tokens on every subsequent turn.
• Detect a repeated identical call and stop or return a clarifying error.

Patterns that work

If your workflow is the first or second shape, consider whether you need tool calling at all — a direct application-side call is cheaper and more predictable. Tool calling earns its overhead when the decision of what to call genuinely depends on the conversation.

Testing a tool-calling integration

A tool loop has more moving parts than a plain completion, and it fails in ways a single-request test never reaches. A useful test suite covers the following, in this order.

Argument handling. Send a request that should call the tool, and assert on the call itself: the right tool name, arguments that parse, and values that match the schema. Feed the tool deliberately ambiguous input and check that the arguments are still valid.

Tool failure. Make the tool raise, and confirm that the model receives a structured error result and produces a sensible next message — not a crash, and not an infinite retry against the same failing tool.

A tool that returns nothing useful. Return an empty or uninformative result. The model should say it does not have the answer rather than inventing one, and the loop should terminate.

Multiple calls. Ask something that legitimately needs two tool calls, and verify both results are returned with matching identifiers.

An unknown tool. Have the model reference a tool that does not exist, and confirm your code handles it as an error rather than crashing on a missing function.

A long session. Let the loop run to its cap and confirm it stops cleanly with the accumulated context intact.

Two testing habits make this much easier. First, log the full conversation — every assistant turn, every tool call, every tool result — for a failing test. Most tool-loop bugs are visible in the transcript and invisible in the final message. Second, make your tool implementations deterministic in tests: a fixed clock, a fixed database, a fixture for the lookup. A tool that returns slightly different data each run produces a flaky suite that trains people to ignore failures.

Function calling checklist

• Every tool has a name, a clear description, and a JSON Schema for its parameters.
• Parameter descriptions state format and constraints.
• Arguments are parsed from the JSON string and validated before execution.
• Authorization and business rules are enforced in your code, not in the schema.
• The assistant turn with the tool calls is appended before the tool results.
• A result is returned for every requested call, including failures.
• Tool results are small and contain only what the model needs.
• The loop has a round cap and a token cap.
• tool_choice is set deliberately rather than left to chance.
• Every call is logged with its arguments and result size.

Further reading

• Coding Agent Integration Guide — the multi-turn tool loop, from the operator's side.
• JSON Structured Output Guide — the alternative to tools when you only need structured data back.
• Chat Completions Parameters Reference — the request fields that control tool behavior.

FAQ

Related reading

• JSON Structured Output Guide
• Agent Framework Guide
• Qwen Streaming and Tool Calling

Terminology and pre-flight checklist

Total parameters, active parameters and the KV cache are three different quantities. Total parameters are all the weights that must be loaded into memory; active parameters are the subset that computes a single token. KV cache is temporary memory that grows linearly with context length, so a 1M window is a ceiling rather than a free default. Quantization lowers the precision of stored weights to shrink files; it does not reduce the parameter count. A vision_config in a config.json describes architecture, not a wire format you may safely invent.

Before shipping anything built on these notes, run six checks. Re-read the numbers from the live source on the day you publish, because download counters move weekly. Open the live /models catalog and confirm the model ID you plan to call is still there. Open /pricing rather than copying a price into your application. Never hard-code a request body you have not validated against the official reference. Record the model ID, input and output token counts, and latency for a representative set of prompts, then keep that set unchanged so a later model swap produces a comparable reading. Read the licence on the model card before commercial use, since vendor-specific terms are not a permissive default. Add exponential backoff for retryable requests, set a timeout, and log the error class so that a 401, a 404 and a 429 never get the same treatment.