You may also see this called tool calling or function calling. The names often refer to the same mechanism. However, some platforms use tools as a wider category that includes hosted search, code execution, file retrieval, or computer control in addition to developer-defined functions.
The useful mental model
Think of the model as a caller, not the machine that performs the work.
The model can produce a structured request such as “call get_order_status with order ID 4821.” That output is still a model response. It does not query a database by itself.
A surrounding application receives the request and controls what happens next. It can reject the call, ask the user to confirm it, check permissions, run code, or return an error. If it runs the tool, it gives the result back to the model as new context. The model can then explain the result or request another tool.
That separation is the central fact of tool use:
User request
↓
Model selects a tool and proposes inputs
↓
Application validates, authorizes, and executes
↓
Tool result returns to the model
↓
Model answers or requests another toolSome providers also offer hosted tools that they execute on their own servers. The API may hide part of the loop, but the logical stages remain: the model requests an operation, an execution system runs it, and the result enters the conversation.
How tool use works
1. The application describes each tool
A tool definition normally includes:
- a name, such as
get_order_status - a plain-language description of when to use it
- the inputs it accepts
- rules for the shape and type of each input
Those input rules are often expressed as a JSON schema. The schema might say that order_id is a required string. The description tells the model what the tool means; the schema tells it how to format a call.
2. The model receives the request and tool definitions
The model considers the user's message alongside the available tool descriptions. It may answer normally, request one tool, or request several tools when the API and application allow that.
Tool selection is a prediction, not a guaranteed plan. A model can choose the wrong tool, omit a needed call, or propose an argument that is well-formed but inappropriate.
3. The model returns a structured tool call
Instead of—or alongside—ordinary prose, the response contains a machine-readable call. A simplified version might look like this:
{
"name": "get_order_status",
"arguments": {
"order_id": "4821"
}
}Provider APIs encode this in different message types and attach an identifier so the eventual result can be matched to the correct call. The exact envelope changes, but the name-and-inputs idea is consistent across major APIs.
4. The application decides whether to execute it
This is where a proposed action becomes a real one—or does not.
The application parses the arguments and checks them against more than the schema. It may verify the signed-in user, enforce access rules, limit values, require approval for a side effect, or refuse a tool that is unavailable. Only then does a tool runner call the database, API, calculator, filesystem, or other system.
A strict schema can improve structural reliability, but it is not an authorization system. "order_id": "4821" can be valid JSON and still refer to someone else's order.
5. The tool result goes back to the model
The application sends the output in a result message tied to the original call. The output might contain data, an error, or a statement that permission was denied.
The model uses that result as additional context. It can produce a final answer, correct an earlier assumption, or make another tool call. A multi-step AI agent may repeat this loop until it reaches a stopping condition.
Worked example: checking an order
Suppose a store exposes one read-only tool:
Name: get_order_status
Purpose: Return shipping information for an order the current user owns
Input: order_id (required string)The user asks:
Where is order 4821?
The model requests:
{
"name": "get_order_status",
"arguments": {
"order_id": "4821"
}
}Before querying anything, the application checks the current session. If the user owns order 4821, it executes the lookup and returns:
{
"status": "shipped",
"estimated_delivery": "Friday"
}The model can now answer, “Order 4821 has shipped and is estimated to arrive Friday.”
If the ownership check fails, the application should not run the lookup or reveal the status. It can return a permission error, which the model can explain without exposing private data.
Nothing in the model's call proved ownership. The tool schema described a valid request; the application enforced the real rule.
Why tool use matters
Tool use gives a language model controlled access to capabilities that text generation alone does not provide.
- Fresh or private data: A tool can retrieve current information or data that was never in the model's training material.
- Exact operations: A calculator, database query, or program can perform work that should not depend on a plausible-sounding estimate.
- Real actions: Tools can create records, send messages, schedule events, or operate other software.
- Existing systems: A tool can turn a natural-language request into a call to an established business API.
- Multi-step work: The result of one call can guide the next call.
Tool use does not remove the model's limitations. It moves some tasks to systems better suited to them and creates an interface between probabilistic decisions and deterministic software.
Reliability and safety depend on the surrounding system
The model is not a trusted source of commands merely because its output matches a schema. Production tool use needs controls around the model.
Validate meaning, not just shape. Check identifiers, ranges, required state, and business rules. Structural validation cannot tell whether a proposed transaction matches the user's request.
Authorize at execution time. Use identity and permissions held by the application. Do not let the model grant itself access by placing a user ID or role in a tool argument.
Limit each tool's power. A narrowly scoped get_order_status tool is easier to control than a general database tool. Read-only and write tools should be distinct when their risk differs.
Treat tool output as untrusted data. External content can be wrong or can contain text designed to redirect the model. A returned document is information to inspect, not a new authority that can replace the user's request or the system's rules.
Handle side effects deliberately. Sending a message or issuing a refund is different from looking up a status. Sensitive actions may need confirmation, idempotency protection against duplicate execution, and an audit record.
Return failures clearly. Timeouts, unavailable services, rejected permissions, and invalid arguments should become explicit tool results. Otherwise the model may fill the gap with an invented success.
Common misconceptions
“The LLM called my API”
Usually, the LLM generated a request and an application called the API. Saying the model “called” the tool is convenient shorthand, but it hides the execution and security boundary.
“Function calling and tool calling are always different”
For developer-defined functions, providers and frameworks often use the terms interchangeably. The difference appears when a platform uses tool as an umbrella for functions plus built-in capabilities such as search or code execution. Check the platform's taxonomy instead of assuming a universal distinction.
“A valid schema means a correct call”
A schema can require a string, number, list, or object and can reject malformed output. It cannot guarantee that the model selected the right tool, inferred the right value, or respected a business rule.
“Adding a tool creates an agent”
Tool use is a capability. An agent is a larger system that can decide, act, observe results, preserve relevant state, repeat the loop, and stop. A one-off structured call can use the same mechanism without becoming an autonomous workflow.
“The tool result must be true”
The model can faithfully summarize a bad result. Tool reliability, data freshness, access control, and resistance to malicious content are properties of the whole system, not of the language model alone.
How tool use fits into an AI system
Tool use sits at the boundary between a language model and the systems around it.
The model interprets language and proposes a structured next step. The application supplies tool definitions, keeps conversation state, validates calls, manages permissions, executes operations, and returns observations. Tools provide narrow capabilities. An agent can use that arrangement repeatedly to pursue a larger task.
This is why tool-use quality is not only a model property. Clear tool descriptions, distinct schemas, execution controls, useful error messages, and sensible stopping rules all affect whether the system works.
Where to go next
Read What Is an AI Agent? to see how tool calls become part of a repeated plan-act-observe loop with state and stopping rules. Read What Is an AI Harness? for the software that validates and executes those calls, then use AI Model vs. Chatbot vs. Harness vs. Agent to place tool use in the wider system.