There is no standard definition. AI harness and agent harness often mean the runtime around a tool-using model. Some developers use the broader term for almost everything in an AI application except the model itself.
How an AI harness works
A model accepts input and produces output. By itself, it does not decide how an application stores a conversation, whether a requested action is allowed, or what should happen after a tool fails. The harness handles those jobs.
A typical harness repeats four steps:
- Prepare the model's input. The harness combines the user's request with instructions and the information needed for this turn. It may load conversation state, retrieve a document, or describe the tools the model may request.
- Call the model. It sends that input to a model provider and receives text or a structured request.
- Interpret and control the output. If the model requests a tool, the harness checks that the tool exists, validates its arguments, and applies permissions or approval rules. If the output must match a format, the harness validates that too.
- Return, act, or continue. The harness can show a final answer, execute an allowed action, or give a tool result back to the model for another turn. It also enforces stopping conditions, such as a turn limit or a timeout.
Not every harness includes every feature. A simple summarization service may only build an input, call a model, validate the answer, and return it. A coding harness may also provide files, a command runner, persistent state, test results, human approvals, and logs of every action.
The key division is this: the model proposes an output, while the harness determines what that output can do. A model may produce a request to send an email, but ordinary application code in the harness authenticates the user, checks the recipient, asks for approval if required, calls the email service, and reports the result.
Example: a refund assistant
Suppose a customer writes, “Please refund the headphones from order 4812.”
First, the harness sends the request to the model with instructions and descriptions of two available tools: get_order and issue_refund. The model asks to call get_order with order number 4812.
The harness checks that the argument has the expected format and that the customer may access that order. It runs the lookup and gives the result to the model. The result says the headphones cost $80 and are still eligible for a refund.
The model then requests issue_refund for $80. Before money moves, the harness checks the refund policy and the customer's identity. If company policy requires approval, it pauses and shows the proposed action to an employee. Only after approval does the harness call the payment system.
The model helped choose the steps. The harness owned the consequential parts: data access, validation, policy enforcement, approval, execution, and the record of what happened.
This separation also makes failures easier to handle. If the order service times out, the harness can retry once and then stop with a clear error. It does not need to hope that the model invents a safe recovery plan.
Why the harness matters
The same model can behave very differently in two harnesses. One may give it clear tool descriptions, relevant context, strict permissions, and useful error messages. Another may overload it with irrelevant information, expose risky tools, and hide failures. Model quality matters, but it does not describe the behavior of the complete system.
Harnesses are also where developers can enforce rules rather than merely ask for them. An instruction such as “never refund more than $100 without approval” can influence a model, but it is not a reliable control. A code-level check that blocks the payment call is enforceable even when the model makes a mistake or untrusted text tries to redirect it.
A well-defined harness also makes an application easier to inspect and change. Logs can show the inputs, tool requests, approvals, results, and stopping reason for a run. A model can sometimes be replaced without rebuilding every surrounding integration, although model-specific prompts and tool behavior still need testing.
A harness is not always an agent
A harness is a layer of a system. An agent is usually the complete system that uses a model to choose actions over one or more steps.
That means a harness can exist without much autonomy. The software behind a basic chat interface still manages instructions, message history, model calls, and displayed output. Those are harness functions even if the model cannot use tools or pursue a task on its own.
The reverse is not true for a deployed agent. A model cannot directly authenticate to a database, enforce an approval policy, or preserve task state after a process restarts. Software around the model must provide those capabilities, whether or not its developers call that software a harness.
The term also appears in evaluation harness. In that setting, a harness runs models through a consistent set of test tasks and records scores. It is a test system, not necessarily the runtime of an agent. Context usually tells you which meaning is intended.
Where to go next
Read What Is an AI Agent? to see how a model and harness can form a system that chooses and performs multiple steps. Read What Is Tool Use in LLMs? for the proposal, validation, execution, and observation loop a harness controls. For the boundaries among the main layers, see AI Model vs. Chatbot vs. Harness vs. Agent.