What Are Output Tokens?

A useful mental model

Think of output tokens as both pieces of the answer and steps in making the answer.

A text model does not write a complete reply behind the scenes and then divide it into tokens. It builds the reply one token at a time. After each token, the growing sequence becomes part of the information used to choose the next one.

That difference explains why a long answer takes longer to generate than a short answer. Each additional output token requires another generation step.

How output tokens are generated

After the model processes the input, it assigns a score to every token that could come next. Generation settings determine how one of those possibilities is chosen. The selected token is appended to the sequence, and the model repeats the process.

process the input
        |
        v
score possible next tokens
        |
        v
choose one output token ----+
        |                   |
        v                   |
append it to the sequence --+
        |
        v
stop when a stopping condition is met

The loop can end because:

• the model reaches a natural end;
• it generates a configured stop sequence;
• it reaches an output-token ceiling;
• it switches to an action such as a tool call; or
• the service applies another model- or policy-specific stopping condition.

The exact token count depends on the model’s tokenizer. A short word might be one token. A longer or less familiar word might be several. Spaces, punctuation, code, and non-English text can also be divided differently. That is why character count and word count can estimate output length, but cannot determine it exactly.

What counts as an output token

For an ordinary text response, the visible reply is formed from generated output tokens. Generated JSON and tool-call arguments are also output from the model, even when an application renders them as an interface action instead of showing the raw text.

Provider usage categories add a second layer. “Output tokens” can refer either to the visible generated sequence or to a metered total. Those totals are not standardized:

• Some APIs call generated tokens completion tokens.
• Some include hidden reasoning in the output total and provide a separate breakdown.
• Some report visible candidate tokens and thought tokens as separate fields.
• Special output types can have their own details or billing rules.

The reliable source for a request is the usage data returned by that provider and endpoint. Do not infer billed output solely by copying the visible response into a tokenizer. Current OpenAI, Anthropic, and Google documentation illustrates why: their labels and breakdowns differ.

A worked example

Imagine an API request with a maximum of 100 output tokens.

The model generates a complete answer and stops after 24 tokens. The usage data reports:

{
  "input_tokens": 18,
  "output_tokens": 24,
  "total_tokens": 42
}

The output-token count is 24. The 100-token setting was only a ceiling. It did not force the model to produce 100 tokens, and it does not by itself mean 100 tokens were billed.

Now suppose the model continues until it reaches the 100-token ceiling. The request can still return a technically successful API response, but the answer may end mid-sentence or mid-structure. A production application should check the response’s stop reason rather than assume that returned text is complete.

There is one more wrinkle. A reasoning model may generate internal reasoning before the visible answer. Depending on the provider, usage might include those hidden tokens in output_tokens or report them in a separate thought field. The visible answer and the metered output can therefore have different token counts.

Why output tokens matter

Cost

Token-priced services usually meter input and output separately, and their rates can differ. Actual output usage, not just the requested ceiling, is therefore part of the cost of each request. Provider-specific quota systems may temporarily reserve capacity based on the ceiling even when billing uses actual generation.

Response time

Output tokens are generated sequentially. More output generally means more generation steps and a longer wait for the complete response. Streaming can show partial output sooner, but it does not eliminate the work required to generate the remaining tokens.

Completeness

An output ceiling helps control length, but a ceiling that is too low can cut off prose, code, JSON, or tool arguments. Check the stop reason whenever completeness matters.

Available context

Input and output both consume room in a model’s context window. A request that uses most of the available room for input can leave less room for generation, subject to the limits and accounting rules of the model and endpoint.

Common misconceptions

“Output tokens are the same as words”

They are not. Tokens can be whole words, parts of words, punctuation, spaces, or other units. The same sentence can have different token counts under different tokenizers.

“The maximum output setting is the expected output”

It is an upper bound. The model can stop before reaching it. Providers may also apply a lower model limit or a shared budget that includes internal reasoning.

“If the API returned text, the answer is complete”

Not necessarily. Hitting the output limit can return partial text with a length-related stop reason. This matters especially for code and structured data, where one missing closing character can make the result unusable.

“The visible response equals billed output”

Often, but not universally. Hidden reasoning, multiple generated candidates, or endpoint-specific accounting can make usage larger than the text you see. Treat the returned usage fields as authoritative for that service.

“Output length is fixed by the prompt”

The prompt influences length, but generation remains conditional. Model behavior, sampling settings, stop sequences, tool use, and safety behavior can all change where the output ends.

How output tokens fit into a model request

A request begins with input tokens. The model then generates output tokens. Reasoning models may also use internal reasoning tokens, whose reporting and billing treatment varies by provider.

The distinction is easiest to see side by side in Input vs. Output vs. Reasoning Tokens. If you are measuring serving behavior, output generation also connects directly to tokens per second and inference latency.

The practical rule is simple: set a sensible output ceiling, read the actual usage, and inspect the stop reason before treating the response as complete.