“Prompt tokens” usually means the same thing. The name can be misleading, though, because the prompt may include material added by the application or API.
The whole request, not just your message
Imagine typing “Summarize the latest report” into a chat box. That sentence is one part of the input. The application might also send:
- instructions that set the assistant's role and behavior;
- earlier user and assistant messages;
- the report itself or passages retrieved from it;
- definitions of tools the model can call;
- results returned by a tool; and
- markers that identify roles, message boundaries, or the point where generation should begin.
All of those parts can become input tokens. The exact set depends on the application, API, model, and enabled features.
This gives input tokens a precise boundary: they are defined by their role in one request. Text that the model generated as output on one turn can be input on the next turn when the application sends it back as conversation history.
How a request becomes input tokens
The path from an interface to a model looks like this:
system instructions ─┐
conversation history ├─> request formatting ─> tokenizer ─> input token IDs ─> model
documents and tools ┤
current message ─────┘First, the application assembles the context for the request. A bare API call may contain only one prompt. A production assistant may add instructions, history, retrieved documents, tool schemas, and other content.
Next, the service formats that material for the selected model. Chat models do not receive a visual transcript with colored message bubbles. They receive a sequence built with a chat template or an equivalent internal format. vLLM's chat API documentation makes this step explicit: structured messages are converted into a text prompt with the model's chat template.
Then a tokenizer converts the formatted sequence into token IDs. As the Hugging Face tokenizer reference shows, this can involve normalization, splitting text into pieces, mapping the pieces to IDs, and adding special tokens. Because models can use different tokenizers and templates, the same visible text does not guarantee the same input count.
Finally, the model processes that token sequence and begins generating. The response's usage metadata reports how the provider accounted for the request.
Worked example
Suppose an application reports this illustrative request ledger:
- System instructions: 36 tokens
- Tool definitions: 74 tokens
- Conversation history: 160 tokens
- Current user message: 12 tokens
- Message formatting and special tokens: 15 tokens
The request contains 297 input tokens, even though the current message contains only 12.
Now suppose the response contains 60 generated tokens. Those 60 are output tokens for this request. On the next turn, the application may include that answer in the history. The same text then contributes to the next request's input count.
Caching changes the processing and price of repeated input, not its conceptual role. If 200 of the 297 input tokens come from a reusable prefix, a provider may report or price those 200 separately as cached input. They are still context supplied to the model. Anthropic, for example, divides cached requests among multiple usage fields, while OpenAI exposes cached input as a distinct input category. Always read the current field definitions instead of assuming that a field named input_tokens is the complete total.
How to count input tokens
For a rough draft, a tokenizer matched to the model can estimate plain text. It cannot reliably reproduce the full API count unless it also applies the same message template, tool formatting, special tokens, media encoding, and provider-side additions.
For a complete request, use the provider's token-counting endpoint when one is available. Anthropic's token-counting endpoint accepts messages, system prompts, tools, images, and PDFs. Google's token guide likewise distinguishes preflight input counts from usage returned after generation. OpenAI's token-counting guidance notes that request structure and non-text inputs can affect the full count.
Treat a preflight count as a capacity and cost estimate. After the request, retain the provider's usage response as the record of how that call was actually counted and billed.
Why input tokens matter
Cost
Many APIs price input, cached input, and output separately. A useful cost ledger is:
input cost =
uncached input tokens × uncached input rate
+ cached input tokens × cached input rateRates and categories change, so use the provider's current pricing page rather than copying a price into application logic. Long system instructions, repeated tool definitions, large retrieved passages, and growing chat history can make input the largest part of a workload.
Available context
Input tokens occupy space in the model's context. More input can leave less room for the response, depending on the model and API's limit rules. Sending extra context is therefore not free capacity: it competes with other instructions, evidence, and conversation turns for the model's attention and token budget.
Response time
The model must process the input before it can produce the first output token. Larger inputs generally require more prompt-processing work. Reusing an eligible cached prefix can reduce that work, but cache behavior and benefits are provider-specific.
Measurement
Input-token counts help compare requests at a level more useful than characters or words. They can reveal an oversized tool schema, a retrieval system that sends too many passages, or a conversation history that grows on every turn.
Common misconceptions
“Input tokens are the words I typed”
Your words are only the visible portion. The model-facing request can also contain instructions, history, documents, tools, media, and formatting.
“One word equals one input token”
A token can be a word, part of a word, punctuation, or another model-specific unit. Token boundaries vary by tokenizer. What Is a Token in AI? covers that underlying unit.
“Cached tokens are no longer input tokens”
Cached tokens are reused input. Caching may change the amount of computation, the price, and the usage fields, but the cached prefix still contributes context to the request.
“Assistant messages are always output tokens”
They are output when generated. If an assistant message is included in a later request's history, it is input for that later request. Input and output describe sides of a request, not permanent types of text.
“A local tokenizer gives the billing count”
It may give a close text count. It can miss chat-template markers, tool schemas, attachments, and provider-side formatting. Use the API's post-request usage for billing records.
How input tokens fit into the wider system
Input tokens are the model's starting sequence for a request. Output tokens are generated after that sequence, while cached input tokens are a processing and billing subdivision of input. Together, these categories explain most of the token ledger shown in API responses and pricing pages.
The important habit is to measure the assembled request. When a short user message produces a surprisingly large count, inspect the system instructions, history, retrieved material, tools, and formatting before blaming the tokenizer.
Where to go next
Inspect one real API response and identify every usage field that contributes to total input. Then run the provider's preflight counter on the same request and compare the estimate with the final usage. If the token boundaries themselves are unclear, start with What Is a Token in AI?. Continue to What Are Output Tokens?, then use Input vs. Output vs. Reasoning Tokens to compare all three accounting categories.