What Is an LLM?

There is no official size at which a language model becomes “large.” The label is relative. It usually signals a combination of many learned parameters, broad training data, substantial computing requirements, and an ability to handle more than one narrow language task.

A useful mental model

Think of a generative LLM as a function that accepts a sequence and returns scores for possible continuations.

The sequence is made of tokens: the text units the model processes, which may be whole words, parts of words, punctuation, or other symbols. The model does not draft a hidden paragraph and then reveal it. It scores what could come next, selects one token, adds that token to the sequence, and runs again.

That loop is the key:

flowchart LR
    A[Prompt and prior output] --> B[LLM scores possible next tokens]
    B --> C[One token is selected]
    C --> D[Selected token is appended]
    D --> B

The model's own output therefore becomes part of its next input. An early choice can change every choice that follows.

This mental model is more accurate than calling an LLM a database. Training can cause a model to reproduce material it has seen, but ordinary generation does not work by searching a collection of stored sentences and pasting the closest match. The model applies learned numerical relationships to the current sequence.

How an LLM works

An LLM has two distinct phases: training and use.

During pretraining

Text is split into tokens. The model is given prediction problems created from that text. A causal language model predicts the next token from earlier tokens. Other language models learn by reconstructing hidden tokens or by transforming one sequence into another.

Each prediction produces an error signal. Training repeatedly adjusts the model's numerical weights so that better predictions become more likely. Across many examples, those weights capture reusable patterns involving spelling, grammar, style, facts, code, and relationships among ideas.

The transformer architecture helps the model combine relevant parts of the available sequence when calculating each representation. It is the dominant architecture behind LLMs, but it is not part of a timeless definition: other architectures can also be used to build large language models.

Pretraining creates a base model, not necessarily a helpful assistant. Additional training can make the model better at following instructions, preferring certain kinds of answers, or refusing certain requests. Those steps change its behavior, but they do not turn it into a database or guarantee that its output is true.

During generation

At inference time, the trained weights are fixed. The prompt and any other supplied context are converted into tokens. The model calculates a score for every possible next token. Software then chooses a token using a decoding rule.

Choosing the highest-scoring token every time is one option. Sampling among several plausible tokens is another. Sampling can produce different responses to the same prompt even when the model's weights do not change.

After a token is chosen, it is appended to the sequence. The model calculates a new distribution conditioned on the prompt and everything generated so far. Generation ends when the model selects a stop token, reaches an output limit, or is stopped by the surrounding software.

A worked generation example

Suppose the current text is:

The capital of France is

For illustration, imagine that a model assigns these probabilities to four possible next tokens:

The omitted possibilities share the remaining probability. These numbers are invented to show the mechanism; they are not measurements from a particular model.

If the system selects Paris, the sequence becomes:

The capital of France is Paris

The model now evaluates that longer sequence. Plausible next tokens may include punctuation or words that continue the statement. Once one is selected, the loop repeats.

Three useful facts fall out of this example.

First, the model does not need a separate “answer question” program. A question, a translation request, a code sample, and a summarization instruction can all be represented as text whose continuation implies the task.

Second, a high-probability continuation is not the same as a verified fact. The training objective rewards predictive fit to data, not a direct check against reality. A false statement can be a smooth continuation.

Third, output is path-dependent. If an unlikely or mistaken token is selected, later tokens are generated in the context of that choice. The model may continue coherently from a bad premise instead of returning to correct it.

Why LLMs can do many tasks

Large and varied pretraining exposes a model to many recurring structures. These include questions followed by answers, code followed by comments, claims followed by evidence, and instructions followed by completed work. Learning to predict those structures produces representations that can be reused.

A prompt can also specify a task at the moment the model is used. You can describe the desired result or provide examples, and the model conditions its continuation on them. This is often called in-context learning because the task information is supplied in the input rather than installed through a new round of weight updates.

“Predict the next token” can therefore sound simpler than the behavior it produces. The objective is simple. The learned function that performs it is not. Ordinary autocomplete and an LLM solve related prediction problems, but they differ greatly in model capacity, context use, training breadth, and the range of patterns they can apply.

Why LLMs matter

One pretrained model can support drafting, extraction, classification, translation, summarization, question answering, and code generation through changes to the input. That makes language a flexible interface for software.

The same flexibility creates risk. LLM output is generated, not looked up from an authoritative source. Results can be wrong, biased, inconsistent, or sensitive to prompt wording. The right question is not simply whether an LLM “knows” a topic. Reliability depends on the task, the training and supplied context, the decoding method, and any checks performed by the surrounding system.

An LLM is also only one layer of an AI product. It does not automatically have a user account, durable memory, web access, private documents, or permission to take actions. Those capabilities come from software around the model.

Common misconceptions

“An LLM is a chatbot”

An LLM is the model that processes or generates language. An AI chatbot is a product that may combine an LLM with a conversation interface, instructions, retrieval, moderation, memory, and other software. The same LLM can be used without a chat interface.

“The model looks up its answer”

The model normally calculates a continuation from its weights and current context. A product can separately search the web or retrieve documents and place the results into the prompt, but that is a capability of the surrounding AI harness, not proof that the LLM itself performed a lookup.

“Fluent language proves that the model understands”

Researchers do not agree on one test for understanding. An LLM can demonstrate useful language competence and form internal representations that support complex tasks. That does not by itself establish human-like meaning, experience, intent, or grounding in the world. It is clearer to describe the behavior being measured than to use “understands” as an all-purpose explanation.

“It is only autocomplete, so it cannot do anything new”

An LLM is an autoregressive predictor, but “only” hides the important part: what the model learned and how much context it can condition on. It can combine patterns in ways that were not present as a complete passage in its training data. Novel output is still not evidence that every claim in it is correct.

“Large has a precise cutoff”

No governing definition sets a minimum number of parameters, training tokens, or units of compute. The boundary between a small language model and an LLM changes with usage and technology. A size claim is meaningful only when it names the quantity being compared.

How an LLM fits into a larger system

A basic stack looks like this:

1. The LLM maps context to possible language continuations.
2. A harness prepares context, calls the model, and may connect it to data or tools.
3. A chatbot presents that system as a conversation.
4. An AI agent adds a loop that can choose actions, inspect results, and continue toward a goal.

Keeping those layers separate makes product claims easier to evaluate. “The assistant searched the web” describes the whole system. “The LLM generated the search query and interpreted the returned text” describes the model's role more precisely.

Where to go next

Read What Is a Token in AI? to understand the units an LLM processes. Then see What Is a Context Window? for the limit on the information available during one generation, What Is Training in AI? for how the model's weights are learned, and What Is Inference in AI? for what happens when a trained model is used.