What Is an LLM? A Plain-English Explainer
A large language model (LLM) is a system trained on enormous amounts of text to predict what word — or part of a word — comes next, given everything that came before it. That's the whole mechanism underneath. Everything an LLM appears to "know" or "reason about" is that prediction process, run at a scale where the outputs look like understanding.
How it actually works, in plain English
Text gets broken into small chunks called tokens — often whole words, sometimes word fragments. The model is trained to predict the next token, over and over, across a training set large enough that it has effectively absorbed patterns of grammar, fact, and reasoning style along the way. When you send it a prompt, it isn't looking anything up — it's generating the statistically most plausible continuation of your text, one token at a time, given everything it learned during training and everything currently in its context window.
This explains both what LLMs are surprisingly good at (fluent, contextually appropriate language across almost any topic) and their most-cited weakness: a fluent, confident-sounding continuation isn't the same thing as a correct one. When a model states something wrong with total confidence, it isn't lying — it's continuing the pattern in the most plausible way it has, and plausible isn't the same as true. That's what hallucination actually is at a mechanical level.
What an LLM can and can't do on its own
On the conceptual ladder, a bare LLM sits at the very first rung — an AI Model. It can generate text, classify, and reason within a single response. It cannot remember anything between calls, take an action in the world, or verify that what it produced is correct. Every capability above that — memory, tool use, iteration, governance — is architecture built around the model, not something the model does by itself.
This is the single most common source of confusion in this whole space: people evaluate "the AI" as if capability and product were the same thing. A model with excellent language ability, wrapped in no memory, no tools, and no permission structure, is not an agent — it's an assistant. See types of AI agents for the full six-rung breakdown, or agentic AI for what actually separates the two.
LLM vs AI agent — where the line actually is
An AI agent uses an LLM as its reasoning engine, but adds everything the model can't do on its own: memory across steps, the ability to call tools and take real action, and a loop that checks whether the action actually worked before deciding what to do next. The LLM decides what to say or propose; the surrounding system decides whether it's allowed to act on it. That division — the model proposes, the system disposes — is the entire basis of safe agent design, covered in more depth in how an AI agent actually works.
Why this matters if you're evaluating an agent platform
Vendors will often describe their product in terms of which model powers it — "built on the latest model" — as if that were the differentiator. It rarely is. Two products built on the identical underlying model can differ enormously in whether they hallucinate on your content, whether your data is isolated, and whether they escalate correctly when unsure. Those differences live in the architecture around the model, and in the governance built around it, not in the model itself.
Common LLM failure modes worth knowing
- Hallucination — confident output not grounded in the source material it was given.
- Context limits — a model can only "see" a finite amount of text at once; anything outside that window is genuinely invisible to it, not just deprioritised.
- Training cutoff — a model's knowledge of the world stops at whenever its training data was collected, which is why agents are given live, current knowledge to work from rather than relying on what the model already "knows."
Every one of these is a reason business-facing agents are scoped to answer from your own content rather than the model's general training, and a reason to weigh the risks of AI agents in business seriously before deploying one — not a flaw specific to any one vendor.
FAQ
Is ChatGPT an LLM? The underlying model is an LLM. The chat interface around it is an application built on that model — the distinction matters because the interface adds things (conversation memory within a session, safety filtering) the bare model doesn't have on its own.
Do bigger models hallucinate less? Not reliably, and not in a way that removes the need for grounding. Scale improves fluency and general capability, but a model with no access to your actual facts will still generate a plausible-sounding wrong answer when it doesn't know the real one.
Can an LLM learn from my conversations with it? Not in real time and not by default — a model's core training is fixed once deployed. What looks like "learning" within a conversation is the model using what's in its current context window, which disappears once the session ends unless a system is built to store it as agent memory.
Do I need to know how LLMs work to use an AI agent in my business? No — you need to know what to scope it to and what to check its output against. This explainer exists so that conversation makes sense, not because you need the underlying mechanics day to day.