LLM (Large Language Model)

What is an LLM?

An LLM is a transformer neural network with billions to trillions of parameters, trained on internet-scale text to do one deceptively simple thing: predict the next token. From that objective — plus post-training to follow instructions — emerge translation, coding, reasoning, and conversation. Examples: GPT, Claude, Gemini, Llama, Mistral, Qwen.

How they’re built (the three stages)

1. Pretraining — predict next tokens across trillions of words. Produces a “base model”: knowledgeable but not helpful.
2. Post-training — supervised fine-tuning on instruction examples plus preference optimization (RLHF and successors) makes it follow instructions and decline harmful requests.
3. Reasoning training — reinforcement learning on verifiable problems teaches models to “think” before answering; see reasoning models.

What developers actually need to know

An LLM is stateless — it remembers nothing between API calls; all “memory” is you re-sending the conversation in the context window. It’s probabilistic — the same input can produce different outputs (see temperature). And it’s a text-in, text-out function — every capability beyond that (tools, retrieval, memory) is engineering built around the model, not inside it.

What people get wrong

• Treating it as a database. Training knowledge is compressed, dated, and unverifiable — see hallucination; facts that matter should come from retrieval.
• Assuming it understands like a human. It excels at pattern completion; failures cluster where surface patterns diverge from underlying logic (arithmetic on unusual numbers, novel puzzle variants).
• Picking by leaderboard. Benchmark rank correlates weakly with performance on your task; run your own evals on candidate models.

What Makes Something a “Large” Language Model

The “large” in LLM refers primarily to parameter count and training data scale, though there’s no strict threshold that separates an LLM from a smaller language model — the term is used loosely to describe models trained on massive text corpora (often trillions of tokens) with billions to hundreds of billions of parameters, capable of general-purpose text generation, reasoning, and instruction-following, in contrast to smaller, task-specific models trained for a single narrow purpose. Modern LLMs are typically decoder-only transformers, trained first on next-token prediction over broad internet-scale text (pretraining), then further refined through supervised fine-tuning and preference-based alignment (like RLHF or DPO) to make them follow instructions helpfully and safely rather than simply continuing text in whatever direction is statistically most likely.

Why the Category Matters Practically

Understanding a system as “an LLM” rather than “an AI” more broadly matters because it clarifies both capabilities and failure modes: LLMs are fundamentally pattern-completion systems trained on text, which explains both their surprising fluency and reasoning ability, and structural limitations like hallucination, sensitivity to prompt phrasing, and a fixed training-data cutoff that no amount of clever prompting alone can fully overcome.