Chain-of-Thought (CoT)

Chain-of-thought (CoT) prompting is a technique for improving LLM accuracy on reasoning-heavy tasks by instructing the model to generate its intermediate reasoning steps before producing a final answer, rather than jumping straight to a conclusion. The term was introduced in a widely cited 2022 Google Research paper, which found that including a handful of worked examples showing step-by-step reasoning — or simply appending a phrase like “let’s think step by step” — produced substantial accuracy gains on math word problems, multi-step logic puzzles, and symbolic reasoning tasks.

Why It Improves Accuracy

Each token an LLM generates is produced by a fixed amount of computation — one forward pass through the network. For problems that genuinely require multiple reasoning steps, forcing the model to answer immediately compresses all of that reasoning into a single token’s worth of compute, which is often insufficient. Generating intermediate steps effectively spreads the reasoning across many forward passes, with each step’s output feeding back in as input context for the next, giving the model more total computation to work with and a chance to catch its own errors partway through.

Chain-of-Thought vs. Reasoning Models

Chain-of-thought started as a prompting technique applicable to any capable base model, but the same underlying insight — that visible intermediate reasoning improves final-answer accuracy — has since been built directly into training. Current “reasoning models” like OpenAI’s o-series and DeepSeek-R1 are trained with reinforcement learning specifically to produce long, self-correcting chains of thought as a core behavior, rather than relying on a prompt to elicit it. This produces stronger reasoning than prompted CoT on a standard model, at the cost of higher latency and token usage per response.

Prompting Techniques That Build on Chain-of-Thought

Several more advanced prompting techniques extend the core chain-of-thought idea. Self-consistency generates multiple independent chains of thought for the same problem and takes a majority vote over their final answers, which tends to improve accuracy further since errors in any single reasoning chain are less likely to be replicated across multiple independently generated ones. Tree-of-thought generalizes the single linear chain into a branching search over multiple possible reasoning paths, allowing a model to explore and backtrack from unpromising directions rather than committing to one path from the start. Both techniques trade meaningfully higher token cost and latency (since they require generating and evaluating multiple candidate reasoning chains rather than just one) for improved accuracy on especially difficult reasoning tasks.