Reward Prediction Error: How the Brain Learns From Outcomes
Author: Ukrainian Psychological Hub · Published: October 12, 2026 · Editorial Policy
Reward prediction error (RPE) is a learning signal that expresses the difference between the value of an outcome and the value that was expected. When an outcome is better than predicted, a positive error can strengthen an expectation or an action. When it is worse than predicted, a negative error can weaken it. With experience, these updates help an organism anticipate consequences and select behavior under changing conditions. Research on dopamine neurons provides some of the strongest biological evidence for this computational principle, while contemporary studies show that the underlying neural signals are more diverse than a single dopamine equation. Schultz, Dayan, and Montague (1997) established the classic connection; Gershman and colleagues (2024) explain its modern limits.
What is a reward prediction error?
A reward prediction error is the signed discrepancy between the reward value predicted before an event and the reward value inferred from what actually happens. “Reward” in this framework is an experimentally defined favorable outcome, such as food, money, access to an opportunity, or another reinforcer that affects behavior. It is not equivalent to conscious pleasure. “Prediction” refers to an estimate shaped by experience, context, cues, and the structure of a task. The error is the new information available for updating that estimate.
A positive reward prediction error occurs when the outcome is better than expected. A negative reward prediction error occurs when it is worse than expected, including when a previously expected reward is omitted. When the outcome matches expectations, the signed error approaches zero. This last case matters: an event can be pleasant, important, or desired without generating a large reward prediction error if its value was already accurately anticipated.
Imagine three trips to the same café. On the first, a free dessert arrives unexpectedly; the positive error can increase the predicted value of that setting. On the second, the café reliably gives that same dessert; its arrival is still welcome, but offers little new information. On the third, the promised dessert is absent; the outcome is worse than expected, producing a negative error. The relevant quantity is the change between expectation and outcome, not simply whether dessert was present.
The sign and size of an RPE depend on how a task defines value, what the person or animal already believes, and when the prediction is evaluated. A five-dollar bonus can be surprisingly good when nothing was expected and disappointing when a larger bonus was anticipated. Researchers therefore estimate subjective value from choices, learning histories, or model parameters rather than assuming that nominal dollar amounts have identical value for everyone. Watabe-Uchida, Eshel, and Uchida (2017) review this mechanism and its neural circuitry.
The basic equation: actual outcome minus expected outcome
In a simple trial-based model, the prediction error can be written as δ = r − V, where r is the reward value observed on the current trial and V is the reward value predicted before the outcome. The learning rule V(new) = V(old) + αδ updates the expectation. The learning-rate parameter α describes how strongly the model changes its estimate in response to the error, usually between zero and one in this simple version.
Suppose a learner expects a reward worth 4 units and receives 7. The prediction error is +3. With a learning rate of 0.2, the updated prediction is 4 + 0.2 × 3 = 4.6. If the same learner instead receives 1, the error is −3 and the updated prediction is 3.4. These are illustrative model quantities, not physiological dopamine concentrations or measurements of subjective feelings.
The elementary formula captures outcome-based updating. Temporal-difference reinforcement learning extends the idea to unfolding sequences in which new cues and future opportunities matter. A common one-step expression is δ at time t = r at time t + γV(next state) − V(current state). Here γ discounts predicted future value. A new cue can therefore generate a positive error even when no immediate reward has yet appeared, because the cue increases the expected value of what comes next.
This computational distinction explains a classic observation: after learning, a dopamine response may occur when a previously neutral cue signals a future reward, while the fully expected reward itself elicits a much smaller transient response. The teaching signal can move toward the earliest informative event in the predictive sequence. This is a theoretical account tested in particular conditioning procedures, rather than a universal description of every dopamine neuron or reward event. Schultz et al. (1997); Amo (2024).
Reward prediction error, uncertainty, and learning rate
The amount of updating depends on more than error magnitude. Learning rate may change with volatility, the reliability of a cue, prior beliefs, and uncertainty about the environment. An unexpected outcome may signal a genuine change in circumstances or a single noisy observation. Rational updating should depend on which explanation is more likely. The Pearce–Hall approach also emphasizes how surprise changes subsequent attention or associability, while Rescorla–Wagner-type models emphasize correction of associative strength. These are related but distinct functions of prediction error. Holland and Schiffino (2016).
An unsigned prediction error measures the magnitude of unexpectedness regardless of whether the outcome is favorable or unfavorable. A signed reward prediction error retains the direction of the value discrepancy. Researchers sometimes find signals related to unsigned surprise, salience, or uncertainty and must distinguish them empirically from reward-specific signed errors. A large unexpected loss and a large unexpected gain could both be surprising while driving updates in opposite directions.
How the idea developed: from learning theory to dopamine neurons
Long before neural recordings provided a proposed biological implementation, experimental psychology showed that associations depend on informative relationships among events. In blocking experiments, an already reliable predictor of reward can reduce learning about a second cue introduced alongside it: the reward is adequately predicted by the first cue, leaving little error to drive learning about the newcomer. This finding motivated models in which surprise or mismatch controls learning, including the influential Rescorla–Wagner framework. Holland and Schiffino (2016) review these theoretical traditions.
The connection to dopamine became influential through recordings from neurons in the midbrain of nonhuman primates. In the classic sequence, an unexpected reward evokes a brief increase in firing; after conditioning, the response shifts toward the reward-predicting cue; when an expected reward fails to appear, activity can dip around its expected time. The pattern aligns with a signed temporal-difference teaching signal and inspired a productive synthesis of associative psychology, machine learning, and cellular neuroscience. Schultz, Dayan, and Montague (1997).
Later experiments tested causality rather than correlation alone. In rats, Steinberg and colleagues (2013) stimulated dopamine neurons at carefully selected moments and showed changes in cue–reward learning consistent with an artificially supplied prediction error. A more recent formal test separated competing explanations in which stimulation might either add value to a reward or modify the learning error itself; it strengthened the argument that dopamine manipulations can alter error-driven learning in the studied tasks. Usypchuk and colleagues (2025). These are strong causal findings within specific animal paradigms, with questions of transfer to everyday human behavior still requiring separate evidence.
A 2026 historical review traces this development from behavioral learning rules through temporal-difference algorithms and the growing debate over heterogeneity of dopamine functions. Dudhabhate and Costa (2026).
Which brain circuits are involved?
Reward prediction error arises through interactions among neural populations and circuits. Midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra pars compacta are central to the classical account. They receive diverse excitatory and inhibitory inputs, and different subpopulations project to different targets. The nucleus accumbens and other regions of the ventral and dorsal striatum participate in learning from outcomes and adjusting behavioral tendencies. Prefrontal regions contribute to representing goals, task states, choices, and expected outcomes. For a broader account of the circuits involved, see Brain Reward System: Circuits, Reinforcement, and Motivation and Neural Circuits: How Brain Networks Generate Behavior.
The orbitofrontal cortex is relevant to predictions about what kind of outcome is expected, not only how much reward it is worth. The hippocampus and related memory circuits can supply contextual and relational information that determines which state or situation is being predicted. The amygdala contributes to learning about motivationally significant cues. The lateral habenula and connected inhibitory pathways have been investigated as components involved in evaluating unfavorable outcomes and regulating midbrain dopamine signaling. These structures participate in larger interacting systems; a one-region account would miss the distributed computation. Watabe-Uchida et al. (2017); Kahnt and Schoenbaum (2025).
A neuron can also represent several task variables depending on the circumstances. Different dopamine cells vary in projection targets, response profiles, and their relationship to movement, sensory events, value, and inferred states. Modern research therefore investigates populations and circuit pathways rather than treating all dopamine neurons as interchangeable copies of a single scalar reward meter. Lee and colleagues (2024); Gershman et al. (2024).
How dopamine may contribute to learning
Transient increases and decreases in dopamine neuron firing can carry teaching-relevant information. Dopamine released at targets such as the striatum can modulate neural plasticity and the consequences of coincident activity at synapses, providing one route by which errors influence subsequent choice or cue associations. The effect depends on receptors, local circuit activity, timing, prior learning, and the specific task. A change in firing in the VTA is not identical to a measurement of extracellular dopamine in a terminal region, and neither measurement directly reveals a person's experienced pleasure. For the larger neurochemical picture, see Dopamine: Reward Prediction, Learning, Motivation, and Movement.
Phasic dopamine activity refers to changes on relatively short time scales, often in relation to a cue, action, or outcome. Slower variations in dopamine-related activity have been studied in relation to motivation, opportunity, movement, and context. Researchers must specify whether they measured spikes, release, concentration, receptor binding, or an indirect hemodynamic signal before describing a result as “dopamine activity.” Berke (2018); Gershman et al. (2024).
How scientists measure reward prediction errors
Behavioral experiments create sequences of cues, choices, probabilities, and outcomes. Researchers first fit a computational model to the participant's observable behavior, often estimating learning rates and predicted values. The model calculates a trial-by-trial RPE. Neural measurements can then be tested for association with that modeled quantity. The fit of a neural signal to one proposed model depends on the model, behavioral quality, experimental design, and comparison with plausible competing accounts.
Single-unit electrophysiology measures the timing of action potentials from recorded neurons and can show responses around particular task events. It does not by itself identify the synaptic concentration of dopamine at a distant target. Calcium indicators and fluorescent dopamine sensors provide other biological signals, but they involve specific temporal dynamics and calibration constraints. Optogenetic interventions can provide causal tests in experimental animals by changing defined populations at selected times; findings require careful interpretation with respect to species, targeting, and behavior.
In human fMRI research, the blood-oxygen-level-dependent (BOLD) signal indexes vascular responses associated with underlying neural activity. Researchers have identified BOLD changes in areas such as the ventral striatum and orbitofrontal cortex that covary with model-derived temporal-difference errors during appetitive conditioning. O'Doherty and colleagues (2003) provided an influential demonstration. A BOLD correlation with an RPE regressor is evidence consistent with a modeled computation under the study's assumptions. It is not a direct assay of dopamine release, proof that a particular cell type performs the computation, or confirmation of causality.
EEG records scalp electrical potentials produced mainly by the summed activity of neural populations. Researchers study feedback-related components and frequency-specific activity in outcome evaluation, but scalp EEG cannot directly measure neurotransmitter levels and source localization is constrained. PET can measure radiotracer-dependent aspects of receptor availability or neurochemical function over much slower timescales; it does not visualize millisecond reward-prediction errors as an ordinary color map. The interpretation of any method depends on spatial and temporal resolution, measurement validity, and how the experimental hypothesis was tested.
Cross-method convergence is especially valuable. Behavioral modeling helps specify the hypothesized computation; single-neuron recording tests temporal patterns; interventions test whether manipulating cells changes learning; and human imaging helps investigate how related computations are distributed during human tasks. The strength of a claim rises when complementary methods support it while alternative value, salience, motor, and sensory models are examined. Watabe-Uchida et al. (2017); Gershman et al. (2024).
What the evidence establishes—and what remains contested
The established core is substantial. Prediction discrepancies are useful explanatory variables in behavioral learning, several midbrain dopamine populations show task-dependent activity resembling signed reward prediction errors, and interventions in animal models show that certain dopamine transients can causally influence learning. Human tasks reveal behavior and distributed imaging signals consistent with formal reinforcement-learning accounts. These convergent lines make the RPE framework a major achievement of computational neuroscience. Schultz et al. (1997); Steinberg et al. (2013); O'Doherty et al. (2003).
The limits are equally important. The classical theory assumes that value can be summarized by one scalar and that a suitable signal broadcasts deviations from that estimate. Real dopamine responses may encode sensory or motor features, belief about hidden states, expected reward identity, context, movement, and information about the range of possible outcomes. The relative explanatory power of generalized prediction-error theories and other accounts continues to be tested. This is an active scientific dispute about mechanisms and model scope, not a rejection of the well-supported original findings. Gershman et al. (2024); Kahnt and Schoenbaum (2025).
Does every dopamine neuron compute the same error?
No. Studies have identified heterogeneity across dopamine cells and projections. A prominent extension proposes that parallel neural populations represent different portions of a distribution of future reward, with relatively optimistic and pessimistic value estimates. Dabney and colleagues (2020) reported experimental support in mouse VTA recordings for a distributional coding framework. It offers a more expressive model than one population-wide average, although its applicability varies across species, tasks, and neural targets.
Another extension concerns features of expected outcomes. A reward might arrive in the anticipated amount but have an unexpected flavor, identity, or sensory property. Pure scalar value error could be zero while meaningful information about the world has changed. Experiments and reviews examine whether dopamine transients help learning about such features and associations, including situations where a cue predicts another cue. Kahnt and Schoenbaum (2025); Lee et al. (2024). Whether a particular neural response should be called a generalized prediction error or a distinct learning signal depends on testable model definitions.
Why do some dopamine signals ramp up before reward?
Gradually changing signals as an animal approaches a reward have encouraged competing interpretations involving anticipation, value, movement, uncertainty about state, and continuous temporal-difference updates. Some ramping patterns can arise within extended RPE frameworks, while others may require additional mechanisms. The correct interpretation is experiment-dependent. A rise in a dopamine-related measure before reward should not automatically be translated into escalating pleasure. Amo (2024); Gershman et al. (2024).
Is dopamine simply the brain's reward or pleasure chemical?
Dopamine is involved in more functions than reward prediction errors, including movement, aspects of motivation, exploration, and action selection. The experience of pleasure and the organization of motivated behavior involve numerous circuits, neurotransmitters, body signals, expectations, and contextual factors. Evidence that dopamine participates in reward learning cannot be converted into the formula “dopamine equals happiness.” Berke (2018); Gershman et al. (2024).
RPE versus related concepts
Reward and reward prediction error describe different variables: the former denotes an outcome's reinforcing value, whereas the latter denotes its deviation from expectation. A reward can be large and fully predicted; a very small outcome can generate a substantial negative error if it violates an even larger expectation. Salience refers broadly to what attracts processing priority and does not necessarily carry the sign of value change. Surprise is often quantified as unsigned deviation or improbability, although precise meanings differ among theories.
Reward prediction error is also distinct from a punishment prediction error. Researchers may code adverse outcomes with separate utility functions or separate learning channels, and identical numerical signs across studies do not guarantee equivalent psychological meaning. Prediction error in sensory processing concerns mismatch between expected and observed sensory information; reward prediction error concerns values assigned within a learning or decision model. These signals can interact without being interchangeable.
Model-free reinforcement learning updates cached values or action preferences based on experience. Model-based learning additionally uses representations of transitions and future consequences. Dopamine responses can reflect model-based information about expected reward features, complicating a rigid identification of dopamine with one kind of model-free algorithm. Langdon and colleagues (2018); Kahnt and Schoenbaum (2025).
How reward prediction errors affect everyday learning
In everyday settings, outcomes can change the attractiveness of actions, cues, and opportunities. A surprising success may alter expectations, while repeated predictable success may gradually yield smaller updating signals. An unexpected setback may prompt revision of a strategy, depending on whether the event is considered informative. These principles offer a conceptual way to understand trial-and-error learning, but laboratory RPE estimates do not independently predict an individual's choices outside the task.
Learning depends on environmental structure and feedback quality. Clear, timely, relevant information can help a learner infer which action led to which outcome. Highly noisy or misleading rewards can encourage unstable estimates. Some contexts require patience with uncertain evidence rather than immediate adjustment after each setback. Educational interventions, management incentives, habit formation, and product design can be discussed using reinforcement-learning concepts, yet their effectiveness must be established with direct studies of those interventions.
For a broader account of automaticity and everyday behavior, see Habit vs. Routine: Differences in Automaticity and Intention. For discussion of nutrition-related reward claims, see Sugar and Dopamine: Reward, Motivation, and Common Myths. The Coffee and Dopamine article addresses a different neurochemical question and keeps caffeine-specific claims under their existing canonical owner.
Does reward prediction error explain addiction or psychiatric disorders?
Reinforcement learning is relevant to research on habits, substance use, mood, and decision-making, but a fitted prediction-error parameter or striatal fMRI contrast does not establish a psychiatric diagnosis, a universal disorder mechanism, or an individual's prognosis. Clinical presentations are multifactorial. Medication studies may help evaluate causal pharmacological influences while remaining sensitive to task design and selection of healthy versus clinical participants.
A 2025 systematic review and meta-analysis of 102 controlled human studies found that dopaminergic manipulation was associated with increased reward learning or sensitivity and reward response vigor, with generally small-to-moderate effects depending on the component. The included experiments were in healthy volunteers; the authors describe candidate mechanistic markers, not validated individual diagnostic tests or established treatment-selection tools. Such results are informative for theory without licensing claims that a person's “dopamine prediction error is broken” based on daily behavior or a consumer questionnaire.
What does this have to do with artificial intelligence?
Temporal-difference learning uses a prediction error to improve an agent's estimates of future reward in reinforcement learning. Some machine-learning systems learn values or policies by updating estimates in response to outcomes, and distributional reinforcement-learning methods model several possible returns rather than only the average. Neuroscience and AI have historically informed each other in this area. The computational resemblance identifies a useful mathematical relationship; it does not establish that biological and artificial systems have identical architectures, goals, experiences, or learning environments. Dabney et al. (2020); Dudhabhate and Costa (2026).
Common misconceptions
“Dopamine is released whenever something feels good.” Dopamine dynamics are more varied, and the act of enjoying an expected event can coexist with a small reward-prediction error.
“A negative prediction error means a person is unhappy.” A negative RPE is defined relative to a modeled value expectation; it is neither a mood score nor a direct self-report of subjective experience.
“A bright spot on fMRI shows dopamine.” fMRI measures a hemodynamic signal. A model-based fMRI association provides indirect evidence for an involved computation and needs convergent methods for neurotransmitter-specific interpretation.
“Surprising rewards always create the best learning.” Surprise has to be interpretable, sufficiently informative, and integrated with prior experience. The learning rate and causal structure of feedback matter.
“Every dopamine neuron broadcasts the same RPE.” Neural subpopulations, timing, targets, and task context generate heterogeneity that current theories aim to explain.
“A reward-prediction-error test can diagnose mental illness.” Laboratory behavioral indices and candidate neurobiological correlates do not presently serve as broadly validated stand-alone psychiatric diagnostic tools.
Frequently asked questions
What is a simple example of reward prediction error?
If you expected no bonus but received one, the result was better than predicted and the error is positive. If you expected a bonus and received none, the error is negative. If the bonus was completely expected, the new information may be small even though the bonus still has value.
What does a positive reward prediction error mean?
It means the outcome, or newly available prediction about the future, has greater modeled value than expected immediately beforehand. In simple learning rules, a positive error increases the predicted value of the relevant cue or action.
What does a negative reward prediction error mean?
It means the obtained outcome has lower modeled value than anticipated. In classical dopamine recording paradigms, an omitted expected reward can be accompanied by a transient dip in some dopamine neurons' activity. Its psychological consequences depend on context and the learning system involved.
Is reward prediction error the same as dopamine?
No. RPE is a computational variable; dopamine is a neurotransmitter participating in several biological processes. Particular dopamine signals can correlate with, and in certain tasks causally influence learning in ways predicted by, RPE models. RPE computations can also be studied in neural networks and artificial algorithms without equating every computation with dopamine.
Does dopamine increase when a reward is unexpected?
Classic recordings often show transient increases in dopamine neuron firing following unexpectedly favorable reward. Other neural populations, circuits, and task conditions can show different response patterns. Whether dopamine release rises in a target region requires a method that measures release there.
Why does the response move from reward to a cue?
After a predictive cue is learned, the cue provides new information that future reward is more likely, so a temporal-difference error can occur at cue onset. If the reward then arrives exactly as predicted, its delivery contributes little additional prediction error.
Can a large reward produce a small prediction error?
Yes. If its value was fully anticipated, the mismatch can be close to zero. Conversely, a modest outcome can produce a meaningful prediction error when it strongly differs from the prior expectation.
Is prediction error always conscious?
No. Researchers infer many prediction-related computations from behavioral and physiological responses that do not require verbal awareness of the mismatch. Conscious expectation can influence learning in some circumstances, but awareness and model-estimated RPE are separate empirical questions.
Can reward prediction error be measured with a blood test or an ordinary brain scan?
There is no routine blood test for an individual's moment-by-moment reward prediction error. Researchers estimate computational errors from tasks and may correlate them with fMRI, electrophysiology, or other measurements. Each technique has distinct inferential limits.
Can people deliberately use reward prediction error to build habits?
Predictable feedback and learning from consequences can help shape repeated behavior, but personal habit formation also depends on context, action opportunities, repetition, attention, and individual goals. RPE offers a mechanistic model, not a validated one-step habit-building intervention.
Reward prediction error is one particular learning computation. For a broader discussion of predictions, sensory discrepancies, and competing neural theories, explore predictive processing models of sensory inference.
Related Articles
References
Amo, R. (2024). Prediction error in dopamine neurons during associative learning. Neuroscience Research, 199, 12–20. https://doi.org/10.1016/j.neures.2023.07.003
Berke, J. D. (2018). What does dopamine mean? Nature Neuroscience, 21, 787–793. https://doi.org/10.1038/s41593-018-0152-y
Dabney, W., Kurth-Nelson, Z., Uchida, N., et al. (2020). A distributional code for value in dopamine-based reinforcement learning. Nature, 577, 671–675. https://doi.org/10.1038/s41586-019-1924-6
Dudhabhate, B. B., & Costa, K. M. (2026). A brief history of dopamine prediction errors. Frontiers in Computational Neuroscience, 20. https://doi.org/10.3389/fncom.2026.1825622
Gershman, S. J., Assad, J. A., Datta, S. R., et al. (2024). Explaining dopamine through prediction errors and beyond. Nature Neuroscience, 27, 1645–1655. https://doi.org/10.1038/s41593-024-01705-4
Holland, P. C., & Schiffino, F. L. (2016). Mini-review: Prediction errors, attention and associative learning. Neurobiology of Learning and Memory, 131, 207–215. https://doi.org/10.1016/j.nlm.2016.02.014
Kahnt, T., & Schoenbaum, G. (2025). The curious case of dopaminergic prediction errors and learning associative information beyond value. Nature Reviews Neuroscience, 26, 169–178. https://doi.org/10.1038/s41583-024-00898-8
Langdon, A. J., Sharpe, M. J., Schoenbaum, G., & Niv, Y. (2018). Model-based predictions for dopamine. Current Opinion in Neurobiology, 49, 1–7. https://doi.org/10.1016/j.conb.2017.10.006
Lee, R. S., Sagiv, Y., Engelhard, B., et al. (2024). A feature-specific prediction error model explains dopaminergic heterogeneity. Nature Neuroscience, 27, 1574–1586. https://doi.org/10.1038/s41593-024-01689-1
Mkrtchian, A., Qiu, Z., Abir, Y., et al. (2025). Differential associations of dopamine and serotonin with reward and punishment processes in humans: A systematic review and meta-analysis. JAMA Psychiatry, 82, 818–829. https://doi.org/10.1001/jamapsychiatry.2025.0839
O'Doherty, J. P., Dayan, P., Friston, K., Critchley, H., & Dolan, R. J. (2003). Temporal difference models and reward-related learning in the human brain. Neuron, 38, 329–337. https://doi.org/10.1016/S0896-6273(03)00169-7
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275, 1593–1599. https://doi.org/10.1126/science.275.5306.1593
Steinberg, E. E., Keiflin, R., Boivin, J. R., et al. (2013). A causal link between prediction errors, dopamine neurons and learning. Nature Neuroscience, 16, 966–973. https://doi.org/10.1038/nn.3413
Usypchuk, A. A., Maes, E. J. P., Lozzi, M., et al. (2025). Disentangling prediction error and value in a formal test of dopamine's role in reinforcement learning. Current Biology, 35, 4019–4027.e7. https://doi.org/10.1016/j.cub.2025.06.076
Watabe-Uchida, M., Eshel, N., & Uchida, N. (2017). Neural circuitry of reward prediction error. Annual Review of Neuroscience, 40, 373–394. https://doi.org/10.1146/annurev-neuro-072116-031109