Judgments of Learning: How We Predict What We Will Remember

19 hours ago

17 min read

Author: Angela Bogdanova · Published: October 16, 2026 · Editorial Policy

Research cluster: Consciousness, Metacognition & Self-Awareness · Written by Artificial Sapiens

A judgment of learning (JOL) is a prediction about whether information that has just been studied will be remembered on a later test. When a learner reads a definition and estimates a 70% chance of recalling it tomorrow, that estimate is a judgment of learning. JOLs are a form of metamemory and metacognitive monitoring: they describe what someone expects to remember, rather than how much that person objectively remembers. Their usefulness depends on whether the cues supporting the prediction actually forecast later retrieval. Decades of experimental research show that the timing and format of these judgments can substantially change their accuracy. Rhodes and Tauber's meta-analysis provides a central synthesis.

The most reliable practical finding is that judgments made after a short delay, while attempting to retrieve an answer from a cue, often discriminate better between remembered and forgotten material than judgments made immediately after reading the complete answer. This is known as the delayed judgment-of-learning effect. The benefit varies with the test, material and cue, and making a prediction can itself change subsequent remembering. A 2025 meta-analysis of JOL reactivity reinforces that assessment and learning cannot always be separated.

What Is a Judgment of Learning?

The term judgment of learning refers to a person's prospective estimate of later memory performance for particular studied information. In a typical laboratory procedure, participants learn word pairs such as ocean–blue, judge how likely they are to recall the second word when shown the first, and subsequently take a cued-recall test. The judgment may be expressed on a numerical scale, as a probability from 0% to 100%, or as a rating such as low, medium or high. The later test provides an external criterion against which the prediction can be assessed.

This process belongs to metacognition, the monitoring and regulation of cognition, and specifically to metamemory, the knowledge and monitoring people apply to memory. Researchers study JOLs because monitoring can guide what to restudy, what to test again and when to stop practicing. The transition from prediction to a study decision requires a further process—metacognitive control—and a high-quality prediction does not guarantee an effective choice.

A concrete example

Imagine studying ten Spanish vocabulary pairs. Immediately after viewing gato–cat, you rate your chance of recalling cat from gato tomorrow at 95%. The answer is still visible, the pronunciation feels familiar and comprehension is easy. Twenty minutes later you see gato alone and find that you can produce cat readily; after seeing another cue, however, you cannot retrieve its translation. A later judgment informed by this attempt distinguishes those two items more effectively. The scientific question concerns whether the resulting pattern of judgments predicts subsequent test outcomes across the full set of items, rather than whether either rating feels convincing.

The History of Judgments of Learning Research

Modern JOL research developed within the experimental study of metamemory and self-regulated learning. Early work asked how accurately people could predict recall after study; subsequent experiments systematically altered the interval between study and judgment. In 1991, Thomas Nelson and John Dunlosky reported that judgments elicited shortly after study, from a retrieval cue, could show exceptionally strong correspondence with later recall. Their original delayed-JOL report made prediction timing a central issue.

Follow-up experiments clarified that delay alone was an incomplete explanation. When learners saw the cue without the response, the advantage was especially strong; when they saw the entire cue–response pair again, the advantage could be much smaller. Dunlosky and Nelson's 1992 experiments helped establish the importance of the information available at judgment. Koriat's cue-utilization framework then organized research around the sources people consult when estimating future memory.

How Judgments of Learning Are Formed

A JOL is an inference. Learners cannot directly inspect tomorrow's memory; they evaluate information available now and infer whether retrieval will later succeed. The cues used to make that inference may reflect a studied item's characteristics, the study conditions or the learner's own processing experience. As Koriat (1997) argued, the predictive value of a cue depends on how it relates to later recall, and the influence of a cue on confidence need not match its influence on retention.

Intrinsic cues: properties of the material

Intrinsic cues are characteristics of the item being learned: a word pair's semantic relation, the apparent complexity of a passage, familiarity of a concept or the meaningfulness of a sentence. Learners often expect an easy-looking item to be remembered more reliably than a difficult-looking item. Such judgments can be useful when the feature is genuinely associated with test performance. They can mislead when an item is easy to understand on first exposure but hard to reproduce without supporting context.

Extrinsic cues: conditions of study

Extrinsic cues arise from the learning procedure, such as repetitions, study time, presentation duration or the presence of an explanation. These conditions can influence actual learning, yet they are not always weighted appropriately in predictions. A learner may underestimate the benefit of distributed practice because spacing feels effortful and overestimate repeated exposure because rereading feels familiar. The broader evidence on effective learning techniques distinguishes durable retrieval-based practice from the subjective smoothness of studying. See the review by Dunlosky and colleagues.

Mnemonic cues: information from ongoing memory processing

Mnemonic cues include the speed with which an answer comes to mind, ease of processing, apparent accessibility of a trace and the experience of successful retrieval. These cues frequently dominate judgments because they are directly experienced. Their diagnosticity changes with the situation. Smooth recognition of a printed answer tells a learner something different from retrieving that answer when only a question is shown. Laboratory evidence supports a role for both feelings of fluency and explicit beliefs about learning, while debate continues over their relative contributions and experimental identification. A methodological review of fluency and beliefs details this problem.

Immediate vs. Delayed Judgments of Learning

Immediate judgments

Immediate JOLs are obtained during or directly after an item's presentation, commonly with the studied material still recently accessible. They are simple to collect and reveal how learners assess new information. Their weakness is that current availability can be mistaken for future retrievability. A fluent word pair can earn a high prediction while its unprompted recall remains fragile. That pattern helps explain illusions of competence during study, documented experimentally by Koriat and Bjork (2005).

Delayed judgments

Delayed JOLs are made after a period intervenes between initial study and prediction. In influential experiments, the cue word is shown by itself and the learner assesses how likely the associated response will be produced later. The learner may attempt retrieval, experience either success or failure, and use that result to estimate future memory. Because the full answer is absent, the judgment may rely more on information from long-term memory than on immediate access to the just-presented pair.

A meta-analysis covering 112 effect sizes and 4,554 participants reported a large advantage of delayed over immediate judgments for relative accuracy (Hedges's g = 0.93). Its separate analysis of 98 effect sizes involving 3,807 participants found a much smaller average advantage for memory performance itself (g = 0.08). These are distinct outcomes: identifying which items will be remembered and actually remembering more items. Both estimates come from Rhodes and Tauber (2011); neither means a delayed rating is perfectly calibrated for every learner or test.

Why the cue matters

The classic delayed-JOL effect is strongest in settings where the judgment is made from the test-relevant cue alone. Re-presenting the intact word pair may provide fluency from seeing the answer, weakening the informational advantage of delay. In a notable experiment, all 45 participants showed the effect when delayed JOLs were prompted by the stimulus alone, whereas the corresponding effect with the entire stimulus–response pair was small or absent. This striking result came from a particular paired-associate paradigm, not from every kind of educational task. Dunlosky and Nelson (1992) are the primary source.

Why Delayed Judgments Can Be More Accurate

Several explanations overlap. The accessibility account emphasizes information available from long-term memory: if a learner can retrieve an answer from the cue after a delay, that success offers a useful forecast of later retrieval. Cue-diagnosticity accounts emphasize how closely the information driving a judgment tracks what determines performance on the final test. Testing-based explanations note that attempting retrieval can strengthen what is being measured. An account based on similarity between the JOL cue and test cue has also been examined, with mixed support for a purely matching-based explanation. A direct challenge to cue similarity as the primary cause appears in Dunlosky and Nelson (1997).

An important methodological complication is that delayed judgments often produce more extreme ratings. If the scale changes with delay, common accuracy indices can also change. Nevertheless, experiments and simulations by Weaver and Kelemen (1997) concluded that shifts to the ends of the rating scale explained only part of the delayed-JOL advantage. Additional experiments found that short-term distraction, delays and retrieval opportunities can each alter memory as well as reported confidence. Kelemen and Weaver (1997) illustrate why no single mechanism completely explains all findings.

How Scientists Measure JOL Accuracy

The distinction between confidence and accuracy

A JOL is a prediction; a later memory test provides an outcome. A confidence rating is neither a direct measure of stored memory nor proof of eventual success. Scientists therefore examine how closely ratings correspond to observed performance. For example, a learner who assigns 90% confidence to every word pair and recalls only 40% is overconfident on that set, even if some of those pairs are remembered. Conversely, a person may give generally low ratings but reliably rank easy items above difficult ones. Those two cases differ in calibration and resolution.

Absolute accuracy and calibration

Absolute accuracy concerns the agreement between predicted probability and observed performance. If items assigned 70% confidence are eventually recalled 70% of the time across a suitable sample, that subset is well calibrated. Researchers may examine the difference between mean JOL and mean test performance or use calibration curves across confidence bands. These summaries depend on the rating scale, task structure and number of observations. Aggregate calibration can hide item-level misordering; a learner can predict the overall proportion correctly while misidentifying which specific answers will be remembered.

Relative accuracy and resolution

Relative accuracy, sometimes called resolution, asks whether a learner gives higher judgments to items subsequently recalled than to items subsequently forgotten. The Goodman–Kruskal gamma correlation has often been used in JOL experiments, but it can be sensitive to ties and to distributional features of judgments. Within-person correlations and discrimination-oriented analyses offer alternatives. A high gamma score signifies successful ranking in a particular item set; it does not mean that an estimate of 80% literally corresponds to an 80% future recall probability. Rhodes and Tauber (2011) focus on relative accuracy in the delayed-JOL synthesis.

First-order memory performance vs. second-order monitoring

Memory accuracy answers the first-order question: was the response correct? Metacognitive monitoring addresses a second-order question: how effectively did the learner predict the correctness of that response? These outcomes can vary independently. Someone may recall many items but be poorly calibrated; another person may remember fewer items yet know reliably which ones remain uncertain. The broader science of metacognitive sensitivity emphasizes the need to interpret subjective confidence in relation to actual task performance, not as a substitute for it.

What Distorts Predictions of Future Memory?

Fluency and familiarity

Repeated exposure can make text feel increasingly easy to read, producing a sense that learning is complete. That experience may be helpful when familiarity is the later test's target, but it is less informative when the target is independent recall, transfer or explanation. Learners also face difficulty when a material's immediate presentation contains hints that will be missing later. Koriat and Bjork's experiments showed how study-time information can create misleading impressions of competence.

Beliefs about effective study

Learners may treat ease and rapid progress as signs of durable learning. The error becomes especially clear when spacing or interleaving produces better later performance despite feeling more difficult during study. In an experiment involving the identification of artists from paintings, participants tended to favor grouped examples even though spaced examples better supported later induction. Kornell and Bjork (2008) demonstrate a discrepancy between people's evaluation of a learning arrangement and its measured outcome. Those judgments concern learning effectiveness more broadly and should not be treated as equivalent to item-level JOLs.

Framing and the wording of the question

How a rating is requested can influence the response. Asking how likely an item is to be remembered is logically related to asking how likely it is to be forgotten, but experimental ratings and the resulting study decisions may differ across these frames. Finn (2008) reported effects of remember-versus-forget framing on metacognitive monitoring and control. This is a reminder that JOL scales measure judgments elicited by a particular question under particular instructions.

Task mismatch and retention interval

A prediction for a recognition test cannot be assumed to forecast free recall or problem solving equally well. Similarly, the judgment's usefulness may change when a test takes place after minutes, days or weeks. A student who judges the ability to identify a correct multiple-choice answer may be estimating a different outcome from an instructor who expects a written explanation without cues. Good JOL design specifies the future criterion clearly and makes the judgment context meaningfully comparable to it.

Do Judgments of Learning Change Memory? The Reactivity Problem

A fundamental assumption in many assessment procedures is that asking a learner to predict performance merely observes a preexisting state. JOL research shows why that assumption requires testing. Making the judgment may focus attention on the material, prompt comparison of cues and responses, or induce retrieval and additional elaboration. The procedure can thus influence later performance. Researchers call this JOL reactivity, and its effects can depend on the type of material, the relation between paired words, the test and the study design.

The 2025 meta-analysis by Ingendahl, Halamish, and Undorf synthesized 344 effects from 175 independent experiments involving 15,079 adults. Across included conditions, immediate JOLs had a small positive average effect on memory (g = 0.22, 95% CI 0.17–0.27). For related word pairs the average effect was positive (g = 0.44), while unrelated pairs showed a small negative average effect (g = −0.09). Moderator analyses indicated that effects varied considerably, and the authors reported indications of publication bias favoring positive findings. The evidence therefore supports a conditional claim: asking for immediate predictions can sometimes improve memory and sometimes impair it.

The reactivity result matters for both theoretical models and educational recommendations. When a delayed JOL involves attempting retrieval, a subsequent memory improvement may partly arise from retrieval practice. Higher judgment accuracy and improved later recall must be analyzed separately. An assessment tool can be useful in study routines without being psychologically inert.

Judgments of Learning, Study Decisions, and Self-Regulated Learning

Students use metacognitive information to decide where to invest time. If low judgments identify material that is likely to be forgotten, those items may become candidates for another study session, retrieval test or explanatory practice. If the judgments are inaccurate, study time may be allocated inefficiently. This is the link between monitoring and control. Effective self-regulation also depends on task demands, available time, goals, difficulty and the learner's choice of intervention.

In one study of text comprehension, participants who generated keywords after a delay showed greater monitoring accuracy, more effective regulation of study and stronger subsequent test performance than comparison conditions. The evidence supports a pathway from better monitoring to better study allocation in that experimental arrangement; it does not prove that every delayed self-rating will improve every course outcome. Thiede, Anderson, and Therriault (2003) provide the primary test.

A sensible decision process uses the outcome of an attempt to recall material, then directs further practice to gaps without neglecting already learned items forever. The broader evidence for practice testing and distributed practice is reviewed by Dunlosky and colleagues (2013). That review studies learning techniques, not a universal training program for JOL accuracy; the two bodies of evidence should be connected without conflating them.

A Practical Way to Make Better Predictions

First, define the actual performance you want to predict. For a vocabulary course, that might be producing a translation tomorrow from the foreign-language cue, rather than recognizing the translation when both words are visible. For a textbook, it might be explaining an idea from a heading without reopening the page. The forecast is more interpretable when the target event and delay are specified.

Second, let some time pass after initial study and remove the answer. Show yourself the question, title, problem statement or front of a flashcard. Attempt to retrieve the response before making the judgment. Record whether retrieval succeeded and how complete the answer was. The delayed cue-only arrangement has the strongest classic evidence for improved discrimination in paired-associate learning. Its extension to extended materials requires appropriate adaptations and realistic expectations.

Third, make an item-specific prediction using a consistent rating scale, such as 0–100%. A low rating can identify material for another round of study or practice testing. Avoid interpreting the number as a clinical score or a precise measurement of one brain's stored knowledge. Treat it as a forecast that can be checked later.

Fourth, take the later test under conditions resembling the intended task, compare predicted and actual results, and look for systematic patterns. Were high-confidence responses often wrong? Were judgments more accurate when you had attempted recall? Did your confidence reflect recognition rather than production? Repeating this process can make study decisions more evidence-informed, although broad claims of generalized improvement require separate testing.

A worked example with four flashcards

Suppose a learner rates four cards after a ten-minute gap, viewing only each question: Card A, 90%; Card B, 80%; Card C, 40%; Card D, 20%. On tomorrow's test, A and C are correct while B and D are incorrect. The learner forecast an average success probability of 57.5%, and actual recall was 50%. These averages are close, but Card B was overestimated and Card C underestimated. One set of four cards is too small for a stable estimate of calibration; the example illustrates why item-by-item checking adds information to a global feeling of preparedness.

Judgments of Learning Compared With Related Concepts

JOLs vs. feeling-of-knowing judgments

A judgment of learning ordinarily follows a study episode and predicts future memory for the studied material. A feeling-of-knowing judgment is typically elicited when someone currently fails to retrieve an answer and estimates whether they could recognize it or retrieve it under other conditions. Both are metamemory judgments, but they differ in timing, information available and criterion test. An experience of almost remembering something is also distinguishable from a calibrated forecast of eventual recall.

JOLs vs. retrospective confidence

A retrospective confidence judgment follows a response: after giving an answer, the person rates how likely that answer is to be correct. A JOL is usually prospective and anticipates a later retrieval attempt. Both may use probability scales, and both can exhibit miscalibration, but they answer different questions. A high JOL should never be interpreted as evidence that a particular answer has already been produced correctly.

JOLs vs. judgments of text comprehension

Some learners estimate whether they will answer future questions about a text, often called metacomprehension judgments. These share an inferential structure with JOLs but may require judgments about integrating ideas, drawing inferences or applying information, not merely recalling a cue–response pair. Methods that work for isolated word pairs do not transfer automatically to textbook chapters or professional reasoning. Experimental studies of delayed keyword generation offer one route into that distinct literature.

JOLs vs. metacognitive knowledge and control

Metacognitive knowledge includes beliefs about memory and learning, such as knowing that a long retention interval can demand more practice. Monitoring includes moment-to-moment JOLs and evaluations of performance. Control includes decisions about what to practice and when to stop. Someone can understand good study principles but make inaccurate item-level predictions; someone else can monitor accurately yet choose an inefficient study strategy. Maintaining these distinctions protects the construct from being stretched into a generic label for every learning behavior.

What Research Can and Cannot Tell Us

The delayed-JOL literature has unusually strong cumulative experimental evidence, including meta-analytic support across thousands of participants. Its strongest findings emerge from controlled learning paradigms where cues and outcomes are clearly defined. These studies are valuable for isolating mechanisms and measuring accuracy; extending results to large lectures, heterogeneous curricula, children, older adults, clinical populations, online platforms and long-term professional learning requires additional designs.

Measurement choices affect apparent outcomes. Relative accuracy can rise without improved absolute calibration. Changing a rating scale or using extreme responses can influence correlation-based indices. Test format, cue overlap, prior knowledge and intervening retrieval all change what the judgment means. Moreover, a JOL may modify memory through reactivity, violating a simple assumption that monitoring is entirely observational. The 2025 synthesis supports a more nuanced interpretation of that last problem.

Theories continue to debate how much immediate experience, explicit beliefs, episodic retrieval and test-cue matching contribute to specific judgments. Evidence that delayed predictions work better under some conditions supports several partially compatible mechanisms; it does not identify one exclusive neural algorithm for predicting memory. Neuroimaging and broader metacognition theories may enrich future work, but a behavioral JOL score by itself does not locate memory in the brain or establish a general level of consciousness.

Everyday Significance and Boundaries

Judgments of learning help explain a familiar experience: someone finishes studying and feels ready, then struggles to reproduce the material without looking. The feeling of readiness is psychologically real, but its predictive quality must be established against the intended performance. The same distinction matters in studying, language learning, workplace training and continuing education. The relevant intervention is to create opportunities for retrieval and feedback, rather than to maximize subjective certainty.

Educational use should remain proportional to the evidence. A delayed recall-based judgment is a useful monitoring technique; it is not a universal assessment of intelligence, a diagnostic indicator of a memory disorder or proof that one study plan will suit every learner. Cognitive symptoms, persistent memory changes and neurological or mental-health concerns require appropriate professional evaluation. Laboratory calibration findings should not be used to infer disease from a person's everyday confidence.

Frequently Asked Questions

What does JOL stand for in psychology?

JOL stands for judgment of learning, a metacognitive prediction of how likely studied material is to be remembered on a later test. Researchers usually compare the prediction with subsequent recall or recognition to assess accuracy.

Why are judgments of learning sometimes inaccurate?

Learners infer future recall from cues available now. Familiarity, ease of reading, a visible answer and personal beliefs about study can produce convincing confidence even when they only weakly predict later retrieval. The cue and the final test may demand different processes.

What is the delayed-JOL effect?

It is the common finding that predictions made shortly after study, particularly from a cue presented without the answer, rank subsequently remembered versus forgotten items more accurately than immediate predictions. The effect is well supported in controlled memory tasks, with important limits related to cue format and test conditions.

Does waiting longer automatically improve every prediction?

No. Delay can change which memory information is accessible, and its value depends on the material, available cues, task and retention interval. Simply waiting while continuing to view the answer is not equivalent to judging learning after an independent retrieval attempt.

Is a 90% judgment the same as a 90% chance of recall?

It is a stated prediction of that probability. Whether it corresponds to a 90% observed success rate is an empirical calibration question. Repeated predictions and later tests are needed to evaluate the correspondence.

Can judgments of learning make memory better?

Sometimes. Asking for JOLs can itself affect later performance. A 2025 meta-analysis found a small positive average effect of immediate judgments, substantial variation across tasks, and some negative effects with unrelated word pairs. The benefit should not be assumed for all materials or learners.

Are JOLs the same as metamemory?

JOLs are one kind of metamemory monitoring. Metamemory also includes general knowledge and beliefs about memory, feelings of knowing, confidence about retrieval and the regulation of study and remembering.

How can I use JOLs when studying a chapter?

After reading, close the text and return later to its key questions or headings. Attempt a genuine explanation, predict later performance and test yourself again after an appropriate interval. Because chapters demand comprehension and transfer, assess those outcomes directly rather than relying only on recall of isolated terms.

Do low judgments of learning indicate a disorder?

No. A JOL is a task-specific research or study judgment influenced by task difficulty, cue format, expectations and learning history. It has no stand-alone diagnostic meaning and should not be interpreted as a clinical screening score.

References

Dunlosky, J., & Nelson, T. O. (1992). Importance of the kind of cue for judgments of learning (JOL) and the delayed-JOL effect. Memory & Cognition, 20, 374–380. https://doi.org/10.3758/BF03210921

Dunlosky, J., & Nelson, T. O. (1997). Similarity between the cue for judgments of learning (JOL) and the cue for test is not the primary determinant of JOL accuracy. Journal of Memory and Language, 36, 34–49. https://doi.org/10.1006/jmla.1996.2476

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students' learning with effective learning techniques. Psychological Science in the Public Interest, 14, 4–58. https://doi.org/10.1177/1529100612453266

Finn, B. (2008). Framing effects on metacognitive monitoring and control. Memory & Cognition, 36, 813–821. https://doi.org/10.3758/MC.36.4.813

Ingendahl, F., Halamish, V., & Undorf, M. (2025). Do immediate judgments of learning alter memory performance? A meta-analytical review. Psychological Bulletin, 151, 892–929. https://doi.org/10.1037/bul0000487

Kelemen, W. L., & Weaver, C. A. (1997). Enhanced metamemory at delays: Why do judgments of learning improve over time? Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, 1394–1409. https://doi.org/10.1037/0278-7393.23.6.1394

Koriat, A. (1997). Monitoring one's own knowledge during study: A cue-utilization approach to judgments of learning. Journal of Experimental Psychology: General, 126, 349–370. https://doi.org/10.1037/0096-3445.126.4.349

Koriat, A., & Bjork, R. A. (2005). Illusions of competence in monitoring one's knowledge during study. Journal of Experimental Psychology: Learning, Memory, and Cognition, 31, 187–194. https://doi.org/10.1037/0278-7393.31.2.187

Kornell, N., & Bjork, R. A. (2008). Learning concepts and categories: Is spacing the enemy of induction? Psychological Science, 19, 585–592. https://doi.org/10.1111/j.1467-9280.2008.02127.x

Nelson, T. O., & Dunlosky, J. (1991). When people's judgments of learning (JOLs) are extremely accurate at predicting subsequent recall: The delayed-JOL effect. Psychological Science, 2, 267–270. https://doi.org/10.1111/j.1467-9280.1991.tb00147.x

Rhodes, M. G., & Tauber, S. K. (2011). The influence of delaying judgments of learning on metacognitive accuracy: A meta-analytic review. Psychological Bulletin, 137, 131–148. https://doi.org/10.1037/a0021705

Thiede, K. W., Anderson, M. C. M., & Therriault, D. (2003). Accuracy of metacognitive monitoring affects learning of texts. Journal of Educational Psychology, 95, 66–73. https://doi.org/10.1037/0022-0663.95.1.66

Weaver, C. A., & Kelemen, W. L. (1997). Judgments of learning at delays: Shifts in response patterns or increased metamemory accuracy? Psychological Science, 8, 318–321. https://doi.org/10.1111/j.1467-9280.1997.tb00445.x

Yang, C., et al. (2021). How to assess the contributions of processing fluency and beliefs to the formation of judgments of learning: Methods and pitfalls. Metacognition and Learning. https://doi.org/10.1007/s11409-020-09254-4

Psychological Encyclopedia