COMPAS Risk Scores: What They Are, Evidence, and Controversy
Author: Ukrainian Psychological Hub · Published: October 11, 2026 · Editorial Policy
Ukrainian Psychological Hub · October 11, 2026 · Forensic & Legal Psychology · Evidence review
COMPAS risk scores are numerical classifications produced by a commercial risk-and-needs assessment system used in parts of the United States criminal justice system. They summarize how characteristics recorded about a person compare with patterns observed in reference populations, particularly in relation to subsequent justice-system contact. They do not establish guilt, diagnose a mental disorder, determine legal responsibility, or establish what any particular person will do.
The controversy surrounding COMPAS is a precisely documented question about what a score means, how well it travels between populations, how error burdens are distributed, and who gets to decide which kinds of error are acceptable when liberty is at stake. COMPAS illustrates why a prediction that has measurable group-level association with future recorded outcomes still requires independent validation, procedural safeguards, and an explicit legal purpose.
What is COMPAS?
COMPAS stands for Correctional Offender Management Profiling for Alternative Sanctions. Developed by Northpointe and subsequently associated with Equivant, it is a family of tools combining structured questions, criminal-history information, risk scales, and assessments of possible intervention needs. The vendor's Equivant's 2017 COMPAS Core practitioner's guide describes different use cases in pretrial, jail, probation, prison, parole, and community corrections settings. The core instrument has been revised, configured, and applied in different ways; findings about one configuration should not automatically be projected onto every version or jurisdiction.
A COMPAS report may include general recidivism risk, violent recidivism risk, a pretrial-specific measure, and separate needs scales. General recidivism concerns recorded subsequent offending or criminal justice events as defined in a validation study. Violent recidivism uses a narrower outcome definition. Needs scales address areas potentially relevant to support and case management. These are different variables with different evidentiary demands. A judge, probation officer, or evaluator must first identify which specific scale appears in a record.
How a COMPAS score is calculated and interpreted
The instrument combines information about prior justice-system contact, aspects of social and personal history, and questionnaire responses, among other factors. The proprietary scoring weights of the commercial assessment have not generally been open to full independent scrutiny. That distinction matters: knowing example inputs or the existence of a questionnaire does not reveal the exact model, scale construction, version, weighting, norm sample, or local implementation.
Several prominent COMPAS risk scales present decile scores from 1 to 10. The Equivant's 2017 COMPAS Core practitioner's guide explains that a decile is the position of a scale score relative to a normative reference distribution: a score of 1 is at the lower end of that distribution, while 10 is at the upper end. In the common classification described by the guide, 1–4 is labeled low, 5–7 medium, and 8–10 high. The manual also cautions that category cut points can differ by scale type.
A score of 8 does not mean an 80% chance of committing a crime. A decile is an ordinal ranking, and translating a rank into an absolute event probability requires outcome-specific empirical calibration with a specified population, time horizon, and measurement procedure. Two people with the same score can have substantially different circumstances, and two populations with identical rank ordering can have different observed arrest or conviction rates.
The name 'risk score' can obscure at least four distinct operations: observing and recording characteristics, aggregating them into a scale, comparing a scale with reference data, and using the output in a legal or correctional decision. Each operation introduces possible measurement error and different responsibilities. A person's answers may be incomplete, criminal-history fields may contain errors, reference populations may differ from the person assessed, and institutional decisions may place more weight on the report than the tool was designed to carry.
What is COMPAS trying to predict?
Risk assessment must identify a target outcome before anyone can evaluate predictive performance. Depending on the instrument and study, an outcome may be any new arrest, a new criminal charge, a new conviction, violent rearrest, a technical supervision violation, or a specified event over one or two years. A recorded rearrest is not interchangeable with a proven offense. Arrest data depend partly on policing, detection, enforcement priorities, and opportunities for surveillance.
Many influential COMPAS analyses concern a two-year follow-up of people arrested in Broward County, Florida, in 2013–2014, rather than an experiment in which an algorithm observed actual future offending directly. In the original 2016 Broward County data analysis, researchers combined assessment records, criminal records, and custody periods. Their outcome definitions and data-linkage decisions were essential components of the result; they were not incidental administrative details.
The relevant time period also matters. A person incarcerated during much of the follow-up has different exposure to opportunities for community rearrest from someone continuously at liberty. Valid research should make explicit how it handles time at risk, censoring, loss to follow-up, and the distinction between predicting events under an existing justice-system policy and predicting what would happen under a new one.
Does COMPAS predict recidivism accurately?
The evidence indicates that COMPAS can discriminate between groups with different observed rates of future recorded outcomes, while the strength of prediction varies with the scale, sample, outcome, threshold, and statistical measure. That is a narrower and more useful conclusion than either 'COMPAS works' or 'COMPAS is always wrong.' It is also necessary to distinguish research specifically on COMPAS from syntheses of many different instruments.
The 2016 Broward analysis identified substantial group-level associations between COMPAS categories and subsequent arrests. But it also documented errors and racial differences in the distribution of errors. Its finding that roughly 61% of individuals labeled higher risk experienced a subsequent recorded event under one reported analysis is a statement about the positive predictive value for that definition and threshold, not a universal claim of 61% overall accuracy. The analysis reported a particularly low positive predictive value for the high-risk violent outcome; event rarity and the chosen classification rule are crucial to interpretation. See the original 2016 Broward County data analysis.
In Dressel and Farid (2018), researchers used publicly available COMPAS-associated data and compared its predictions with judgments from people given limited case descriptions. The study found that human respondents and a simple model using a small number of features could perform comparably under the examined conditions. That finding undermines an assumption that proprietary complexity necessarily supplies superior prediction. It does not show that an unaided layperson is an equally suitable decision-maker in every legal context or that every specialized forensic instrument is interchangeable.
A broader systematic review, Fazel et al.'s systematic review (2022), examined validation studies of 11 risk assessment instruments used at sentencing. Across 36 studies and 597,665 participants, predictive performance ranged from poor to moderate; in larger independent validations, reported AUC values generally ranged from 0.57 to 0.75. Most studies lacked important information about calibration and classification errors. This is evidence about the assessment field, not a single pooled accuracy estimate for COMPAS.
What do AUC, calibration, sensitivity, and false positives mean?
Discrimination describes how well a score ranks people who later have a recorded outcome above people who do not. The area under the receiver operating characteristic curve (AUC) summarizes this ranking ability across thresholds. An AUC of 0.50 corresponds to chance discrimination in the usual binary setting; 1.00 indicates perfect ranking in the evaluation sample. AUC alone tells us neither the probability attached to a particular score nor the legal consequences of threshold-based decisions.
Calibration asks whether predicted probabilities agree with observed event frequencies. If a tool explicitly estimates 20% risk for a well-defined cohort over two years, a calibration check compares that number with the cohort's observed rate under the specified follow-up conditions. An ordinal COMPAS decile, however, must not be treated as a probability without a validated conversion. Calibration should also be examined in relevant subgroups and over time.
Sensitivity is the proportion of people with a recorded future event who are classified above a chosen threshold. Specificity is the proportion without the event correctly classified below that threshold. A false positive is someone classified high risk who does not experience the study-defined event during follow-up; a false negative is someone classified low risk who does. Positive predictive value measures the share of high-risk classifications followed by an event. Negative predictive value measures the share of low-risk classifications without one.
Threshold selection changes the balance: a lower cutoff usually identifies more later events while also classifying more people as high risk who will not have the recorded event. The consequences of both errors must be articulated rather than hidden in the model. False positives can contribute to restrictive supervision or detention; false negatives can matter for public safety or missed supportive intervention. Neither type of error proves a moral quality of the person assigned the label.
Reported predictive performance also needs uncertainty intervals, adequate independent sample sizes, subgroup event counts, base-rate estimates, and information about missing records. A score model evaluated after an outcome has already been influenced by an earlier score cannot be interpreted as though assignment and subsequent treatment were unrelated.
Why COMPAS became the center of a racial fairness controversy
A 2016 investigation using Broward County assessment and public-record data reported that among Black defendants who did not have the study-defined subsequent event, approximately 45% were classified at higher risk, compared with approximately 23% of White defendants without the event. Among defendants who did have a subsequent event, approximately 28% of Black defendants and 48% of White defendants had been classified lower risk. These are conditional error rates in a particular historical dataset, measured for a particular outcome, not immutable rates for every version of COMPAS.
That analysis brought attention to false-positive-rate and false-negative-rate disparities. The vendor disputed important interpretations of the data, emphasizing other measures of predictive performance and the relevance of different base rates. The published historical record includes both the original 2016 Broward County data analysis and ProPublica's technical exchange with Northpointe. A defensible evidence review examines the actual denominators and definitions before deciding whether a specific claim follows.
Race was not identified by the vendor as a direct input to the relevant predictive model. The absence of an explicit race field does not guarantee the absence of racial disparities: criminal records, neighborhood conditions, access to stable employment, and other measured variables can correlate with structural inequalities or enforcement differences. Conversely, observing a statistical disparity does not, by itself, identify which input or institutional process caused that disparity.
Can a model be calibrated across racial groups and still have unequal error rates?
Yes. Predictive parity, sometimes framed as similar positive predictive value across groups, is different from equality of false-positive and false-negative rates. A system can have roughly comparable observed event proportions among people placed in the same score category while producing unequal burdens of erroneous high-risk or low-risk classifications.
In Chouldechova (2017), the mathematical analysis explains why certain fairness criteria become incompatible when the prevalence of the measured outcome differs between groups. Kleinberg, Mullainathan, and Raghavan (2016/2017) formalized related trade-offs, and Berk et al. (2021) reviewed different meanings of fairness in criminal justice risk assessment. The impossibility results concern specified mathematical properties under stated assumptions; they do not prove that all practical fairness improvements are impossible.
Different stakeholders may reasonably prioritize different harms: a court may focus on restrictions imposed on individuals who would not have the specified recorded outcome, an agency may prioritize sensitivity to particular safety events, and researchers may focus on well-calibrated estimates. Choosing a fairness objective is partly a normative and legal decision about acceptable burdens. A mathematical model cannot choose that public value without human governance.
Changes to the threshold, training data, outcome definition, population, or measurement methods can change both apparent accuracy and fairness. Modern work, including a 2025 fairness-focused study, evaluates mitigation techniques on historical datasets, but technical improvement on a benchmark is not proof of improved real-world justice outcomes after deployment.
Are COMPAS risk scores racially biased?
The accurate answer identifies the metric, group, population, version, and consequence. The historical Broward analysis found substantial disparities in conditional classification errors by race. Other authors focused on calibration or comparable predictive value and argued that those results supported a different sense of fairness. Both kinds of statements can be empirically meaningful because they answer different questions. Whether a deployment is unlawfully discriminatory depends on the applicable law and facts, not a single statistical metric.
A contemporary audit should report discrimination, calibration, false-positive and false-negative rates, base rates, subgroup uncertainty, and the policies governing thresholds and human override. It should assess performance for gender, age, relevant disability or language groups, and intersectional subgroups when data and sample size permit. Findings from Broward County should not be presented as current nationwide validation, and missing subgroup evidence should remain explicitly unknown.
The difference between arrest, offending, guilt, and a risk score
An arrest is an event in the justice system; it is neither proof of guilt nor necessarily a complete measure of criminal conduct. Recidivism is defined differently across studies and agencies. A risk factor is a statistically associated characteristic, not a demonstration that it caused a particular crime. A score combines selected characteristics for a predictive purpose. A forensic opinion integrates relevant evidence within a specified professional role. A legal finding is made under rules of evidence and an applicable legal standard.
COMPAS does not establish that an accused person committed the charged offense. It cannot by itself determine competency to stand trial, criminal responsibility, the validity of an insanity defense, a clinical diagnosis, or an individual's moral culpability. The existence of an assessment report cannot replace the presumption of innocence, adversarial fact-finding, or individualized legal reasoning.
The term 'algorithm' should also be used precisely. A proprietary statistical assessment is not necessarily a generative artificial intelligence system. Its historical use is mainly structured predictive scoring. Comparing it with contemporary machine-learning fairness methods is valuable for evaluation, but it does not make every COMPAS deployment an instance of the latest AI technology.
State v. Loomis: what the Wisconsin Supreme Court actually decided
In the Wisconsin Supreme Court opinion in State v. Loomis (2016), the Wisconsin Supreme Court considered a sentencing challenge in which a presentence investigation included COMPAS scores. The court held that consideration of a COMPAS assessment, when properly limited and contextualized, did not in that case violate due process. This ruling was a legal judgment about a particular use of the information, not scientific certification of COMPAS's accuracy or fairness.
The opinion expressly constrained permissible use. COMPAS scores should not determine whether a person is incarcerated or the severity of the sentence, and should not be the determinative factor in deciding whether community supervision is viable. The court required written advisements with presentence reports concerning limitations including proprietary weighting, group-based estimates, questions about disparities, the need for local validation, and ongoing monitoring. It distinguished supervision and intervention planning from determining deserved punishment.
This is a Wisconsin ruling applying that jurisdiction's legal framework and record. Other states and countries can have different statutes, evidentiary rules, due-process doctrines, sentencing frameworks, and permissible uses of predictive instruments. A practitioner should check the law currently applicable to the specific court, the particular tool version, and the decision for which the score is offered.
How COMPAS could influence a judge or correctional decision
A score can influence decisions directly when it is expressly considered, or indirectly when it shapes a presentence report, a supervision recommendation, a case-management allocation, or the options presented to a judge. Automation bias can arise when an apparently precise number is given more epistemic weight than the underlying data justify. An opposite risk is inconsistent human use: one decision-maker discounts a score while another treats the same category as decisive.
The outcome of concern must be matched to the decision. A risk assessment validated for correctional supervision is not automatically validated for bail, sentence length, incarceration, probation eligibility, parole, or a particular intervention. Each decision has a distinct legal purpose, harm distribution, and required evidentiary threshold.
The observational and modeling work of Engels, Linhardt, and Schubert's 2024 study (published in the 2025 volume; corrected in 2026) draws attention to the policy assumptions embedded in cutoff decisions and the competing consequences of over- and under-classification. Its analysis uses historical data and modeling assumptions; it does not establish how all judges actually used COMPAS, or show that any proposed correction has produced better liberty or safety outcomes after real-world implementation.
Transparency, proprietary scoring, and the right to contest
Meaningful scrutiny requires more than knowing that a risk score is produced. Relevant documentation includes the version, target variable, collection dates, input provenance, missing-data rules, norm group, weighting or an adequate description of model behavior, subgroup validity, and prior independent evaluations. The ability to correct an inaccurate record can matter as much as a technical explanation of the calculation.
A proprietary system can make independent reproducibility, disclosure, and adversarial examination difficult. A defendant may challenge the accuracy of recorded facts even when full model internals are not available, but the legal entitlement to a particular kind of model disclosure varies by jurisdiction and procedural context. Courts, agencies, and vendors should specify accessible mechanisms for correction, contestation, and documented review.
The NIST's AI Risk Management Framework emphasizes validity, generalizability, documentation, transparency, and ongoing evaluation in context. It provides a general AI risk-governance framework, not a binding legal rule specific to COMPAS. A sound governance process assigns responsibility for procurement, validation, threshold selection, monitoring, and remedy when a score contributes to harm.
Population, time, and measurement limits
Transportability is a substantive empirical problem. Age composition, offense patterns, supervision intensity, policing practices, incarceration exposure, employment conditions, local services, and statutory definitions differ across jurisdictions. Even a score performing adequately in one state and period may have different calibration in another. Evaluation requires contemporary local data and a sufficiently long, clearly defined follow-up.
Sex and gender, age, cultural background, language comprehension, literacy, socioeconomic position, disability, and survey administration conditions may affect the data collected or the interpretation of a response. A validity study that reports only an overall AUC can conceal subgroup uncertainty. Particular caution is required when a measure designed for adult correctional populations is proposed for juveniles or other populations without appropriate validation.
Historical rearrest labels may reflect differential surveillance. A policy that increases supervision for people with high scores can increase detection of some technical violations, which in turn can affect future labels. These feedback mechanisms cannot be understood fully through a static test set alone. Ideally, evaluation combines predictive validation with analysis of the decisions actually changed and the downstream benefits and harms.
Could a simpler and more transparent model perform as well?
Sometimes. Dressel and Farid (2018) compared COMPAS-related predictions with straightforward alternatives and found that complexity did not ensure superior accuracy in that particular task. Simplicity can improve inspectability and correction of data, but it is not a guarantee of fairness or calibration. Transparent models can also reproduce biased targets and proxy relationships if their data and goals remain unchanged.
Model selection should compare candidate tools under the same population, outcome, follow-up, and prospective decision process. The relevant choice includes predictive quality, calibration, interpretability, operational burden, independent reproducibility, and decision impact. A model should improve a defined decision process compared with a credible baseline, rather than merely achieving an attractive score against another algorithm.
How to critically evaluate a COMPAS report
A careful reviewer starts with the exact product, version, scale, date, and administering agency. The next questions are which event was predicted; over what period; on which reference sample; whether the result is an ordinal decile, a category, or a validated probability; and which variables and records were supplied. A high score is incomplete information without those answers.
The reviewer then checks independent validation for the specific population and use, including AUC, calibration where meaningful, error rates at the operational threshold, uncertainty, subgroup results, and changes in event prevalence. They examine errors in criminal-history data, missing answers, detention time, surveillance, and whether the same information is already counted elsewhere in a sentencing report.
Finally, the legal decision should be independently justified. Documentation should identify how the score was presented, how the decision-maker interpreted it, what non-algorithmic factors were considered, which objections were raised, and what procedural route exists to correct disputed facts. This structure allows an assessment to contribute appropriately without silently becoming a legal verdict.
Common misconceptions about COMPAS
Myth: a score of 9 means a 90% chance of reoffending. Correction: the common 1–10 COMPAS presentation is a normative decile ranking; the associated event rate must be established separately for the relevant outcome and population.
Myth: the algorithm directly detects criminality or dangerousness. Correction: it predicts an operationally defined future recorded outcome from historical variables. Neither criminal guilt nor inevitable behavior can be read from the classification.
Myth: excluding race from an input list eliminates racial disparities. Correction: correlated variables, target definitions, dataset construction, and institutional practice can still produce different error burdens.
Myth: matching predictive values across demographic groups settles the fairness question. Correction: calibration and error-rate equality are distinct criteria and may conflict under different outcome base rates.
Myth: a high AUC establishes the fairness of using a score at sentencing. Correction: the legal purpose, calibration, threshold, human interpretation, procedural protections, and actual consequences remain separate questions.
Myth: the Wisconsin Supreme Court declared COMPAS scientifically valid. Correction: the court permitted a limited sentencing use in State v. Loomis while imposing warnings and restrictions, and did not certify the underlying science.
Frequently asked questions
What does COMPAS stand for? Correctional Offender Management Profiling for Alternative Sanctions. It names a family of structured correctional risk-and-needs assessment tools.
Is COMPAS still used in the United States? COMPAS has been used in multiple US justice settings, but deployment and versions vary by agency and over time. The historical research cited here does not establish an up-to-date national inventory. A present-day use claim requires confirmation from the responsible jurisdiction.
What is a high COMPAS score? On common normative decile scales, 8–10 is classified high relative to the reference group. Cutoffs are scale-dependent, and the number is not an individual percentage probability.
Is COMPAS an AI system? It is usually described as a proprietary algorithmic or actuarial risk assessment. Its design and use precede current generative-AI systems; whether it falls within a legal definition of AI depends on that definition.
How accurate is COMPAS? Accuracy varies by endpoint, threshold, sample, follow-up period, and measure. Published analyses show predictive signal and significant errors. There is no scientifically adequate single accuracy percentage for every COMPAS score and application.
Why did ProPublica and Northpointe disagree? They prioritized different statistical properties and interpretations of the same general problem, including error-rate balance, predictive performance, and outcome base rates. Researchers later formalized why several fairness criteria can conflict.
Does COMPAS use race? The vendor maintained that race was not a direct predictive input for the model under examination. This does not resolve whether other variables, outcomes, or deployment practices produce racial disparities.
Can a score prove that someone will commit a violent crime? No. Even a well-validated risk tool provides probabilistic group-level information tied to defined future events. Individual action is uncertain, and a score is never proof of a future act.
Can a judge use COMPAS to determine a prison sentence? Legal rules vary. The Wisconsin Supreme Court allowed limited consideration while rejecting the score as a determinant of incarceration or sentence severity in State v. Loomis. Other jurisdictions require separate legal analysis.
Can a defendant challenge a COMPAS assessment? Disputed facts, methods, and uses can raise procedural and evidentiary questions, but the available remedies and rights to disclosure depend on the jurisdiction and stage of proceedings. A qualified local attorney should examine the specific record.
Does a mental health diagnosis determine a COMPAS score? No diagnosis establishes guilt, inevitable offending, or future violence. Forensic assessment of mental state or legal capacity requires separate instruments and legal questions; risk scoring cannot substitute for clinical evaluation.
Conclusion
COMPAS is a structured probabilistic instrument with measurable predictive associations and consequential limitations. The research establishes three enduring requirements: specify the event being predicted, evaluate performance and error burdens in the population actually affected, and keep the legal decision accountable to applicable standards and individualized evidence. What matters is not merely whether a model can rank historical records, but whether its limited information is used fairly, transparently, and for a decision it is fit to inform.
Related articles
Violence Risk Assessment: What It Can Predict and Where It Fails · Schizophrenia and Violence: What Research Really Shows
Related Articles
References
Berk, R., Heidari, H., Jabbari, S., Kearns, M., & Roth, A. (2021). Fairness in criminal justice risk assessments: The state of the art. Sociological Methods & Research, 50(1), 3–44. https://doi.org/10.1177/0049124118782533
Chouldechova, A. (2017). Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big Data, 5(2), 153–163. https://doi.org/10.1089/big.2016.0047
Dressel, J., & Farid, H. (2018). The accuracy, fairness, and limits of predicting recidivism. Science Advances, 4(1), eaao5580. https://doi.org/10.1126/sciadv.aao5580
Engels, C., Linhardt, L., & Schubert, M. (2024; 2025 volume, subsequent corrections). Code is law: How COMPAS affects the way the judiciary handles the risk of recidivism. Artificial Intelligence and Law. https://doi.org/10.1007/s10506-024-09389-8
Equivant. (2017). Practitioner's guide to COMPAS Core. Source document (PDF)
Farayola, M. M., Tal, I., Saber, T., Bendechache, M., & Connolly, R. (2025; 2026 volume). A fairness-focused approach to recidivism prediction: Implications for accuracy, trust, and equity. AI & Society, 41, 2783–2801. https://doi.org/10.1007/s00146-025-02452-1
Fazel, S., Burghart, M., Fanshawe, T., Gil, S. D., Monahan, J., & Yu, R. (2022). The predictive performance of criminal risk assessment tools used at sentencing: Systematic review of validation studies. Journal of Criminal Justice, 81, 101902. https://doi.org/10.1016/j.jcrimjus.2022.101902
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016/2017). Inherent trade-offs in the fair determination of risk scores. https://arxiv.org/abs/1609.05807
Larson, J., Mattu, S., Kirchner, L., & Angwin, J. (2016). How we analyzed the COMPAS recidivism algorithm. ProPublica. Original data-analysis methodology and historical evidence. Source methodology
National Institute of Standards and Technology. (2023). Artificial Intelligence Risk Management Framework, AI RMF 1.0. Official framework
Scaria, A. G., Subramanian, V., George, N. K., & Sengupta, N. (2024). Algorithms and recidivism: A multi-disciplinary systematic review. Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, 7(1), 1292–1305. https://doi.org/10.1609/aies.v7i1.31724
State v. Loomis, 2016 WI 68, 371 Wis. 2d 235, 881 N.W.2d 749 (Wisconsin Supreme Court, July 13, 2016). Court opinion