Mimesis Law
7 August 2020

Sentencing Empiricism Beyond the Wonkfest

August 9, 2016 (Fault Lines) — After Iowa Senior District Judge Mark Bennett asked the question, Are Federal Judges Addicted to Incarceration?,” there was a non-zero chance that Nebraska Senior District Judge Richard Kopf would respond.  Judge Kopf has long been interested in, and has become quite knowledgeable about, the use of empirical tools in sentencing.  And with some reason:

It is true, as Mark has said, that he is an empiricist (and, I should add, a humanist). He is also profile writer about empirical questions and data derived from those empirical questions. (For example, he could explain to you in his sleep what an “R2” means!) So, it is with some trepidation that I respond to his post entitled, Are Federal Judges Addicted to Incarceration?

The answer to Mark’s titular question is, undoubtedly, “yes.” There are many reasons for this addiction. Largely, however, that addiction is due to the psychological impact of “anchoring” that occurs when using the Sentencing Guidelines, as the Guidelines call for prison sentences in the great majority of federal criminal cases. Indeed, Mark’s own scholarship on “anchoring” makes that plain. In short, old habits die hard, particularly when a judge has no pre-Guidelines experience.

Essentially, Judge Bennett explained the addiction to incarceration as a byproduct of reliance on the now-not-too-mandatory federal Sentencing Guidelines which, he explains, were largely made up numbers reflecting the politics of crime at the time, and have since been taken as gospel by the judiciary as if they represented the proper length of sentences for magic reasons.

And Judge Kopf, who exerted Herculean efforts to become an empirical umpire, took a different direction, without quite agreeing that the Guidelines were made of whole cloth, that empiricism could serve to take the voodoo out of sentencing.

Rather, I want to be practical and get down to the here and now. That is, I want to specifically address Mark’s reference to my Federal Sentencing Reporter article where I suggested, among other things, that we ought to use an empirically validated risk assessment tool at sentencing similar to the one known as PCRA (Post Conviction Risk Assessment).  PCRA is used in most federal courts now, but only when the offender gets out of prison and begins the supervised release stage. I believe PCRA should also be used at sentencing.

Realizing that reliance on the PCRA was likely to rile up defense lawyers, who inherently believe in our own magic abilities to obtain the best possible sentence, he closed with a bone for the angry mutts:

I agree with Mark that many of us are addicted to long prison sentences. But that is changing. I am convinced that the use of PCRA (at least the officer section) at sentencing as exemplified by the District of North Dakota provides us an opportunity to help offenders even when we feel compelled to impose prison sentences. It is a start, and one that founded upon a solid empirical foundation.

If you read between the lines, what you will see is that the PCRA is likely to produce lower sentences of imprisonment. What criminal defense lawyer doesn’t want his client to get a lower sentence?  There’s the kicker.

So are we all on board? Well, not exactly. While statistics give me, like most lawyers, hives, that doesn’t mean others aren’t pretty good with them, enough so as to raise questions that can’t be wiped away by claims of empirical validity, like the folks at ProPublica, who called bullshit. And when taken to task by the company profiting off their empiricism for sale, called bullshit again. Who’s right? Beats me. I’m just a lawyer, but I wouldn’t take one-sided validity for granted,

Or maybe the problem isn’t just with the fact that the empirical criteria are thinly veiled proxies for race and poverty, but that they play to the confirmation bias.

We find that while several appropriate features predict sentence length, such as details of the crime committed, other features seemingly unrelated, including daily temperature, baseball game scores, and location of trial, are predictive as well.  Unrelated events were, surprisingly, more predictive than race, which did not predict sentencing length relative to the guidelines.  This is consistent with recent research on racial disparities in sentencing that highlights the role of prosecutors in making charges that influence the maximum and minimum recommended sentence.  Finally, we attribute the predictive importance of date to the 2005 U.S. Supreme Court case, United States v. Booker, after which sentence length more frequently fell near the guideline minimum and the range of minimum and maximum sentences became more extreme.

Doug Berman calls this “seemingly somewhat kooky,” and, indeed, who would even think to study baseball game scores as sentencing predictors? But when you think about it, it makes a lot of sense. A happy judge is a more merciful judge, and a judge whose team won last night is a happy judge.

But for all this very thoughtful, even wonkish, focus on empiricism, there is one player in the game of sentencing that never makes it into the mix.

If part of the justification for sending a human being to be caged for a period of years is to produce a person on the backend who will not emerge to commit crimes, harm people, have no skills or knowledge that will allow the person to assume a place in society where he can live a law-abiding life, then prisons have failed.  And if prisons have failed, then why is the focus not on the failure of the institution of prison to fulfill its legal mandate, rather than on the defendant’s unrepaired past?

Why will no one mention that the Bureau of Prisons, the governmental entity whose job it is to take these broken defendants and return them to society prepared to be law-abiding, productive citizens, is a miserable failure at its job? All the empiricism in the world doesn’t excuse the fact that incarceration is a failure when it comes to rehabilitation, and yet judges keep sending people into its clutches with the expectation they’re going to come out better for it.

What about rehabilitation, judges? When will you turn your attention away from the recidivism cues and instead ask why the place you’re sending them remains a miserable failure?

11 Comments on this post.

Leave a Reply



Comments for Fault Lines posts are closed here. You can leave comments for this post at the new site, faultlines.us

  • Richard G. Kopf
    9 August 2016 at 10:04 am - Reply


    Two things. One wonky. One not.

    Nobody in the article you cite called bullshit on PCRA. The study you cite deals with a state risk prediction instrument. In fact, the study you cite states:

    “The largest examination of racial bias in U.S. risk assessment algorithms since then is a 2016 paper by Jennifer Skeem at University of California, Berkeley and Christopher T. Lowenkamp from the Administrative Office of the U.S. Courts. They examined data about 34,000 federal offenders to test the predictive validity of the Post-Conviction Risk Assessment tool that was developed by the federal courts to help probation and parole officers determine the level of supervision required for an inmate upon release.

    The authors found that the average risk score for black offenders was higher than for white offenders, but that concluded the differences were not attributable to bias.”

    Second, you are right to suggest that the breakdown between sentencing and supervised release in the federal system is at the BOP. Why not concentrate on “rehabilitation? Do me a favor, and try to find the word “rehabilitation” in 3553(a). You might infer the word from section 3553(a)(2)(D) but even then the object is entirely utilitarian if you read the statute as a whole.

    Please don’t misunderstand me, I am all for giving prisoners tools to avoid coming back to prison. God knows we go to great efforts to do so in the supervised release context after they come back to us upon completion of a prison sentence.

    But Congress needs to tell BOP to get serious on what you call “rehabilitation.” Until then PCRA offers us an opportunity to identify risk and measures to ameliorate risk at sentencing and on supervised release. We lack the power to instruct the critical intermediary–BOP–to address those matters in a serious fashion.

    All the best.


    • shg
      9 August 2016 at 10:19 am - Reply

      Yes, the Northpointe risk assessment program wasn’t PCRA, but then, is there a reason why PCRA is valid but COMPASS isn’t? Has PCRA been subject to serious scrutiny? How long did it take before someone figured out that fingerprints, once the gold standard of empirical identification at 7 matches, turned out to be kinda malarky? Maybe PCRA is sound, even if it doesn’t include the baseball scores, but how many people will be sentenced under it over the decades until somebody realizes it’s nonsense?

      And of course 3553(a)(2)(D) is rehab.

      (D) to provide the defendant with needed educational or vocational training, medical care, or other correctional treatment in the most effective manner;

      Congress isn’t giving convicts educational and vocational training so they can be the handiest guy in the cellblock. Why let BOP off the hook? How is it BOP has no responsibility for recidivism, cutting loose convicts who are no better off, and likely much worse, than they came in? How does that fit into the PCRA paradigm? Maybe they should subtract 10 points for anybody who ends up being sentenced to a term of imprisonment?

      • Richard G. Kopf
        9 August 2016 at 12:44 pm - Reply


        1. For validation purposes, please see Construction and Validation of the Federal Post Conviction Risk Assessment, Federal Probation, Vol. 75 No. 2 (which underwent a double-blind peer review prior to publication). You can find it here: http://www.ned.uscourts.gov/attorney/judges-information/richard-g-kopf (scroll down to “Other Items of Interest” and “Federal Sentencing Reporter (2014-2015).

        2. Next, I call your attention to 18 U.S.C.§ 3582(a) which must be read in connection with section 3553(a)(2)(D), to wit:

        “(a) Factors to be considered in imposing a term of imprisonment.–The court, in determining whether to impose a term of imprisonment, and, if a term of imprisonment is to be imposed, in determining the length of the term, shall consider the factors set forth in section 3553(a) to the extent that they are applicable, RECOGNIZING THAT IMPRISONMENT IS NOT AN APPROPRIATE MEANS OF PROMOTING CORRECTION AND REHABILITATION.” (Emphasis in caps ’cause this damn comment platform only allows for minimal word processing–or maybe I’m just an idiot).

        3. Regarding the damn BOP, I apologize for being unclear. I am not letting the BOP off the hook. It is not an exaggeration that BOP makes my angry virtually every week. But I do want Congress to explicitly state what it intends so BOP will do more worthwhile things for offenders in prison to see to it that offenders do not offend again. If Congress intends rehabilitation IN PRISON then it should say so loud and clear and then appropriate the money that is required to do so. After that, it should hold the BOP’s ass to the fire.

        4. I thought I explained in my earlier post how PCRA can help at sentencing even now. Again, I apologize for my lack of clarity. Assuming that you don’t use PCRA to increase or decrease prison sentences at sentencing so as to encourage offender interviews, it is valuable for three reasons: (1) it enables the judge to make specific programming recommendations to the BOP (which, admittedly, it is free to ignore); (2) it allows the judge to better tailor specific supervised release conditions suited to the characteristics of the individual offender; and (3) it serves as a baseline for the offender as he or she starts supervision after prison–the supervising probation officer knows what the prior PCRA suggested, and, as a result, that officer (a) can better interpret the second administration of the PCRA and (b) the first administration of the PICTS (“criminal thinking” scale). (Bring to mind “continuity of care” if you like irony.)

        All the best.

        Rich Kopf

        • shg
          9 August 2016 at 1:42 pm - Reply


          Since I’m no statistician, validation studies always sound great to me. It’s other statisticians who know enough to point out the flaws. Plus, it gives me hives, and I hate hives. Has anyone disinclined to approve of PCRA gone through it and validated it? Maybe Propublica?

          As for 3553(a), Judge Kane’s recent opinion in US v. Cheever, which I discuss here, discussed in great detail the relative length of imprisonment and rehabilitative purposes of sub (D). As for the caveat, “RECOGNIZING THAT IMPRISONMENT IS NOT AN APPROPRIATE MEANS OF PROMOTING CORRECTION AND REHABILITATION,” it’s unfortunate that there is such a flagrant disconnect between the express text of sub (D) and the fact that BOP is such a massive failure that, as a matter of law, it’s not appropriate at a critical part of its job.

          Does that mean when determining a length of sentence that the BOP should be written off has serving any purpose other than warehousing bodies, and the rehabilitative goals should be sought elsewhere? Perhaps so. One might think that for all it costs to warehouse inmates, the BOP might be a very appropriate means of promoting correction and rehabilitation, but apparently, they’re better at nutraloaf and counts, and turning out convicts better at being criminals than they came in.

          • SPM
            9 August 2016 at 2:24 pm -

            Unfortunately, the only rehabilitative program that has consistently proven across all populations to be effective is what the physicians call “the tincture of time.” To quote one old warden from decades ago, “When they get so old they can’t run out of the bank after a robbery, they retire.” It is unfortunate (and dated), but true. If someone goes in at age 20, and comes out at age 90, then it is pretty clear their chance of recidivism is low. Even when it comes to things like vocational training and education, it is not clear if the benefit (if any) really comes from the programs themselves, or from the fact that the individuals are simply older. While you can cherry-pick any study you like, the data as a whole is at best equivocal.

            In some strange way, the fact that we haven’t found some magical way to change human behavior reassures me far more than our failure to do so saddens me.

  • Richard G. Kopf
    9 August 2016 at 2:52 pm - Reply


    Let me call your attention to Jennifer Skeem and Christopher T. Lowenkamp, Risk, Race, & Recidivism: Predictive Bias and Disparate Impact, http://ssrn.com/abstract=2687339 (last revised June 14, 2016). While Lowenkamp is with the Administrative Office of the US Courts (and the “father” of PCRA), Professor Skeem is the Mack Distinguished Professor and Associate Dean of Research at UC Berkley School of Social Welfare. Professor Skeem’s current work addresses a recent surge of interest in the use of risk assessment to inform criminal sentencing—including how this practice may affect racial and economic disparities in imprisonment. I doubt anyone would call her biased.

    This is what they found:

    “Based on a sample of 34,794 federal offenders, we examine
    the relationships among race, risk assessment (the Post Conviction Risk Assessment [PCRA]), and future arrest.

    . . . .

    In summary, PCRA scores are useful for assessing risk of future crime, whether an offender is Black or White.”

    Please note that the second author Lowenkamp “specifically advises against using the PCRA to inform front-end sentencing decisions or back-end decisions about release without first conducting research on its use in these contexts, given that the PCRA was not designed for those purposes.”

    To a degree, this caution reflects internal politics at the A.O. largely driven in my opinion by former Attorney General’s Holder’s attack on using these types of instruments. I am also sure it reflect Lowenkamp’s proper and genuine caution as a social scientist. That said, the manner in which PCRA is used at sentencing in North Dakota, highlighted in my earlier post as well as my comment above, ameliorates much of Lowenkamp’s concerns. That is PCRA is used not to increase or decrease a prison sentence but to develop measures to assist the offender from reoffending.

    In short, North Dakota’s “experiment,” like the one I hope Nebraska will conduct, is one way to conduct the type of research that Lowenkamp urges. Scott, I sincerely hope this helps.

    All the best.

    Rich Kopf

    • shg
      9 August 2016 at 3:43 pm - Reply

      I don’t disbelieve you, Judge. But the road to hell is paved with well-intended unbiased opinions (Berkeley School of Social Welfare? Where kale was invented and Birkenstocks have never gone out of fashion?). Since this began as a discussion of the addiction to incarceration, and you’ve sworn off rehab, I’m a little confused where PCRA fits in, other than a less volatile measure than penile plethysmography. I can’t say for sure, but I’m fairly certain that would give me hives too.

      I hope the ND experiment works out, not because I’m convinced, but because I would hate to see defendants used as lab rats in a failed experiment.

  • Christopher Lowenkamp
    11 August 2016 at 7:14 am - Reply

    If I might add to the discussion. See the following article to be published in Federal Probation in September: http://www.crj.org/cji/entry/false-positives-false-negatives-and-false-analyses-a-rejoinder.

    Also see this blog: https://www.chrisstucchio.com/blog/2016/propublica_is_lying.html and


    The reporters(not researchers) at ProPublica made some serious errors in their analysis and interpreting their findings. We sent them the paper linked above back in July and have not heard any comments back from them. I am not an advocate of the Northpointe COMPAS but do advocate for the use of risk assessment at certain points in the criminal justice system.

    The problem is not the math but the philosophy and ethics (see https://arxiv.org/pdf/1503.03666v1.pdf andhttps://arxiv.org/pdf/1406.5540.pdf).

    The PCRA, which I was involved in developing, has about 10 studies conducted completed on it. Many are peer reviewed but all are not. And there is one peer reviewed study from an external evaluator that shows the instrument validity (http://cjb.sagepub.com/content/early/2016/06/02/0093854816650481.abstract).

    This is not to say that concerns over bias are not relevant. They certainly are and that is why we ran bias tests for race, gender, age and ethnicity on the PCRA. But the analysis ProPublica conducted and reported on is misinformed and misguided by existing standards for assessing test bias.

    • shg
      11 August 2016 at 7:30 am - Reply

      Thanks for the additional info. It’s great to sit in the cheap seats and watch people far more knowledgeable argue over empirical methodologies and statistical validity, and I hope it happens long and hard before courts embrace it. While I may be clueless when it comes to statistics, my memory is great when it comes to a long and sordid history of empirical mistakes that courts were quick to accept and sent tens of thousands of people to prison based on false premises. Think everything from fingerprints to cute doggies, DNA (10 gazillion to 1 chance it’s wrong!!!) to narcotics field tests, not to mention captive crime labs finding coke without the need to test, bite mark analysis to duct tape analysis.

      This isn’t to say that any new effort is wrong, but that we have a long and sordid history of embracing claims that sound all science-y, replete with cool jargon and peer reviewed studies that turned out to be total garbage, and only later, usually decades, realize that it was all bullshit. Except by that point, not only have we sacrificed tens of thousand of innocent (or at least not quite so guilty) defendants on the altar of science, but we can’t undo the law’s acceptance of deeply embedded malarkey (there are those drug doggies, again).

      Is yours different? It may well be, but let’s make sure, really, really sure, before we applaud the system taking away people’s freedom. Just in case.