Judge Kopf, Criminal Defense Is No Country For Metrics
Mar. 8, 2016 (Mimesis Law) — Yesterday, Judge Kopf asked the question: is the practice of criminal defense no country for old men? He read a law review article called Criminal Defense Lawyer Moneyball: A Demonstration Project which claims there is a negative correlation between a lawyer’s experience and client results. In other words, it suggests the older a lawyer gets, the crappier a job he does. If that’s the case, yeah, let’s get rid of these old guys before they screw everything up.
Of course, it’s not the case. Delving into the application of statistics to criminal defense lawyer performance really only proves one thing. Statistics are no way to measure a criminal defense lawyer.
The Moneyball article is interesting. Two law professors decided there was some value to rating criminal defense attorneys through statistical performance. In an experiment only an academic could love, the professors came up with a way to determine which attorneys were doing a good job.
If you really want to get into the weeds with some scientific formulas, read the article. But here is the lowdown. The professors used North Carolina for their sample, which has a sentencing guidelines system. They weighted cases based on the charge at the time of the arraignment and the defendant’s criminal history category. Whoever got the biggest sentence reductions got the higher scores as a criminal defense attorney.
Great! We can plug in what every criminal defense lawyer does and see which one sucks, right? Assign each lawyer a number score based on performance and no one ever has to worry about hiring a crappy lawyer again.
The experiment revealed another surprise that triggered Judge Kopf’s post. Once a lawyer passes 8 years in practice, performance falls.
Our most surprising discovery is that experience actually has a negative correlation with performance after the first eight years: the more time an attorney has spent in the profession, the more likely that her clients will obtain a more severe sentence.
Sounds like experience is bad for a criminal defense lawyers. The professors kindly added a few potential reasons, which Judge Kopf noted in his post:
[A]ttorneys might devote less energy to their cases after many years on the job and may invest less time in learning about any changes in criminal law and practice—particularly for complex changes in the law, such as the arrival of sentencing guidelines. Similarly, more experienced attorneys may over time come to accept local norms about acceptable outcomes and stop pressing so hard against that accepted courthouse culture.
Before all you old guys start packing up your offices and you young guys start signing up for capital murder cases, it might be worth taking a look at why this whole idea of statistically rating criminal defense lawyers is a bunch of nonsense.
Criminal defense is as personal as it gets. What’s happening doesn’t lend itself to a slick academic formula or a fancy business metric. The government is pointing its finger at one of its citizens and accusing the citizen of violating the law. It wants all their money, or their liberty, or their life.
The Moneyball study went off the rails immediately by failing to recognize the differences in cases. As a base level, it used the charged crime and the criminal history category. Neither of which tells them a thing about what the person has done and what the prior criminal record really looks like.
Take an armed robbery, for example. One guy is a drug addict who sticks his finger in his pocket like it’s a gun and tries to rob the convenience store so he won’t die from heroin withdrawal. The other guy takes a machine gun into a bank with a crew and executes a well-planned heist. Same crime. In fact, the bank robber may have less of a criminal record than the drug addict.
How do you compare the outcome of cases like that? How do you determine if a lawyer did a good job? How can you tell if the sentence each defendant got tells us anything about the lawyer’s skill? The answer is easy. You can’t.
Every single case that walks into a criminal defense lawyer’s office is different. Each client is different. Some are smart. Some are dumb. Some are mean. Some are pitiful. Some are desperate. Some are sociopathic.
The crimes they are accused of are just as unique. Some are guilty. Some are innocent. Some are guilty of something, but not what they are accused of. Some are guilty, but had a good reason for doing it.
This combination of unique clients accused of unique crimes is why there will never be a metric to measure the performance of a criminal defense lawyer. It’s a job with a strange measure of what is victory.
A person can get sentenced to life without parole in prison. Sounds like a loss. Unless they were facing execution and weren’t ready to die. Then it’s a win. Maybe they walk out of court free, with nothing more than a little probation. Sounds like a win. Unless that felony conviction just ruined their future. Then it’s a loss.
The same muddled measure of wins and losses casts doubt on the idea that more experienced lawyers don’t perform as well. In fact, the answer is so clear it’s strange two law professors didn’t notice it.
Remember all those different cases and clients? Well, sometimes a really bad guy is really screwed. Or a not so bad guy who just hasn’t been able to stay out of trouble. Who do you think that guy hires? The rookie out of law school? Or maybe the more experienced lawyer?
That’s another part of lawyering that can’t be measured. The better you are, the tougher the case that hires you. So as you gain experience, and become a better lawyer, you get worse results. No wonder criminal defense lawyers are going crazy.
At the end of the day, the older, more experienced lawyer is not only who clients usually need to hire, but who younger lawyers should look up to.
A few years ago I spent 8 weeks in a federal courtroom trying a racketeering case with a lawyer named Herb Louthian. He is in his 80s and has been practicing law for over 50 years. Herb’s lawyering was brilliant. His experience had taught him when to be quiet and when to make a point. The result? His client walked away from the case a free man after the judge granted a verdict of acquittal.
So is there room for old men in the practice of criminal defense law? Yes. There is just no room for statistical performance metrics. Because this job isn’t measured in control groups and variable groups. Its measured in real people and real cases. The more experience you have, the more that makes sense. Whether that goes for judges, Judge Kopf, I can’t say.