Mimesis Law
24 May 2020

Intellectual Yet Idiot Forensic Scientists

September 23, 2016 (Mimesis Law) – Before science blossomed, any idea about the universe and how it works was just as good as any other. So long as you could reason coherently, the sun could be said to circle the earth and be considered true. Things like experimentation and observation were seen as secondary to the thoughts of big thinkers. Once Aristotle explained bees, then that’s all that needed to be said about that subject. What’s good enough for granddad is good enough now.

Apart from the rationalized universe, there was the supernatural. Before science, astrogeology and fortune telling were mystical undertakings. People may have thought that they were accurate, but these fields were thought to spring from the gods, spirits, demons, or whatever supernatural force was convenient. Not at all like scientists using studies of a handful of people and inferring the results to the general population.

While technology and industry progressed successfully through trial and error, the academic explanations that followed discoveries were often wrong. And mere guesses, such as Miasma Theory, were treated as valid explanations. As before, the explanations of perceived smart people were elevated and relied upon, as had the soothsayers of old. And it’s not exactly been washed away by science, as Nassim Taleb explains:

But the problem is the one-eyed following the blind: these self-described members of the “intelligenzia” can’t find a coconut in Coconut Island, meaning they aren’t intelligent enough to define intelligence hence fall into circularities — but their main skill is capacity to pass exams written by people like them.

With psychology papers replicating less than 40%, dietary advice reversing after 30 years of fatphobia, macroeconomic analysis working worse than astrology, the appointment of Bernanke who was less than clueless of the risks, and pharmaceutical trials replicating at best only 1/3 of the time, people are perfectly entitled to rely on their own ancestral instinct and listen to their grandmothers (or Montaigne and such filtered classical knowledge) with a better track record than these policymaking goons.

Indeed one can see that these academico-bureaucrats who feel entitled to run our lives aren’t even rigorous, whether in medical statistics or policymaking. They cant tell science from scientism — in fact in their eyes scientism looks more scientific than real science.

(For instance it is trivial to show the following: much of what the Cass-Sunstein-Richard Thaler types — those who want to “nudge” us into some behavior — much of what they would classify as “rational” or “irrational” (or some such categories indicating deviation from a desired or prescribed protocol) comes from their misunderstanding of probability theory and cosmetic use of first-order models.)

Taleb calls these people intellectual yet idiot (“IYI”). These IYIs don’t just harm themselves with undeserved certainty of thought; their wrong ideas harm academia, politics, popular culture, and, important to lawyers, the law. A great many of these bad ideas with the seal of science rush into law through social justice warriors attempting to re-make society from the top down, as many communist revolutions did last century. Also, forensic science is an area where both well-intentioned and duplicitous IYIs peddle snake oil.

The latest exposé on forensic science is courtesy of the White House. The Wall Street Journal obtained an advance copy of it and reported the following:

The report, a copy of which was reviewed by The Wall Street Journal, raises questions about the use of bite-mark, hair, footwear, firearm and tool-mark analysis routinely used as evidence in thousands of trials annually in state and federal courts.

“It has become increasingly clear in recent years that lack of rigor in the assessment of the scientific validity of forensic evidence is not just a hypothetical problem but a real and significant weakness in the judicial system[.]”

This news wasn’t exactly welcomed:

“What they’ve done is turn the accepted reliability of expert witnesses and their evidence on their heads,” said Jim Pasco, executive director of the Fraternal Order of Police. “As a result there will be people who are not going to go to jail who should be incarcerated and some who are currently incarcerated will be released. The effect will be a threat to the public safety of American citizens.”

That’s right; the safety of American citizens is at risk and people won’t go to jail all because forensic science isn’t scientific. It’s not like it’s been advertised as science and been labeled as such.

And we’re told that

“Just because there is a lack of science does not mean the analysis is inaccurate or done wrong or is not worthwhile,” [said Jim Bueermann, president of the Police Foundation].

That sounds a lot like alchemy, whereby you get gold at the end of pseudo-science process with materials that have no practical hope of getting you what you want. And it appears like Bueerman wanted a conviction with the veneer of scientific certainty. The actual truth of the process is really beside the point.

All of this grumbling got a little louder when the report was actually released. The Attorney General of the United States, the same one appointed by the guy living in the White House, was having none of it:

In a statement, Attorney General Loretta Lynch said the Justice Department had taken unprecedented steps to strengthen forensic science, including investments in research, draft guidance to lab experts when they testify in court and “reviews of forensic testimony in closed cases.”

“We remain confident that, when used properly, forensic science evidence helps juries identify the guilty and clear the innocent, and the department believes that the current legal standards regarding the admissibility of forensic evidence are based on sound science and sound legal reasoning,” Ms. Lynch said. “While we appreciate their contribution to the field of scientific inquiry, the department will not be adopting the recommendations related to the admissibility of forensic science evidence.”

In other words, we’re all for science unless that science calls into question the things we call science. That’s kind of cute, in the misuse of science sends people to prison sort of way.

Maybe the report made logical leaps; let’s check:

The report said legal standards in cases involving scientific evidence should be “based upon scientific validity.” It added that “neither experience, nor judgment, nor good professional practices (such as certification programs and accreditation programs, standardized protocols, proficiency testing, and code of ethics) can substitute for actual evidence of foundational validity and reliability.” * * *

PCAST expresses no view on the legal question of whether any past cases were ‘erroneously decided,’” the report said. However, PCAST notes that, from a scientific standpoint, subsequent events have indeed undermined the continuing validity of conclusions that were not based on appropriate empirical evidence.

Nope. For the Justice Department, doing science in a scientifically valid way is just a bridge too far. After all, it’s not like they are like they are pursuing forensic science as pure intellectual endeavor, they use it to send bad guys to jail. Departing from science in the name of criminal justice is no doubt worthy of a pass. Let the eggheads play with their supercolliders and wear lab coats.

Cops and prosecutors complain; defense attorneys and, likely, real scientists cheer. At least one judge, Alex Kozinski, sees this as a big deal:

The White House will release a report Tuesday that will fundamentally change the way many criminal trials are conducted. The new study from the President’s Council of Advisors on Science and Technology (PCAST) examines the scientific validity of forensic-evidence techniques—DNA, fingerprint, bitemark, firearm, footwear and hair analysis. It concludes that virtually all of these methods are flawed, some irredeemably so. * * *

The PCAST report recommends developing standards for validating forensic methods, training forensic examiners and making forensic labs independent of police and prosecutors. All should be swiftly implemented. Preventing the incarceration and execution of innocent persons is as good a use of tax dollars as any. The report will also immediately influence ongoing criminal cases, as it provides a road map for defense lawyers to challenge prosecution experts.

Hope springs eternal. The NAS report did not change anything. Daubert did not change anything. Thomas Szasz spent fifty years arguing that the roots of psychiatry were rotten and nothing ever changed. But there is some reason to hope. We no longer see fortunetellers examining chicken entrails to be reliable. Perhaps one day the credentialed forensic scientist testifying about things like scent profiles will seem just as quaint.


6 Comments on this post.

Leave a Reply



Comments for Fault Lines posts are closed here. You can leave comments for this post at the new site, faultlines.us

  • Jeff Gamso
    23 September 2016 at 10:34 am - Reply

    Does this mean that you’ll no longer be using faux-scientific forensics to prosecute folks?

    Or that when the criminalist tells the jury that the fingerprints point to the defendant or that the shell casing came from the gun found in the defendant’s car you’ll be sure to have the same criminalist tell the jury that his conclusion is based on nothing more than a sheer guess, that it might be right but might be bullshit and he really doesn’t know?

    Inquiring minds want to know.

    • Andrew Fleischman
      23 September 2016 at 11:47 am - Reply

      The report was actually a lot kinder to ballistics than I expected. While it pointed out problems, the studies it cited suggested more accuracy in a “blind” test than I would have assumed, with only a handful of examiners routinely making bad calls.

      • Andrew King
        23 September 2016 at 11:51 am - Reply

        I recall from the NAS report, some of the issues among the fields were that the reliability had not been established with any rigor. So, it is possible that some methods of questionable reliability will later be established to be reliable. I don’t remember it being kind to ballistics, so perhaps it’s gotten more rigorous.

        • Andrew Fleischman
          23 September 2016 at 3:29 pm - Reply

          Sorry, I meant the PCAST report. Basically, they said that almost all testing of ballistics examiners is crap. But, the one true, double blind study that came back showed only a small number of false matches. There was however, a 33% “inconclusive” rate when ballistics examiners weren’t given the extra nudge that they get in real cases.

          “Among the 2178 different-source comparisons, there were 1421 eliminations, 735 inconclusives and 22 false
          positives. The inconclusive rate was 33.7 percent and the false positive rate among conclusive examinations was
          1.5 percent (upper 95 percent confidence interval 2.2 percent). The false positive rate corresponds to an
          estimated rate of 1 error in 66 cases, with upper bound being 1 in 46. (It should be noted that 20 of the 22 false
          positives were made by just 5 of the 218 examiners—strongly suggesting that the false positive rate is highly
          heterogeneous across the examiners.)”

  • Andrew King
    23 September 2016 at 11:48 am - Reply

    Have I ever done that?

    My take on how the application can work is reflected in what I wrote about Aaron Hernandez and the shoe impressions.

    This discussion is good because it can ultimately screen out some of the junk and re-calibrate expectations on what certain fields can accomplish. The level of certainty in fingerprint analysis is not as high as DNA, but that doesn’t mean that they have no value at all. Whereas you might be willing to prosecute a rape case solely on recovered DNA, you might not want to do that with a single shoe impression at a murder scene.

    I don’t think infallibility is the standard we ought to have and not the one required under Daubert. After all, nothing involving humans is truly infallible.

  • maz
    23 September 2016 at 6:19 pm - Reply

    Speaking of alchemical transmogrifications, this is quite a nice silk purse of a blog entry you made from what has to be the ugliest pig-ear of an article I’ve had the misfortune to read in quite a long time.