Fuzzy Math: Advances in DNA Mixture Interpretation Uncover Errors in Old Cases
Oct. 8, 2015 (Mimesis Law) — With the exception of DNA, no single forensic technique has the potential ability to establish a definitive link of an evidence sample to its source. Potential, however, can be very different from reality, and even DNA evidence has its limitations and weaknesses. The latest reality check: the statistics utilized in the interpretation of DNA mixture evidence could be so wrong that they require retesting of thousands of cases.
A quick science lesson on DNA mixture evidence: DNA mixtures are, just like it sounds, a mix of DNA from more than one contributor. There are a variety of reasons for mixtures, including contribution by multiple donors during the crime, existence of DNA on the item prior to the crime, contamination during evidence processing, deposit of DNA after the crime due to evidence handling by someone other than law enforcement.
From there, it can be difficult to individualize each contributor’s DNA just as a matter of laboratory process. Compiling the problem, crime labs use varying approaches in interpreting mixed samples, and the forensic science community has not reached agreement on the best approach to use with mixture interpretation. Consequently, precision and accuracy (or, to put it in legal terms, “reliability”) suffers in these cases.
Does your head hurt yet? It gets worse. Crime labs also differ in how they conclude that a mixture is “resolvable.” Resolvable means that at least one DNA profile in the mixture can be positively linked to a person. This is further complicated by the fact that the DNA test may not be blind. If the lab has a comparison sample from a suspect, the analyst may use that sample to resolve (or work backwards to solve) the mixture problem.
Last week’s headline, however, was not aimed at the process; it was aimed at the quantification of DNA mixture evidence. We use population genetics to arrive at a statistical answer. Anyone who has watched an episode or two of Law and Order or CSI [fill-in-the-blank] can tell you that testimony about DNA is delivered in terms of “1 in 5 quadrillion” when linking a suspect to a crime.
What does that number actually mean? In short, it means that it is 5 quadrillion times more likely that the DNA profile originated from the suspect than from an unknown individual in the U.S. population. Often, there will be different numbers attached to different races in the statistical evidence (Caucasian, Hispanic, African-American) and a disclaimer that the numbers assume unrelated individuals.
Those improvements in DNA science and interpretation techniques have caused practitioners to change how they calculate probabilities when it comes to singling out a defendant based on DNA mixtures (i.e., when biological evidence includes samples from more than one person). In the most extreme instance, the new method reduced a one in a billion probability that evidence matched a particular suspect to around one in 50.
After that detour, let’s get back to the story. Time and again, courts, lawyers, crime labs, and the media have referred to DNA evidence as the “gold standard.” We likely can make that claim when you have a single profile to look at: one profile in the evidence to compare with one suspect. 1:1 is always the easiest. As you can gather from above, the mixtures present a far more difficult conundrum and don’t come close to being a gold standard.
But DNA research is not static, and development of best practices in mixture interpretation has been ongoing. Progress in analysis and data interpretation techniques have caused practitioners to modify how they calculate probabilities when it comes to individualizing a suspect from a DNA mixture. On the one hand, this is a good thing: Better science equates to more reliable convictions. On the other hand, lots of cases were subject to substandard practices.
In August, the Texas Forensic Science Commission publicly revealed that there are some serious issues with DNA mixture interpretation. Among the revelations: in May 2015, the FBI notified crime laboratories it had identified “minor discrepancies” in its population databases that have been used by labs in DNA analysis since 1999. The FBI attributed the discrepancies to human error and technological limitations.
No doubt about it: minor or not, this “discrepancy” is a major problem. Chief among them is the shocking admission that in one case the new method reduced a 1 in a billion probability that the evidence implicated a particular suspect to roughly 1 in 50. Only one zero behind that five instead of six, nine, or twelve. Those numbers are not in any way, shape, or form “minor.”
In response, Texas CODIS laboratories (CODIS being the software that runs DNA profiles across the country) notified prosecutors, the criminal defense bar and the Texas Forensic Science Commission about the corrected data. This of course begs the “obvious question” that the letter noted: “whether and to what extent the probabilities associated with any particular inclusion changed because of the database errors.”
As Grits for Breakfast noted, this became public at the August 14th Forensic Science Commission meeting, and then broadcast in a “blandly titled notice” from by the Texas Forensic Science Commission: “Notification re: FBI Allele Frequency Corrections and DNA Mixture Interpretation Issues in Texas.”
To be fair, the FBI discovered only a small number of errors in its database (51 problems out of tens of thousand of entries) and then distributed a notice explaining that the modification of probabilities would be insignificant. A few prosecutors have even asked for the probabilities to be recalculated.
Great for Texas, but what about the rest of the country where there are more cases that used these calculation methods? Are these just a drop in the bucket? How do we go back and evaluate old cases?
Mixture evidence is a quagmire steeped in controversy, and it has been that way for decades. The Texas Forensic Science Commission noted that this issue came to light because of “[c]hanges in mixture interpretation [that] have occurred primarily over the last 5-10 years.” Five-to-ten years is a long time to be recalculating numbers and retooling formulas without letting anyone know about it. But there were hints out there.
In 2013, a survey by the National Institute of Standards and Technology (NIST) asked DNA analysts from 108 labs examine a three-person mixture and form a conclusion about whether a suspect’s DNA was present. The results were astounding: 70% of the analysts determined that the suspect could be a contributor; 24% found the data inconclusive. A mere 6% got it right: the suspect was not in the sample.
As we are finding with other forensic errors, these things take time and are often the tip of the iceberg. I applaud Texas for at least getting the issue out there and hopefully they will tackle it head on. There was a time that I liked to pick on Texas’s backwards crime and justice policies, but recently I’ve been a fan of the state’s willingness to explore changes and transparency. Other states should follow suit.