Evaluating Scientific Evidence in Court with the “Daubert Test”

By Sheeva Azma

Have you ever thought about the ways science is used in the “real world,” such as in the courts?

Thanks to my membership in the National Association of Science Writers, I was lucky enough to attend a virtual, two-day conference on science and the courts for free. The conference happened in person at the American Association for the Advancement of Science (AAAS) headquarters in Washington, DC, with a Zoom option that I happily used, since I do not live in DC anymore. The AAAS is where I spent a lot of time as a graduate student, learning about science policy and networking with science policy insiders. I was only able to attend the first day of this conference, but it was exciting and packed with information all the same.

I am excited to recap my experience at the AAAS Scientific Evidence and the Courts conference in a series of blogs! In this first blog, I talk about the first session of the day, which was about something called the “Daubert test.” If you don’t know what that is, it’s a way to establish the validity of scientific evidence in court.

Keep reading for a deep dive on the Daubert test, its challenges, and ways it’s been helpful to engage scientists in the judicial system. Want to read all the recaps? Find them here.

The conference kicked off with comments from Sudip Parikh, CEO and Executive Publisher of the Science Family of Journals, AAAS, and Ms. Theresa Harris, Director, AAAS Center for Scientific Responsibility and Justice.

Uniting Different Modes of Thinking about Knowledge

“It takes a small group of people, bringing their energy to the fray…[recognizing] different types of knowledge…and different perspectives.” I didn’t know what he was talking about, as science is the only way I’ve ever really tried to understand the world for decades, until I heard the first speaker. Then, the disconnect between law and science started to make sense.

This first session was introduced by Dr. Anne-Marie Mazza, who used to be a presidential science advisor and now serves as Senior Director of the Committee on Science, Technology, and Law within the National Academies of Sciences, Engineering, and Medicine (NASEM).

“Where Law and Science Meet”

Evidence law expert Dr. Jennifer Mnookin, currently also Chancellor at the University of Wisconsin-Madison, kicked off the conference with the first talk, “Where Law and Science Meet.” A main theme of her talk: evaluating scientific evidence is a difficult task for which scientists do not themselves have a process, so the challenge was simply passed on to courts. In the legal system, when scientific evidence is involved, the judges become gatekeepers of knowledge.

How Should Courts Use Scientific Evidence?

Mnookin recapped the past 30 years since the landmark Supreme Court Case, Daubert v. Dow Pharmaceuticals, Inc. Often known as “Daubert,” this case was decided in 1993. Daubert set a new standard for the use of scientific evidence in courts. A famous quote from the case, as Mnookin related, is as follows: “In a case involving scientific evidence, evidentiary reliability will be based on scientific validity.”

Prior to this decision, there was an existing mechanism to evaluate scientific evidence, and Daubert added a second way. The previous standard, called “Frye,” had been established through a 1923 US Supreme Court case called Frye v. United States. Frye came from a 1923 case involving a lie detector that measured systolic blood pressure. Frye asks: Is this evidence generally accepted by scientists in a relevant field? Basically, for the Frye standard, a scientist had to state that the evidence was acceptable within the scientific norms of their field. There’s “strict Frye” and “lax Frye,” according to Dr. Mnookin, where the question becomes: does this evidence have to be accepted by practitioners of a specific field or a whole group of scientists?

When the Daubert decision came out in 1993, Mnookin was a second-year law student, and a first-year PhD student studying the history of science and technology. She was interested in the intersection of science and law at the time, and was working as a summer associate for a DC law firm. After the Supreme Court issue the opinion, the law firm discussed it over lunch shortly thereafter. They asked themselves what Daubert said and what it meant. The lawyers at the lunch had different opinions. They all agreed that Frye was no longer the law of the land for scientific evidence. The lawyers wondered: did Daubert incorporate Frye or replace it?

Mnookin talks about the context of the time, not mentioned in the Daubert opinion, but important to understand it: “junk science” was becoming a problem in the courts, leading to unjust legal outcomes, driving products off the market, preventing innovation (since people feared lawsuits), and more.

Mnookin says the challenges of Daubert persist 30 years later. She says Daubert asked judges to play what she calls a “gatekeeping role” to assess if the science was valid and reliable. However, she says, Daubert offers ways to do that, that in practice, might leave the answer to that question pretty open ended.

What Is the Daubert Test, Exactly?

Mnookin says Daubert provided a non-exhaustive checklist to help courts decide scientific validity (after saying it would not provide a checklist):

1. Could the scientific claim be tested?
2. Had it been published via peer-review, which is the scientific process for producing knowledge?
3. Is there an error rate for this scientific claim, and if so, what is it?
4. Was the scientific information derived from validated scientific techniques?
5. The original question asked in the Frye test was still a criterion, though no longer the exclusive one: Is the scientific claim in question, accepted, generally, by the relevant scientific community?

Daubert provided broad criteria for judges to evaluate science evidence, but these inquiries wouldn’t come too naturally to judges, Mnookin says, since most judges don’t have deep expertise in science. The court also did not guide judges on which, if any, criteria should be more important than others. The court also did not provide judges with ways to apply the criteria to be scientifically reliable or valid enough for the legal system.

It wasn’t clear what Daubert would mean at that lunch back in 1993, though the lawyers all agreed on a few things. Clearly, Daubert would be important, though it wasn’t clear or easy-to-use. As a multi-pronged test, it would also be complex to apply. As a result, judges, tasked with the gatekeeping responsibility of assessing scientific evidence, would likely have great flexibility applying this “multi-pronged, somewhat ethereal” standard.

30 Years of the Daubert Test

Over the past 30 years, general themes have emerged with respect to Daubert, as well as general interest and “salience” around scientific evidence, with academic research, training, gatherings, and overall, greater opportunity to look at these issues, Mnookin says.

One unanswered question, Mnookin states, is: what should be admissible in court in the name of science and expertise? Furthermore, how should courts grapple with this question? Procedurally, things have changed over the past 30 years. Daubert pre-trial hearings have been a staple of civil courts; on the criminal side, they also exist, but are less common. More and more appeals cases have also grappled with admissibility of evidence. Law school training has changed, Mnookin says, with more focus on Daubert and the questions it raises. There are more conversations on the need to train people in the legal system in science.

Daubert changed the way courts evaluate and handle scientific evidence. According to Mnookin, Frye asked, “what does the scientific community think?” To Mnookin, it didn’t involve judges making a determination; that was the job of experts. With Frye, the judge asked the experts: what do you think about this evidence? Daubert, on the other hand, created a checklist of criteria that required judges to examine reliability and validity of evidence themselves.

Do courts reach their own conclusion to be educated enough, or do they simply defer to the judgement of the scientific community? Daubert didn’t clearly answer this question. Is it possible for a judge, who is normally not trained in science, to understand scientific evidence well enough to make an informed, scientifically valid decision?

Daubert might make more sense than Frye, Mnookin says, but it also “might actually be impossible.” We might not want courts to defer to experts, but we might not “have faith that judges are actually good at this.”

Limitations of the Daubert Test

Indeed, some limitations exist. Daubert doesn’t talk about how courts should think about multiple pieces of evidence. So, if evidence is flawed, it’s not clear whether they should be evaluated one-by-one or collectively. “Frankly, this is a hard question for scientists,” Mnookin says. She mentions that scientists themselves do not have a clear mechanism for doing that. If scientists struggle with this, how can the courts do it? “Daubert is actually asking courts to do really hard things, and we shouldn’t be terribly surprised when it turns out that they are probably too hard.”

Daubert in the Civil versus Criminal Courts

Daubert has affected both the civil and criminal sides of the courts, but divergently so. On the civil side, the impacts are far more clear. Daubert makes it more difficult for plaintiffs to prove things important to their case, because it raises the bar for admissibility of evidence. Daubert gives judges tools to exclude data as they see fit. This can especially impact civil cases involving causation – cause-and-effect relationships between an action and a result. Daubert can help judges prevent trials where people are persuaded by harm or negligence more than proof of causation – that an action caused a specific result. However, Mnookin also notes, if civil courts use Daubert too aggressively, this could create injustice: cases that should go to trial don’t; cases that shouldn’t be won are won; and cases that should not have lost, lose.

Daubert impacts the criminal side less. There is more engagement and more scrutiny around forensic science, often, but overall, has not shifted the admissibility of evidence, even around scientifically questionable stuff, such as fingerprint identification, which can be relatively subjective. Thanks to Daubert, though, experts no longer claim the fingerprint error rate is zero, or that they are 100% certain of something, or that a fingerprint can be matched exactly to a unique person.

Another difference in civil and criminal cases post-Daubert is that civil cases often have matched resources, but criminal cases often do not. Public defenders in the criminal system often do not have money to bring in experts, for example. Furthermore, while people can seemingly understand forensic evidence such as fingerprints, lawyers can be quick to overstate their applicability and accuracy. Forensic evidence’s scientific importance can be overstated, often without an expert to rebut it. Broader cultural beliefs may also play a role here. Fingerprints are thought of as a gold standard in the society’s discourse about forensics, so when they get to court, they are not often doubted. This is despite the fact that fingerprint identification analysis is not always entirely accurate and can be used to convict the wrong person.

Structural Challenges with the Daubert Test

Mnookin also notes that it’s worth saying that “there’s core structural challenges around the use of expert evidence that a standard can’t actually do that much to solve.” Two structural challenges unchanged by Daubert include partisanship and what Mnookin calls “epistemic competence.” In an adversarial system in which two parties are pitted against each other, such as in the courts, parties each call their own experts who share their own views.

Mnookin describes the idea of “epistemic competence” as such: “no matter how well-intentioned a smart creative judge is, they are not going to be a full citizen of the scientific form of knowing that’s at issue in the case – even if they are a PhD in science, actually.” There are places where judges and juries as decision-makers are not going to be able to decide on the questions they’re being asked. Science courts and expert juries won’t solve the problem, she says. “We can’t solve it, but we can name it,” Mnookin says. She believes it will be a lingering problem in the system.

Mnookin concludes her talk by saying that Daubert has created spaces to talk about these issues. Maybe there are no solutions, but we can work towards “methods and mechanisms” for recognizing when these tensions pose a genuine risk to fair outcomes in the justice system.

How Can Scientists Contribute to the Daubert Discussion?

So, what should the science community do on Daubert? It’s a difficult question, Mnookin says. Courts have resisted the idea that they must defer to scientists in the legal world. However, there’s been some change; in parts of the forensic community, there is more interest in these questions, for example. In 2016, the President’s Council of Advisors on Science and Technology (PCAST) published a report on law and science, and courts referenced it, but didn’t take it very seriously.

Overall, Daubert has created a space for these discussions, as well as space for the discussion of science literacy among judges and juries. In the 30 years since its publication, the Daubert decision seems to amplify a question scientists ask themselves daily: what makes science valid?

You can watch the whole session, and the whole symposium, on YouTube, or read more of my first-day recaps.