Are Judges Gatekeepers of Science in the Courts?

By Sheeva Azma

Thanks to my membership in the National Association of Science Writers, I was lucky enough to attend a virtual, two-day conference on science and the courts for free. The conference was happening in person at the headquarters of the American Association for the Advancement of Science.

I was only able to attend the first day of the conference, and I’ve already blogged about the first session. Here’s a recap of the second session I attended! If you want to read all of the recaps, you can find them here.

The second session of the day at the Scientific Evidence and the Courts conference examined the role of judges as gatekeepers. Judges determine what scientific evidence and science expert testimony can be used in trials. This session, a panel discussion, also discussed ways to help judges be better able to fulfill this gatekeeping role.

What Challenges Face Courts in Evaluating Scientific Evidence and Expert Testimony?

The panelists included a retired judge, Hon. Nancy Gertner, currently Senior Lecturer at Harvard Law School; Prof. Tess Neal, a psychology professor at Iowa State University, and Dr. Beth Wiggins, Director of the Research Division at the Federal Judicial Center. The discussion was moderated by Dr. Alison Renteln, a professor of political science, anthropology, public policy and law at the University of Southern California.

Hon. Gertner’s comments kicked off the panel discussion. She began her part of the session saying, “It’s wonderful being here, but I have nothing to say. Mnookin said it all.”

She recounted that in 2016, the Presidential Council of Advisors on Science and Technology or PCAST issued a report about forensic evidence in criminal courts. “There is stuff out there that is clearly junk,” such as bite mark testimony, Hon. Gertner says. She also says that judges don’t examine the forensic evidence closely to eliminate the junk: “I know the scientist said this, but who cares, I am going to let it in.”

Where does judges’ gatekeeping of science break down?

Hon. Gertner then talked more specifically about a few places where judges’ gatekeeping of science breaks down, mostly on the criminal law side. Four places where the gatekeeping breaks down are precedent, advocacy, trial courts, and appellate review. Three that Hon. Gertner mentioned in detail (that I could follow as someone with no legal background) were precedent, advocacy, and appellate review.

Precedent: Courts have permitted certain types of forensic evidence for decades, even if the science doesn’t fully support it. Types of evidence in this category include DNA evidence, ballistic evidence (such as bullet holes), fingerprinting, bite marks, and other types of forensic evidence. When these types of evidence has been accepted for decades, this creates a precedent and therefore, a disincentive to challenge the evidence – judges are reluctant to exclude evidence that previous judges have allowed. 

Advocacy: Precedent ties into the need for legal advocacy, which uses the judicial system to defend and protect people’s rights and advance social change goals. Lawyers see that evidence has been admitted over and over again, and take advantage of this, submitting more. If lawyers on the other side of the case don’t bring up the issue, then it will be admitted without objection. Lawyers have limited resources, and so they won’t take up their resources to challenge scientific evidence that has limited validity.

Trial Courts: A trial court is first to hear a case. Gatekeeping is most important here since the evidence that is excluded is unlikely to be admitted in the future through appellate review, which is the next thing that Hon. Gertner talked about.

Appellate Review: Appellate review refers to what people who follow court decisions, such as those of the Supreme Court, know as an appeal to a case or a challenge to the outcome. There is no real impetus to challenge problematic evidence such as certain types of forensic evidence (finger prints, bullet holes, etc.) until appeal courts begin reversing admissibility decisions. “It’s a vicious cycle,” says Hon. Gertner. If you take your chances on introducing evidence, you’re affirmed, and that’s how lawyers get their evidence into the case. Hon. Gertner says that appellate standards should be made more clear for the exclusion of evidence.

Examples of “Precedented” or “Pedigreed” Science in the Courts

The judge recounted an example of a case she judged in which bad evidence actually falsely convicted someone. She says these types of cases much more often happen on the civil side, where there are much more resources and cases proceed slowly. The judge has more time and more resources as well, as Hon. Gertner pointed out, but because certain types of evidence are always admitted, this sets a precedent for what will be admitted, even if it is not scientifically valid.

Hon. Gertner gave an example of an arson case she once judged, which she described as “a tough case.” A dog handler testified that the dog wagged their tail in a way that the handler identified. “I literally dropped my pen,” Hon. Gertner says. The judge called for a break and walked up to the lawyer on the opposing case, asking him if he would challenge it. The lawyer said that he would not. The judge did not know much about the case, and let the case go on, and the dog handler went on to testify that the dog wiggled its nose as evidence. Frustrated by the lack of scientific merit, Hon. Gertner kept calling for breaks. In the end, the defendant was convincted of arson. There was no motion to set aside the verdict. Because there had been no pre-trial of this case, there was no way to dig deeply into it. “To make a long story short,” says Hon. Gertner, a later revisiting of the case, which involved getting into the details of arson science, granted release to the defendant under habeus corpus, or unlawful imprisonment.

Hon. Gertner gave a second example. She says she was on her way to go to a session about the 2009 National Academy of Forensic Science report (read about its impacts at the Innocence Project), when she was affirmed in a fingerprinting case for not having held a hearing, so she held a hearing. The lawyer had submitted a one-page motion to challenge the fingerprints, saying she had a background in fingerprint science. The jury is selected and waiting outside. The lawyer looks at the judge and says “I have nothing.” Hon. Gertner had given them a hearing to discuss fingerprint evidence and they had not done their research! Then, the lawyer said, “you give me too much credit.” Hon. Gertner listed out fingerprinting issues and everything she knows about them. Sadly, the defendant was convicted, and this case was seen by the Supreme Court as US vs. Peña. While the first circuit court affirms the judge for not holding a hearing for evidence over and over again for several reasons, it also gives reasons the fingerprint evidence should be admitted. As this example shows, “automatic” review of evidence makes it more likely that fingerprint evidence will be admitted and could likely decide the case, even if it’s based on shaky science.

In sum, Hon. Gertner stated, criminal courts rely on forensic data which has been used for years and is rarely challenged. Some judges just always admit them, and they are tough to throw out.

The takeaway message? Appellate lawyers need to be better at figuring out the scientific evidence piece, because evidence rarely gets thrown out at the level of appeals. What’s more, lawyers must figure out how to deal with changed science, as there are certain types of scientific evidence that always get admitted – “precedented” or “pedigreed” science, as Hon. Gertner calls it – things like fingerprints, handwriting, etc.

Are there models for changing the courts’ take on scientific evidence?

A model for changes in the courts relates to eyewitness identification. A report on the science of eyewitness identification was admitted into one of the Supreme Court cases. By the time the report came out, a consensus accumulated for change. Hon. Gertner says that the key is not to announce that something is wrong or isn’t enough – there have to be institutional supports for that. Until the “abusive discretion standard” is changed and judges’ decisions are reversed based on what they actually did versus didn’t do, there will not be a meaningful change on the criminal side.

Unfortunately, Hon. Gertner says, the structure of judicial decisionmaking has made it resistant to what she calls a “chorus of challenges” that have pervaded the courts since at least 2009.

What quality of scientific evidence are psychologists bringing to court?

Prof. Tess Neal was the next panelist to speak. Prof. Neal, a psychology professor at Iowa State University, stated that the stakes of expert evidence are high and that this is a “very very hard task,” but that evidence review is changing for the better in federal courts right now. She won a Fulbright scholarship to focus on studying the evidence rules in Australia, which are different from those here in the US, to learning any lessons we can from the Aussies. Her Fulbright research showed that, while Australia’s approach is less stringent than the US’s, it’s not working any better (though it also is not worse despite being more lax).

Psychology evidence is particularly complex, both in the courts and outside of them, because each expert brings their own methodology. They bring in their own evidence with respect to psychological assessment, and there are tens of thousands of different assessments out there. Therefore, it’s “an extremely tall order” to decide on the validity of these tools, just in the field of psychology.

The difference between “good” and “junk” science, Prof. Neal states, is the method used by the expert. Has it been tested? Is it generally accepted by other experts in the same field? Has it been peer-reviewed by experts in the same field? What is its error rate? These are the standards set by Daubert, and its predecessor, Frye, which Daubert replaced, as well as the rules of evidence.

Prof. Neal conducted a study (Neal, et al., 2019) to determine how scientifically valid these assessment tools were, as used by psychologists as evidence in court. Are courts calibrated to the scientific validity of these tools? In other words, she asked the question: are courts keeping junk science?

She found that the courts were not calibrated to make the decision of whether psychological evidence was scientifically sound. Sometimes the psychologists’ expert evidence was valid, but sometimes it was junk science. In fact, there was no evidence for 51% of the psychological tools used in her subset of evidence. When there was evidence of general acceptance, 33% of tools were generally accepted, but 8% had conflicting or unclear evidence (which can be tough to grapple with for a judge), and 8% was not accepted.

Furthermore, when it comes to admissibility of psychological assessment tools, Prof. Neal found, in only 5% of the 372 cases she studied was there a challenge at all. Judges are stuck if lawyers don’t raise an issue, as Hon. Gertner noted earlier, and this data seems to provide numbers to this phenomenon that can be used to unjustly convict people based on false evidence.

In the 5% of challenges that she found in the 372 cases, about a third succeeded in convincing the judge that the evidence was not admissible because it was not scientifically valid. There are two issues that relate to the scientific validity, actually: admissibility and something else called weight, which relates to persuasiveness of the evidence. Judges in Prof. Neal’s study who admitted the challenged evidence did so because they decided the non-scientifically-valid research was less persuasive (carried less weight), but still admissible. As a document from the United States Courts about Evidence Rules from November 2021 states, many courts have decided that a scientist’s expertise relates to weight rather than admissibility, so the evidence gets presented in court. This passes the challenge on to the jury to decide about the scientific evidence. That doesn’t solve the problem, but passes the buck to someone else who is not a scientist to decide what is valid science and what is not.

How can the courts improve scientific evidence’s use in judicial decisionmaking?

Lastly, Prof. Neal talked about solutions to improve scientific evidence’s use in decisionmaking. Educating judges, juries, and lawyers can help the parties better vet and critically analyze scientific evidence, as can improving gatekeeping within expert domains. Consultants and non-adversarial experts can also play a role, as can revising the Federal Rules of Evidence, which occurred on December 1, 2023.

She points to the 2016 White House President’s Council of Advisors on Science and Technology (PCAST) on forensic science in criminal courts; the 2017 Boston Law School Symposium on Daubert, and the appointment of the subcommittee on the Rule 702 revision, which deals with expert testimony. The new 702 rules went into effect on December 1, 2023, after being drafted and approved at the Boston College Law School symposium, subjected to public comment, sent to the Judicial Conference of the US and approved, and lastly, approved by the Supreme Court, which sent a letter to both Congress (where you can read the amendments) and the Vice President.

Science literacy in government branches is lacking

Dr. Beth Wiggins spoke last in the panel event. She is Director of the Research Division at the Federal Judicial Center. The goal of the FJC is to educate judges and other court staff and to encourage and conduct research to improve courts. She says that courts must think outside the Rule 702 context; the need for judges to understand science transcends situations involving expert science testimony. She names addiction research as one area where judges can benefit from increased science knowledge. “The task is more than scientific evidence; it’s scientific education,” Dr. Wiggins says. Rather than teaching judges about science, Dr. Wiggins states that we must “teach them how to be good consumers and critiquers of scientific evidence.”

Dr. Wiggins started her talk discussing the ways different government branches use or do not use science. The executive branch, The White House, has dedicated science advisors; Congress has two committees on science and several committees on science oversight, and can request research from non-governmental organizations and in-house experts, though the members themselves don’t have too much scientific knowledge.

In the judicial branch, access to science is remarkably less. Judges often, without science backgrounds, use an adversarial process, in which briefs and evidence are partisan. This is important because it creates a problem. Scientific evidence is needed more and more in the courts but it is lacking. This results in what Dr. Wiggins says amounts to “asking people to do some very hard things.” How do we help people in the judicial system do those things?

These issues are not unique to the judicial system, but increasingly, involve it. Administrative agencies must point to clear congressional authorization on questions of major national significance, and Congress is not supposed to delegate this authority to other entities, like scientists. The Supreme Court has also been limiting decisionmaking authority of government agencies.

How do you bring general science to judges?

The FJC has two programs to help improve judges’ scientific acumen. One is educational programs that cover topics like law in technology, biosciences, neuroscience, artificial intelligence, and more, in order to increase cooperation with the scientific community and educate judicial decisionmakers. They have what they call “the trilogy of science evidence”: a manual for complex civil litigation, a benchbook for US district court judges, and a reference manual written by scientists and judges working together. The first chapter of the reference material is about admissibility of evidence. There’s also a chapter on science methods and the scientific process, which is really good grounding for judges.

Individual AAAS fellows also develop materials for judges on the FJC webpage on anything from fingerprint identification to the opioid crisis. The FJC also holds “science days” in which experts come in to teach an area of science, and talk about what makes science effective and fair as a mechanism.

The moderator, Dr. Renteln, agreed with Dr. Wiggins that a lack of science literacy is “the elephant in the room,” and that the solution is to teach more science to new judges.

Watch the session here or see the whole playlist of conference sessions on YouTube. Check out my other recaps here.