Bad Science Goes Up in Smoke

Forensic scientist John Lentini says he was as surprised as anyone at the Lime Street burn, a videotaped inferno that ultimately forced arson experts to abandon a set of flawed guidelines they had relied on for decades.

“We all believed what we’d been taught,” Lentini says. “Lime Street proved us wrong.”

Lime Street was a test conducted on behalf of the prosecution in a 1990 Florida murder case. Lentini, an expert on arson, had been called in because the circumstantial evidence against defendant Gerald Lewis was weak, even though the forensic evidence seemed solid. Arson investigators had said, for example, that the living room would have required 15 to 20 minutes to reach “flashover”—the point when the room explodes from the heat—not the 3 to 5 minutes Lewis had reported. A sofa cushion should smolder, not combust, the experts said, and the scorched trail down the hallway floor was the “classic” signature of liquid accelerant. Before taking the case to trial, the prosecutor wanted to know:  How reliable were those expert opinions?

Lentini and his team saw the perfect place to find out: the house next door, built on the same floor plan, probably at the same time, and slated for demolition. Investigators installed duplicate carpet and furnishings, and then, with firefighters on hand, turned on the video cameras and set a fire on the living room sofa, where Lewis claimed the flames had started. What happened next caught everyone by surprise. From Lentini’s published statement about the test burn:

The living room went to flashover in 4-1/2 minutes, only 60 – 90 seconds after we surmised it would have been necessary to evacuate the living room… There were firemen in the hallway outside the living room breathing fresh air only fifteen seconds before the fire flashed over and extended into the hallway.

The hallway floor was scorched, and the smoke travelled with the fire, “consistent with the low carbon monoxide levels found in the victims,” Lentini later wrote.

Gerald Lewis was cleared, and researchers started conducting controlled scientific tests to understand burn patterns and flashover conditions. The change took time, though. For the moving story of a man executed before the new understanding could derail his death sentence, see David Grann’s New Yorker piece about Cameron Todd Willingham.

After the test, Lantini says, he was surprised that a “substantial number” of other fire investigators, even after learning about the test burn and the exculpatory circumstantial evidence, looked at the physical scene evidence and concluded that the Lime Street defendant was guilty beyond a reasonable doubt. From his published statement:

This is the frame of mind which leads to wrongful convictions. Every one of us would agree that, in the American system of justice, a defendant is innocent until proven guilty. Agreement would be less widespread, however, to the proposition that every fire is accidental until proven incendiary. In fact, the presumption of innocence of an individual and the presumption of accidental origin of a fire are exactly equivalent. If this fire was set, the suspect set it. All the State needed to do was prove beyond a reasonable doubt that this was an intentionally set fire, and they would have proven beyond a reasonable doubt that the suspect was guilty.

Local coverage of the Lime Street burn offered this explanation for why fire investigators continued to trust the rules they’d been taught, even in the face of videotaped proof those rules were wrong:

Traditional theories about how fires burn had been developed over generations by firefighters who’d observed thousands of fire scenes. It wasn’t quantifiable theory, more a set of beliefs rooted in experience. But it carried the weight of fact. And as with any tradition in the close fraternity of firefighters, it was honored for the very fact that it represented the wisdom of veterans.

Replace the word “firefighters” in that paragraph with “child-abuse experts,” and I believe the observation still holds, which illustrates the flip side of something John Lentini said to me when I interviewed him last year. He had followed the Goudge hearings on faulty diagnoses of child abuse in Ontario, Canada, and he had noticed the parallels between old theories of fire investigation and the faith-based opinions of the child-abuse experts. “They’re in the same place,” he claims, “You can exactly replace ‘forensic pathology’ with ‘fire investigation’ in Goudge, and it describes what we had in the 1980s with fire investigation.”

He offers an example from the final Goudge report. The original:

The interpretive nature of forensic pathology—both in evaluating the findings made at the autopsy and determining what, if any, conclusions can be drawn from them—reinforces the limitations of the science. Even when the controversy does not divide the forensic pathology community, there are diagnostic challenges that limit what a forensic pathologist can reasonably say about an individual case, and the level of confidence or certainty with which he or she can say it.

Modified for fire investigation:

The interpretive nature of fire investigation—both in evaluating the findings made at the fire scene and determining what, if any, conclusions can be drawn from them—reinforces the limitations of the science. Even when the controversy does not divide the fire investigation community, there are diagnostic challenges that limit what a fire investigator can reasonably say about an individual case, and the level of confidence or certainty with which he or she can say it.

Most forensic investigation looks at the question of “Who did it?” Lentini points out, but fire investigators and child-abuse physicians are both trying to answer the question, “Was a crime even committed?” Unfortunately, he adds, the doctors are still relying on faith, not science.

Although his specialty is arson, Lentini was also trained in microscopic hair comparison while working at the Georgia Bureau of Investigation Crime Lab. “I was terrible at it,” he recalls, “The samples all looked the same to me.” He never incorporated that training into his work, which is probably a good thing, because the techniques he was being taught have failed the test of time.

Just this week, for example, a judge in Washington, D.C. acknowledged that DNA testing has exonerated 50-year-old Kevin Martin, who spent 26 years in prison for murder after an FBI visual analysis incorrectly matched him to a hair collected at the crime scene. Martin’s case is only the most recent refutation of hair evidence from that era:  The 2012 Innocence Network Journalism Award went to Washington Post reporter Spencer S. Hsu for his series about the unraveling of the subjective and statistically flawed techniques.

“It’s not a bad technique for elimination of matches,” says Lentini, “But it’s not reliable for positive identification.”

In a posting earlier this week about the FBI’s hair-matching cases, Nancy Petro at the Wrongful Convictions Blog observed that “these miscarriages were prompted by an unjustifiable trust in unreliable science presented by a highly credible source.” Again, I am struck by the parallels with the shaken baby arena:  The “highly credible source” for the microscopic hair analysis was the FBI. The “highly credible source” for the shaken baby testimony is the American Academy of Pediatrics. We want both of those institutions to always be right, but sometimes they’re wrong.

Fire investigators have now adopted scientifically established criteria, and DNA testing has replaced visual hair analysis in court, although the criminal justice system has not yet caught up with either development (see, for example, Martin Yant’s commentary on the Wrongful Convictions Blog). I do have faith that the truth will ultimately prevail in the shaken baby arena, but I am discouraged at how slow the change is in coming.

For a video demonstration of flashover, see https://www.youtube.com/watch?v=ZH0k-NthgTY

For a review of the dramatic changes in arson science in the past two decades, see Phil Locke’s treatment on the Wrongful Convictions Blog.

If you are not familiar with the debate surrounding shaken baby syndrome, please see the home page of this blog site.

copyright 2014, Sue Luttner