Creating a High-Quality Criminal Justice System | John Hollway | TEDxPenn

Key Concepts

• Wrongful Conviction: The conviction of an innocent person for a crime they did not commit.
• Exoneration: The act of clearing someone of guilt or blame.
• Sentinel Event Review (SER): A blame-free, root cause analysis of unintended negative outcomes in complex systems.
• Swiss Cheese Model: A conceptual model illustrating how multiple layers of defense (slices of Swiss cheese) can have holes (opportunities for error), and when these holes align, an error can pass through undetected.
• Blame-Free Analysis: An approach that focuses on understanding systemic failures rather than assigning individual blame.
• Contributing Factors: Multiple elements that, in combination, lead to an undesirable outcome.
• Never Events: Incidents in healthcare that should never happen, such as wrong-site surgery.

Marshall Hail Case Study: A Wrongful Conviction

The video opens with the case of Marshall Hail, who was wrongfully convicted of rape in Philadelphia in 1984 at the age of 24. He served 33 years in prison before being exonerated in 2017. The key to his release was a blood test conducted by the Philadelphia Police Crime Lab on the same day as his trial concluded. However, neither the prosecution nor the defense was aware of this test. The report was not delivered to Mr. Hail for 10 years, by which time he was unrepresented and lacked the scientific understanding to utilize it. It wasn't until 2009, when the Pennsylvania Innocence Project took on his case, that a scientific expert reviewed the evidence, leading to his eventual release through appellate litigation. DNA testing, which could have proven his innocence earlier, was not available at the time of his trial, and the physical evidence had been lost. This case highlights how miscommunications and systemic blunders can lead to prolonged incarceration, even when the evidence for innocence exists.

The Scope of Wrongful Convictions in the US Criminal Justice System

Cases like Marshall Hail's are not isolated incidents. The National Registry of Exonerations has documented over 3,600 instances of innocent individuals being convicted and incarcerated, accumulating over 33,000 "life years" lost in prison. Researchers study these cases to identify systemic flaws and prevent future injustices. The unifying sentiment from those involved in these cases is a shared desire to avoid such outcomes, yet they continue to occur. The criminal justice system is described as a complex, dynamic, and stressful human interaction system where decisions are made with imperfect information, and lives are at stake. While human error is inevitable ("to err is human"), the system is designed with checks and balances to catch mistakes. However, sometimes errors slip through these safeguards and go undetected for extended periods, or potentially, forever.

Comparing Criminal Justice Error Rates to Other Complex Systems

The speaker proposes learning from other complex, high-stress human interaction systems to improve criminal justice.

Transportation Safety

Car travel, despite its inherent risks, is considered relatively safe due to extensive efforts to enhance safety over time. For every 300,000 car trips, approximately three people are injured, meaning only one in 100,000 trips results in an injury. This high level of safety fosters public trust in services like Uber and Lyft.

Medical Safety

Medicine also deals with high-stakes situations, identifying "never events" – incidents that should never occur, such as operating on the wrong limb or leaving surgical instruments inside a patient. The medical field has significantly improved its safety and quality systems to prevent these events. For every 300,000 surgeries, there are approximately 25 never events, making it more dangerous than car travel but still remarkably safe.

Criminal Justice Error Rates

Comparing these to criminal justice, it's challenging to determine the exact number of innocent people currently incarcerated. However, data from individuals convicted of murder and sentenced to death who were later exonerated provides a statistically significant sample. Analysis of this data suggests an error rate of 4% in capital punishment cases, meaning one in every 25 capital cases results in a wrongful conviction on guilt or innocence. With approximately 2,250 people on death row, this implies about 90 are likely innocent, though their identities are unknown. The speaker conservatively estimates the error rate in all criminal cases to be 2%, leading to an estimated 7,500 mistakes in every 300,000 trials. This highlights criminal justice as significantly less safe than transportation and medicine in terms of guilt or innocence outcomes.

Learning from Sentinel Event Reviews in Other Industries

To understand how other industries reduce errors, researchers interviewed hospital administrators and the National Transportation Safety Board. They discovered that when a "never event" occurs (e.g., wrong-site surgery, plane crash), quality and safety engineers conduct a sentinel event review (SER). This is a blame-free root cause analysis aimed at understanding why an unintended outcome bypassed existing safety measures.

The Principle of Blame-Free Analysis

The crucial difference in SERs is the blame-free element. This doesn't mean absolving individuals of the need for certifications or training. Instead, it's an endeavor not to find fault and punish individuals. The focus is on understanding how good people, with talent, training, and good faith, all aiming for a positive outcome, can nonetheless participate in an undesired one. By presuming competence, different questions are asked, leading to different solutions.

The Philadelphia Eagles Analogy

The concept is illustrated with an analogy of the Philadelphia Eagles and their quarterback, Jalen Hurts. Even with highly talented and trained individuals operating in a high-stress environment, unexpected events can occur during a play, leading to an interception (a sentinel event). While spectators might immediately blame the quarterback, the team understands the complexity of the play and that Hurts intended the best outcome. The team's approach is to review the play frame-by-frame, asking "Why did you do that?" not to punish, but to understand contributing factors and redesign the play to put the team in a better position for future success.

The Swiss Cheese Model of Systemic Error

In transportation, medicine, and professional football, SERs consistently reveal that sentinel events are not caused by a single factor but by a confluence of contributing factors. To explain this, the Swiss Cheese Model is introduced.

Applying the Swiss Cheese Model to Marshall Hail's Case

Imagine Marshall Hail's wrongful conviction as a block of Swiss cheese, spanning the 33 years from the crime to his exoneration. Each human decision made during this period represents a slice of Swiss cheese. The holes in each slice are opportunities for error. Normally, errors like an inaccurate arrest are caught by existing checks and balances (prosecutor screening, defense attorney alibis, judicial review). However, in Mr. Hail's case, all the holes aligned, allowing the error to pass through undetected. This is akin to a wrong-site surgery or a plane crash.

Learning Opportunity from Systemic Flaws

While exposing systemic flaws is concerning, it presents a significant learning opportunity. The goal is to close the holes in each part of the system, as any single closed hole could have prevented the negative outcome.

Sentinel Event Review of Marshall Hail's Case

Researchers applied this model to Mr. Hail's case, gathering all involved agencies for a blame-free SER. They found no intentional misconduct. The victim's mistaken identification was sincere, but police protocols for evaluating it were outdated. The crime lab had communication issues with prosecutors and defense attorneys, and supervisory oversight was lacking. The lab has since committed to rewriting reports for better understanding. The courts, lacking expertise in physical evidence handling, mishandled evidence that was eventually lost due to flooding.

Findings and Recommendations

The review identified 46 contributing factors and situations, any one of which, if addressed, could have prevented Mr. Hail's wrongful conviction or accelerated his release. These factors led to 29 recommendations for systemic improvements. This process has been replicated in other criminal justice "never events" with similar results: dozens of contributing factors and recommendations. The focus remains on improving the system by putting good people in better situations to succeed.

Addressing Intentional Misconduct within the Swiss Cheese Model

While acknowledging that police and prosecutorial misconduct does occur, the speaker argues that it happens less frequently than perceived. Even in cases of intentional misconduct, the Swiss Cheese Model remains applicable. The question shifts to "Why did that person think that acting intentionally would work?" and "Why didn't we detect it?" This leads to examining upstream holes in the system that can improve detection and deterrence of misconduct. Thus, the system can still be improved even within a blame-free framework.

Dramatic Results of Integrating SERs into System Culture

The impact of incorporating SERs into a system's quality, change, and safety culture is dramatic. The graph shows a significant reduction in fatal aviation crashes per million flights over 50 years, coinciding with the use of SERs. This increased safety has led to a four-fold increase in the number of flights taken, as people trust the system's safety.

Contrast with Officer-Involved Shootings

In contrast, officer-involved shootings in the US over the last 10 years show a different trend. The current approach in criminal justice has been to remove officers rather than learn from events. This doesn't help good officers navigate difficult situations better. The speaker emphasizes that human behavior cannot be changed, but the conditions in which humans operate can be.

The George Floyd Case and the Need for Systemic Change

The murder of George Floyd on May 25, 2020, is presented as a tragic example. While accountability reviews and criminal trials have led to sanctions for the officers involved, the question remains whether this is sufficient to prevent future deaths in custody or protect officers from situations requiring deadly force. The speaker posits that if SERs had been implemented 50 years ago (like in aviation) or 30 years ago (like in medicine), the holes in the Swiss cheese might have been closed, potentially preventing Mr. Floyd's death and the subsequent outrage. This could have prevented the repeated occurrence of similar sentinel events and the anguish felt by millions.

Conclusion: Sentinel Event Reviews as a Preventative Measure

The deaths of George Floyd and Marshall Hail are described as tragedies composed of numerous small errors that converged to cause immense harm. The speaker concludes by stating a belief that sentinel event reviews can prevent "never events" in criminal justice, including wrongful convictions, officer-involved shootings, and deaths in custody, by systematically identifying and addressing contributing factors.