Creating a High-Quality Criminal Justice System | John Hollway | TEDxPenn

Key Concepts

• Wrongful Conviction: The conviction of an innocent person for a crime they did not commit.
• Exoneration: The act of clearing someone of blame or guilt.
• Sentinel Event: An unintended outcome in healthcare or other industries that should never happen, such as wrong-site surgery or leaving a surgical instrument inside a patient.
• Root Cause Analysis (RCA): A systematic process for identifying the underlying causes of an incident or problem.
• Blame-Free Sentinel Event Review: A type of RCA that focuses on understanding the systemic factors contributing to an event without assigning individual blame.
• Swiss Cheese Model: A model used to explain how errors can occur in complex systems. It visualizes layers of defenses (slices of Swiss cheese) with holes representing opportunities for error. When the holes align, an error can pass through the system undetected.
• Contributing Factors: Multiple elements that, when combined, lead to an undesirable outcome.
• Checks and Balances: Safeguards within a system designed to prevent errors.

Marshall Hail Case Study: A Wrongful Conviction

The video opens with the case of Marshall Hail, who was wrongfully convicted of rape in Philadelphia in 1984 at the age of 24. He served 33 years in prison before being exonerated in 2017. The key to his release was a blood test conducted by the Philadelphia Police Crime Lab on the same day as his trial concluded. However, neither the prosecution nor the defense was aware of this test. The report was not delivered to Mr. Hail for 10 years, by which time he was unrepresented by counsel and lacked the scientific understanding to interpret its significance. It wasn't until 2009, when the Pennsylvania Innocence Project took on his case, that a scientific expert reviewed the file, leading to his eventual release through appellate litigation. Crucially, DNA testing, which could have proven his innocence earlier, was not available in 1984, and the physical evidence had been lost. This case highlights how miscommunications and systemic blunders can lead to prolonged incarceration, even when the evidence for innocence exists.

The Scope of Wrongful Convictions in the U.S. Criminal Justice System

Cases like Marshall Hail's are not isolated incidents. The National Registry of Exonerations has documented over 3,600 instances of innocent individuals being convicted and incarcerated, collectively serving over 33,000 "life years" in prison. The speaker and their colleagues study these cases to identify preventative measures. Interviews with prosecutors, judges, defense attorneys, and police officers involved in these cases reveal a common sentiment: no one involved wanted these injustices to occur. The question then arises: if no one desired these outcomes, why do they happen?

Criminal Justice as a Complex Human System

The criminal justice system is described as a complex, dynamic, and stressful series of human interactions where decisions are made with imperfect information, and lives are at stake. Like any human-designed system, it is prone to errors. While checks and balances are in place to catch mistakes during investigation, trial, or appeal, some errors occasionally slip through undetected, as in Mr. Hail's case. The speaker posits that some errors may never be caught at all.

Learning from Other High-Stakes Systems: Transportation and Medicine

To understand how to reduce errors in criminal justice, the speaker's team examined other complex, high-stress systems where human lives are at stake: transportation and medicine.

• Transportation: Despite the inherent risks of car travel, the system is remarkably safe. For every 300,000 car trips, only about three people are injured (a 1 in 100,000 injury rate). This high level of safety instills confidence in services like Uber or Lyft.
• Medicine: Medicine also deals with "never events" – incidents that should never occur, such as operating on the wrong limb or leaving surgical instruments inside a patient. For every 300,000 surgeries, there are approximately 25 never events, making it more dangerous than car travel but still relatively safe.

Comparing Error Rates: Criminal Justice vs. Other Systems

The speaker then contrasts these with criminal justice. While it's impossible to know the exact number of innocent people currently incarcerated, data from exonerated individuals convicted of murder and sentenced to death provides an estimate. Statistical analysis of this dataset suggests an error rate of 4% in capital punishment cases, meaning one in every 25 capital cases gets the guilt or innocence wrong. Given that there are approximately 2,250 people on death row, this implies around 90 are likely innocent, though their identities are unknown. The speaker conservatively estimates the error rate in all criminal cases to be 2%, leading to an estimated 7,500 mistakes in every 300,000 trials. This raises the question of why criminal justice has a significantly higher error rate and what can be learned from transportation and medicine.

The Power of Blame-Free Sentinel Event Reviews

Interviews with hospital administrators and the National Transportation Safety Board revealed a key practice: sentinel event reviews. When a never event occurs (e.g., wrong-site surgery, plane crash), quality and safety engineers conduct a blame-free root cause analysis. This process aims to understand why the event evaded existing safety measures and led to a tragic outcome, with the goal of preventing recurrence.

The critical difference from criminal justice is the blame-free nature of these reviews. This doesn't mean absolving individuals of responsibility for their training or certifications, but rather an endeavor not to find blame and punish individuals. Instead, it presumes talent, training, and good faith, asking how good people, all aiming for a positive outcome, could nonetheless participate in an undesired one. This approach leads to different questions and, consequently, different solutions.

The Philadelphia Eagles Analogy: Understanding Complex Plays

To illustrate this concept, the speaker uses the example of the Philadelphia Eagles and their quarterback, Jalen Hurts. Even highly talented and well-trained individuals operating in a dynamic, high-stress environment can make mistakes. When an interception occurs (a sentinel event), the team doesn't immediately blame the quarterback. Instead, they conduct a frame-by-frame video review of the play, asking "Why did you do that?" This analysis focuses on the interplay of factors, such as player movements and their impact on the quarterback's decision, to redesign the play and better position the team for future success, rather than punishing individuals.

The Swiss Cheese Model: Visualizing Systemic Failures

Scientists studying sentinel events in transportation, medicine, and professional football consistently find that these events are not caused by a single factor but by a confluence of contributing factors. The Swiss Cheese Model is used to explain this.

In the context of Marshall Hail's wrongful conviction, the 33 years from the crime to his exoneration represent a block of Swiss cheese. Each human decision made during this period is a slice of cheese, and the holes in each slice represent opportunities for error. Normally, errors like inaccurate arrests are caught by existing checks and balances (prosecutor screening, defense attorney alibis, judicial review). However, in cases like Mr. Hail's, all the holes align, allowing the error to pass through undetected. This leads to outcomes like wrong-site surgery, plane crashes, or wrongful convictions.

While this exposes systemic flaws, it also presents an amazing learning opportunity. By identifying and closing the holes in each layer of the system, future errors can be prevented.

Applying the Swiss Cheese Model to Criminal Justice

The speaker's team applied this blame-free sentinel event review to Mr. Hail's case, involving all agencies that participated. They found no evidence of misconduct or intentional manipulation. Instead, they identified:

• Outdated protocols for evaluating mistaken victim identifications.
• Communication challenges within the crime lab regarding report dissemination.
• Supervisory issues that failed to ensure report delivery.
• Inadequate physical evidence handling protocols by the courts, leading to the loss of crucial evidence due to flooding.

Their analysis revealed 46 different contributing factors and situations, any one of which, if addressed, could have prevented Mr. Hail's wrongful conviction or accelerated his release. These factors led to 29 recommendations for change to close holes in the system.

This approach has been replicated in other "never events" within criminal justice, consistently revealing dozens of contributing factors and leading to numerous recommendations for improvement. The focus remains on putting good people in better situations to succeed, not on assigning blame.

Addressing Intentional Misconduct within the Swiss Cheese Framework

The speaker acknowledges that intentional misconduct does occur in criminal justice. However, the Swiss Cheese Model can still be applied. The question shifts to: "Why did that person think that acting intentionally would work?" and "Why didn't we detect it?" This prompts an upstream analysis of the holes in the system that could be closed to improve detection and deterrence of intentional misconduct.

Dramatic Improvements Through Systemic Change: The Aviation Example

The impact of incorporating sentinel event reviews into a system's culture is dramatic. The graph of fatal aviation crashes per million flights shows a significant reduction over the 50 years since aviation began using these reviews. This is attributed to learning from safety events and continuously closing holes in the Swiss cheese. Concurrently, the number of flights taken has quadrupled, indicating increased public trust due to enhanced safety.

Contrasting with Officer-Involved Shootings

In contrast, officer-involved shootings in the U.S. have not seen similar systemic improvements. The current approach in criminal justice often involves removing officers from the equation, which doesn't equip good officers to handle difficult situations better. The speaker emphasizes the need to change the conditions in which humans operate, rather than trying to change the human condition itself.

The George Floyd Case and the Need for Prevention

The tragic murder of George Floyd is presented as another sentinel event. While accountability reviews and criminal trials have occurred, the speaker questions whether punishment alone is sufficient to prevent future deaths in custody or protect officers from situations where they feel compelled to use deadly force. The question is posed: if sentinel event reviews had been implemented decades ago, could such tragedies have been prevented by closing systemic holes and altering the course of events? This could have potentially averted the widespread protests and anguish stemming from a system that allows similar events to recur.

Conclusion: Prevention Through Sentinel Event Reviews

The speaker concludes by reiterating that both Mr. Hail's wrongful conviction and Mr. Floyd's death were tragedies resulting from a confluence of numerous errors. They express belief that sentinel event reviews can prevent "never events" in criminal justice, including wrongful convictions, officer-involved shootings, and deaths in custody. Quoting Benjamin Franklin, "An ounce of prevention is worth a pound of cure," the speaker advocates for a proactive, systemic approach to improving the criminal justice system.