Why are driverless cars still so bad at driving? | Jennifer Dukarski | TEDxDetroit

Key Concepts

• Inclusion by Design: A design philosophy that prioritizes the needs and experiences of all users, particularly marginalized or underserved communities, from the outset of development.
• Autonomous Vehicles (AVs): Vehicles capable of sensing their environment and operating without human involvement.
• AI Engine: The core artificial intelligence system that processes data and makes decisions for AVs.
• Data Sets: Collections of information used to train AI models.
• Bias in AI: The tendency for AI systems to reflect and perpetuate existing societal biases present in their training data.
• Vision Systems: The sensors and software that enable AVs to "see" and interpret their surroundings.
• Unintended Consequences: Outcomes of a technology or system that were not foreseen or planned for.

Summary

The speaker, with a background in automotive engineering and law, expresses a deep passion for driverless cars, not just for their technological advancement but for their potential to create a more inclusive future for the disabled, elderly, and underserved communities. This vision was solidified by an encounter with the Nevada delegation and the story of Sam Schmidt, a paralyzed race car driver who was among the first to be licensed to drive an adapted vehicle.

Challenges in Current AV Technology

Despite the rapid advancement of AVs, which are described as "computers on wheels," significant challenges remain. The speaker uses Tesla's Full Self-Driving (FSD) system as an example to illustrate these issues.

• Data Collection and User Demographics: Tesla FSD relies on data collected by drivers. However, the demographics of typical Tesla drivers (high income, predominantly male, white, and without children at home) suggest a potentially limited and biased data set.
• Lending Tree Statistics: Recent statistics from Lending Tree indicate that Tesla drivers are more likely to be involved in vehicle incidents and crashes per thousand vehicles, raising questions about the system's real-world performance across diverse driving conditions and user groups.
• Bias in Vision Systems: The speaker highlights research demonstrating inherent biases in AV vision systems.
  • Georgia Tech Study (2019): Found a nearly 10% difference in the ability of vision systems to detect dark-skinned individuals compared to light-skinned individuals.
  • King's College London Study (2023): While improvements were noted (a 7% difference), this study also revealed that AV systems are 20% more likely to detect adults than children.

The Imperative of Inclusion by Design

These findings underscore the critical need for "inclusion by design" in AV development, drawing a parallel to the "privacy by design" approach in the privacy realm. Inclusion by design means integrating the needs of all users from the very beginning, rather than as an afterthought.

Illustrative Stories of Inclusion by Design

The speaker presents three stories to illustrate the principles of inclusion by design:

1. The Wheelchair and the Duck (2016): Google's autonomous vehicle successfully identified and stopped for both an electric wheelchair with a person and a duck. This demonstrates the system's ability to recognize and prioritize different entities, including people, regardless of their mode of transport. The takeaway is the need for AI and data sets that can identify all people irrespective of gender, age, or race.

2. Honking Cars in San Francisco: Driverless vehicles in San Francisco, when congregating in parking lots, engage in a behavior that leads to prolonged honking, disrupting nearby residential communities. This highlights the importance of considering the bigger picture and the unintended consequences of AV behavior on surrounding communities, even if the vision system itself doesn't directly perceive them.

3. The Taco Truck in San Francisco (2017): An autonomous vehicle detected a taco truck and stopped, but then became unable to resume its journey due to continuous pedestrian activity. The safety driver had to intervene. This story emphasizes the necessity of considering location, surrounding communities, and interactions (both suburban and urban) in the design process.

Moving Towards Total Inclusion by Design

The speaker concludes by advocating for a shift towards total inclusion by design. This requires:

• Education: Teaching inclusion by design principles in engineering schools.
• Standards Bodies: Consideration and adoption of these principles by standards organizations.
• Regulation: Proactive thinking and action from regulators to ensure truly inclusive mobility.

By embracing inclusion by design, remembering the lessons from the wheelchair and duck, honking cars, and the taco truck, the vision of truly inclusive mobility can be realized.