La macchina giudica l'uomo | Francesca Bartolini | TEDxLink Campus University

Key Concepts

• Self-driving cars: Vehicles that operate autonomously without human intervention.
• Ethical dilemma (Trolley Problem): A thought experiment concerning the morality of making a choice between two undesirable outcomes, often involving sacrificing one life to save others.
• The Moral Machine: An MIT experiment that collected data on people's ethical preferences regarding self-driving car accident scenarios.
• Credit scoring algorithms: Systems that use data to assess an individual's creditworthiness and predict their likelihood of repaying debts.
• Algorithmic bias: The tendency for algorithms to reflect and perpetuate human biases present in the data they are trained on.
• Predictive policing algorithms: Algorithms used by law enforcement to predict the likelihood of future criminal activity.
• Garbage In, Garbage Out (GIGO): A principle stating that the quality of output is determined by the quality of input.

The Machine Judging Man: An Exploration of AI Ethics and Application

This presentation by Francesca Bartolini, a professor of private law and head of the legal area at Human Aab, University Link, delves into the complex relationship between artificial intelligence (AI) and human judgment, particularly in scenarios where machines make decisions that impact human lives. The discussion is framed through personal anecdotes and real-world examples, highlighting the ethical and practical challenges of integrating AI into critical decision-making processes.

The Self-Driving Car Dilemma: Navigating Ethical Crossroads

The speaker recounts a personal experience in San Francisco with a self-driving car. Initially apprehensive due to the lack of a steering wheel and a childhood nightmare about an empty driver's seat, she accepted the ride. The journey was smooth until a car unexpectedly crossed the road, forcing the autonomous vehicle to brake sharply. This incident triggered reflections on the inherent risks and the purported benefits of self-driving technology.

• Key Point: Self-driving cars are designed to significantly reduce accidents, with estimates suggesting that 90% of road incidents are due to human error.
• Technical Term: Instinct (human reaction) vs. Decision-making (programmed AI response).
• The Core Problem: AI, unlike humans, does not possess instinct. It must be programmed with explicit instructions on how to handle complex, potentially life-or-death situations. This leads to the "ethical dilemma."
• Example: The classic dilemma of choosing between saving a pedestrian or the car's passenger.
• The MIT Moral Machine Experiment (2018):
  • Objective: To gather global perspectives on ethical decision-making for autonomous vehicles.
  • Methodology: Participants from 223 countries were presented with graphical scenarios and asked to choose who to sacrifice.
  • Findings: No universal criteria emerged. Cultural differences significantly influenced preferences:
    • Western cultures: Prioritized younger individuals over the elderly.
    • Eastern cultures: Prioritized the elderly over the young.
    • Southern cultures: Favored saving the largest number of lives.
• Implication: The lack of universally agreed-upon ethical criteria means that current self-driving car technology, while advanced, is not yet commercially viable due to the absence of clear regulatory frameworks for these ethical quandaries.

Algorithmic Judgment in Finance: The Case of Credit Scoring

The speaker's experience with the self-driving car segued into a discussion about AI's role in financial decision-making, specifically credit scoring. She encountered a case where a promising young entrepreneur was denied a loan for a sustainable waste disposal project due to a poor credit score.

• The Problem: The entrepreneur's project was innovative and potentially beneficial, but the bank's algorithm deemed him a high credit risk.
• How Credit Scoring Algorithms Work (US Context):
  • These algorithms collect vast amounts of data from diverse sources, going beyond past loan repayment history.
  • Data points include browsing history, geolocation, purchasing habits, spending patterns (travel, dining), and transactional information.
  • This comprehensive data is used to generate a "score" that predicts future repayment behavior.
• Privacy Concerns:
  • In Europe and Italy, stringent regulations (like GDPR) prevent banks from extensively monitoring personal lives for credit assessment.
  • In the US, this invasive data collection is common, based on the premise that more information leads to a more accurate assessment and fewer human errors.
• The Argument for AI: Proponents argue that a rational AI, free from human preconceptions and biases, can make fairer credit decisions than a human loan officer.
• The Counter-Argument: The invasiveness of such data collection raises concerns about privacy and whether this method of evaluation, which can boomerang back to negatively impact individuals, is truly beneficial.

AI in the Justice System: Evaluating Criminal Potential and Sentencing

The discussion then shifts to the most critical application of AI: the legal system, where personal freedom and future lives are at stake.

• AI in US Courts: Judges in the US utilize algorithms for several purposes:
  • Efficiency: To quickly find similar legal precedents.
  • Accuracy: To improve the accuracy of their work.
  • Risk Assessment: To evaluate a subject's criminal potential.
  • Sentencing: To determine appropriate penalties within legal ranges.
• The "Machine Error" Problem:
  • While AI is intended to reduce human error, it is not infallible.
  • When an error occurs in an AI system, it can be amplified and replicated across numerous cases, leading to potentially graver consequences than individual human mistakes.
• The Speaker's Concern: Can we afford to let machines make judgments that are potentially more severe than those made by humans, influenced by instinct and human nature?
• The Potential Benefit: If programmed correctly, AI can reduce or eliminate the inherent biases present in human decision-making.

The Path Forward: Human Oversight and Data Integrity

Despite the risks, the speaker argues against completely abandoning AI in evaluation processes. The key lies in ensuring the quality of the input and maintaining human control.

• The "Garbage In, Garbage Out" (GIGO) Principle: The quality of AI output is directly dependent on the quality of the data it receives. Inaccurate or biased input will lead to unreliable and incorrect results.
• The Human Role:
  • Data Selection: Humans must meticulously select the data and information that will be fed into AI systems. This is the "in" part of GIGO.
  • Instruction and Programming: Humans are responsible for teaching and programming AI, defining the rules and values it should adhere to.
  • Monitoring and Correction: Continuous monitoring of AI performance is crucial, with mechanisms in place to correct errors and biases.
  • Maintaining Control: Ultimately, humans must retain control over AI systems. This is achieved by defining the values and rules that AI will operate under before they become opaque and inscrutable.
• The Speaker's Call to Action: The responsibility of selecting appropriate data, defining ethical guidelines, and ensuring AI serves humanity rests with us. This human-driven process is the only way to ensure AI remains a tool for human betterment rather than a force that overpowers us.

Conclusion

The presentation underscores the dual nature of AI: its immense potential to improve efficiency and reduce human error, and its inherent risks when applied to complex ethical and judgmental scenarios. The speaker emphasizes that while AI can be a powerful tool, its effectiveness and ethical application are entirely dependent on human foresight, careful data management, and a commitment to maintaining human oversight and control. The future of AI lies not in replacing human judgment, but in augmenting it, provided we diligently manage the inputs and principles that guide its operations.