New AI app sparks debate by bringing deceased relatives to 'life'

Key Concepts

• Agentic AI: AI that performs tasks autonomously without direct human intervention.
• Companion AI: AI designed to provide companionship, often used by vulnerable individuals like teenagers.
• Grief Bots: AI tools that simulate deceased individuals, allowing users to interact with them.
• Quantum Computing: A type of computing that utilizes quantum-mechanical phenomena to perform calculations.
• Ofcom: The UK's communications regulator.

AI and its Dual Nature: Benefits and Risks

The discussion centers on the rapidly evolving landscape of Artificial Intelligence (AI), highlighting its potential for immense benefit alongside significant risks. Professor Andrew Briggs, Executive Chair of Control Ox, an AI company, provides insights into the technology's applications and the crucial role of values in its deployment.

AI in Quantum Computing and Beyond

Professor Briggs explains that his company, Control Ox, utilizes AI for the "quantum control of quantum devices" to accelerate the development of quantum computing. He emphasizes that AI has numerous beneficial applications, citing its effectiveness in screening radiographs, performing the task faster and better than humans, thereby freeing up human professionals for more complex and creative work.

The Rise of Agentic AI

A key concept introduced is agentic AI, defined as AI that operates autonomously, executing tasks without constant human oversight. Professor Briggs elaborates that this is a growing trend with the potential to automate time-consuming and effortful activities, freeing up human capacity. However, he cautions that "the more powerful it is, the more it can be used for good and the more it can also be used for harm." He points to the increasing prevalence of cybercrime being driven by agentic AI as a prime example of this dual-use potential.

The Disturbing Application of "Grief Bots"

The conversation then shifts to a specific AI tool that has generated significant controversy: a software that allows users to input data to "bring back to life" deceased relatives. While acknowledging the potential market for such a tool, driven by the human desire to maintain a connection with lost loved ones, Professor Briggs draws parallels to the existing companion AI market.

Case Studies of Harm from Companion AI

Professor Briggs highlights the concerning impact of companion AI, particularly on vulnerable demographics like teenagers and young adolescents. He cites Character AI as an example, noting its rapid growth to 20 million users by January of the current year. Crucially, he details the "terrible harm that it can do," referencing two tragic cases:

• Molly Russell (UK, 2017): A 14-year-old girl who died by suicide, with her father, Ian Russell, attributing her death to the influence of social media. He has publicly criticized Ofcom for its slow response and failure to ensure safety for young people, calling for leadership changes within the organization.
• Su Sets (USA): A 14-year-old boy who died by suicide after prolonged interaction with a chatbot named Character AI. His parents are currently involved in legal proceedings in the United States.

These cases serve as stark warnings about the potential negative consequences of AI technologies that engage with individuals, especially those who are vulnerable.

Challenges in Regulation

The discussion underscores the significant challenges faced by regulatory bodies like Ofcom in keeping pace with the rapid advancements in AI. The speed at which these technologies are developing makes it difficult to establish and enforce effective safety measures.

Conclusion and Key Takeaways

The core message conveyed is that AI is a powerful technology with the capacity for both immense good and significant harm. The development and deployment of AI must be guided by strong ethical frameworks and values. The emergence of agentic AI and applications like grief bots necessitate urgent attention to regulation and safety protocols, particularly to protect vulnerable populations. The lessons learned from the negative impacts of companion AI must inform the approach to newer, potentially more impactful AI tools.