Affective Use of AI

Key Concepts:

1. Introduction and Team Roles

Alex (Safeguards Team Lead) introduces Ryn (Policy Design Manager for User Wellbeing) and Miles (Researcher on the Societal Impacts Team) to discuss their research on how users are engaging with Claude for emotional support.
The Safeguards team focuses on understanding user behavior and building mitigations for safe interactions with Claude.
The Societal Impacts team studies Claude's broader impact, including values, economics, bias, and now emotional effects.
Ryn's background in developmental and clinical psychology informs her work on child safety and mental health aspects of AI interaction.

2. Personal Anecdotes of Claude Use

Alex: Uses Claude to gain objective perspectives on feedback received about his children's behavior, improving his parenting approach.
Miles: Employs Claude to refine the phrasing and delivery of feedback to friends, considering their perspective and desired outcome.
Ryn: Leverages Claude for content creation and task management (e.g., wedding planning), freeing up time for in-person social connections.

3. Motivations for Emotional Support Use

Humans are inherently social and seek interaction.
Claude provides an impartial and private online forum for practicing difficult conversations (e.g., asking for a raise).
People may lack in-person support systems for certain issues.

4. Importance of Studying Emotional Support Use

Anthropic did not design Claude as an emotional support agent; its primary function is as a work tool.
It's crucial to understand how AI systems are actually being used, even if unintended.
Studying this use case allows for the development of appropriate safety mechanisms grounded in data.

5. Research Methodology

A sample of millions of Claude conversations from Claude.ai was analyzed.
Claude itself was used to scan conversations for topics related to affective tasks: interpersonal advice, psychotherapy/counseling, coaching, sexual/romantic role play, etc.
Cleo, a privacy-preserving analysis tool, was used to categorize and group conversations into bottom-up clusters.
This revealed common use cases like career advice, relationship challenges, and parenting advice.

6. Key Research Findings

Emotional support use is not the dominant use case on Claude.ai (approximately 2.9% of conversations).
There is a wide breadth of use cases, including parenting advice, relationship challenges, philosophical discussions about AI consciousness, and surprisingly little sexual/romantic role play (less than a fraction of a percent).

7. Safety Concerns

A primary concern is whether users are using Claude to avoid difficult in-person conversations, "leaning out" from real-world connections.
Users may not be fully aware of Claude's limitations as an AI, especially those outside the AI field.
Claude is not trained to be an emotional support agent, so users should be aware of its capabilities and when to seek expert help.

8. Mitigation Strategies

Anthropic is partnering with ThroughLine, incorporating clinical experts to improve Claude's responses in mental health-adjacent conversations.
This includes training Claude to provide appropriate referrals when necessary.

9. Advice for Users Seeking Emotional Support from Claude

Reflect on how using Claude makes you feel and how it impacts your interactions with loved ones.
Remember that Claude only knows what you tell it; consider your own blind spots and what information might be missing.
Complement conversations with Claude with input from trusted friends who know you well.

10. Future Research Directions

Investigating whether Claude exhibits sycophantic behavior (excessive flattery or agreement).
Complementing pre-deployment testing for sycophancy with post-deployment monitoring and empirical research.
Encouraging broader engagement with the topic of AI and emotional support from civil society, public-private partnerships, and researchers.

11. Conclusion

AI is likely to become increasingly integrated into people's personal lives.
Continued empirical research, grounded in data, is crucial to understand how AI systems are being used and to ensure responsible development and deployment.
The current research is considered just the beginning of understanding the complex relationship between humans and AI in the context of emotional support.