How do we keep big tech in line? || Insiders: On Background

Key Concepts:

• Child Sexual Abuse Material (CSAM)
• E Safety Commissioner
• Transparency Reports
• Neural Hash
• Content API
• Trust and Safety Personnel
• Deep Fake Pornography
• Image-Based Abuse
• AI Regulation
• Social Media Minimum Age Bill
• Age Assurance Technologies
• Online Safety Codes

1. Lack of Transparency from Tech Giants:

• The E Safety Commissioner's report reveals that Google and Apple are unable to provide basic data on CSAM complaints received on their platforms, response times, and staffing dedicated to platform safety.
• Julie Inman Grant, the E Safety Commissioner, criticizes the tech giants for not being able to answer fundamental questions about their efforts to combat child abuse, despite collecting vast amounts of user data.
• The report covers eight major technology companies, including Microsoft, Meta (WhatsApp, Facebook), and Snap.

2. Technology and its Underutilization:

• Established technologies like photo DNA have been available for over 15 years to identify known CSAM.
• Apple's neural hash and Google's content API can identify CSAM in a privacy-preserving manner by creating numerical values that can be matched against databases like those held by the National Center for Missing Exploited Children.
• These technologies are not being fully utilized across all services, including YouTube, iCloud, iMessage, and live streaming platforms.
• AI offers new technologies for scaling detection with greater accuracy, but companies are choosing not to implement them fully.

3. Reasons for Underutilization:

• Potential added expense and slowing down of live streaming are cited as possible reasons.
• A major concern is the lack of transparency regarding the number of trust and safety personnel employed by these companies.
• Following Elon Musk's acquisition of Twitter (now X), there was a significant reduction (up to 80%) in trust and safety staff, which other companies quietly followed.
• The New York Times reported that YouTube rolled back content moderation policies to allow more harmful content on the platform.

4. Positive Examples and Progress:

• Some smaller companies like Discord and Snap are using language analysis technologies to detect grooming and sexual extortion.
• The E Safety Commissioner's transparency reports led to Microsoft deprecating Skype after it was found to have a 19-day response time to CSAM reports. Skype was a major vector for live stream child sexual abuse material, particularly in markets like the Philippines and Indonesia.

5. New Powers and Enforcement:

• The E Safety Commissioner now has powers to enforce codes and standards, with potential fines of up to $49.5 million for violations under the Social Media Minimum Age Bill.
• An enforcement task force has been established to target non-compliant platforms.
• The need for global regulatory coherence and partnership is emphasized, as Australia is a relatively small regulator compared to others like Ofcom in the UK.

6. AI and Deep Fake Pornography:

• AI is contributing to the proliferation of deep fake pornography and child abuse material online.
• The technology to create deep fakes has become easily accessible, allowing teenagers to create credible deep fakes of classmates using free apps.
• Deep fake image-based abuse is occurring approximately once a week in schools across Australia.
• The E Safety Commissioner has sent letters to education ministers and developed a deep fake incident management plan for schools.
• The image-based abuse scheme has a 98% success rate in getting content taken down.

7. AI Regulation and the Future:

• The initial "AI drag race" in 2023, focused on speed to market, raised concerns about safety.
• There is a debate about AI sovereignty and the need for appropriate guard rails without undermining innovation.
• The E Safety Commissioner advocates for harnessing AI for safety by design, including mindful data scraping, training, and testing.
• The Productivity Commission suggests right-sized regulation as a last resort, using existing regulations where possible.
• The E Safety Commissioner is assessing industry-developed codes for AI chatbots and will impose standards if they do not provide adequate safeguards.
• The US Take It Down Act replicates Australia's image-based abuse scheme and deep fake initiatives.

8. Social Media Ban for Under 16s:

• A technical trial is underway to implement the social media ban for under 16s, considering privacy, security, and safety.
• Age assurance technologies, many with AI components, are being tested.
• The E Safety Commissioner has been in discussions with major platforms about implementing the Social Media Minimum Age Bill.

9. Criticism and Response:

• Shadow Minister for Communications Melissa Macintosh raised concerns about the E Safety Commissioner's remit and called for an investigation.
• The E Safety Commissioner responded by stating that the online safety codes were developed by the industry and that she is simply implementing powers given to her by parliament.

10. Conclusion:

The interview highlights the challenges in regulating big tech companies and protecting children from online harm. While technologies exist to combat CSAM and deep fakes, their implementation is inconsistent. The E Safety Commissioner is pushing for greater transparency, enforcement, and global cooperation to address these issues. The discussion also touches on the broader debate around AI regulation, emphasizing the need for a balanced approach that encourages innovation while mitigating risks.