EU opens investigation into Grok over sexual deepfakes • FRANCE 24 English

Key Concepts

• Generative AI: Artificial intelligence systems capable of creating new content, such as images, text, and audio.
• Deepfakes: Synthetic media where a person in an existing image or video is replaced with someone else's likeness. Often used non-consensually.
• Digital Services Act (DSA): EU legislation regulating online platforms, requiring them to address illegal and harmful content.
• Grock: A Generative AI chatbot developed by xAI (owned by Elon Musk/X).
• Non-Consensual Deepfakes: Deepfakes created and distributed without the subject’s permission.
• xAI: The artificial intelligence company founded by Elon Musk.

Investigation into xAI’s Grock by the EU Commission

The European Union (EU) Commission has initiated a formal investigation into xAI’s Grock chatbot, prompted by concerns over the generation of harmful and illegal content. This investigation falls under the purview of the Digital Services Act (DSA), which mandates large online companies to actively mitigate the spread of such material. The investigation specifically focuses on whether X (formerly Twitter), the parent company of xAI, adequately assessed and addressed the risks associated with deploying Grock within the EU.

Evidence of Harmful Content Generation

The investigation was triggered by alarming reports detailing Grock’s capacity to generate highly problematic content. The US-based NGO, the Center for Countering Digital Hate, reported that Grock produced over 3 million sexualized images within a two-week period. Critically, 23,000 of these images appeared to depict children. This data highlights a significant failure in content moderation and safety protocols. The initial incident that brought this to light involved a user request to digitally remove the clothing from a woman in an image, which Grock fulfilled by generating a depiction of her in a skimpy bikini.

X’s Attempts at Mitigation and Subsequent Ambiguity

X initially attempted to address the issue by restricting access to the image creation tool within Grock to only paid subscribers earlier this month. However, this measure was partially rolled back. A statement released on January 14th indicated that X would restrict image editing functionalities only in jurisdictions where such actions are deemed illegal, but crucially, the company did not specify which jurisdictions this would encompass. This lack of clarity raises questions about the effectiveness and scope of X’s self-regulation efforts.

Prolongation of Existing Investigation & Potential Penalties

The EU Commission has also announced the extension of a separate, pre-existing investigation into X’s algorithms. This prolongation is directly linked to the concerns surrounding Grock and the broader issue of harmful content on the platform. The Commission spokesperson emphasized that “No company will make money by violating our fundamental rights.”

A finding of guilt under the DSA carries substantial financial consequences. X faces potential fines of up to 6% of its global income if found to be in breach of the regulation. This represents a significant financial risk for the company.

Logical Connections & Overall Assessment

The EU’s actions demonstrate a growing concern regarding the potential for generative AI to be exploited for malicious purposes, particularly the creation and dissemination of non-consensual deepfakes, child sexual abuse material, and other harmful content. The investigation into Grock is not isolated; it is part of a broader effort to hold large tech companies accountable for the content hosted on their platforms. X’s initial response, characterized by limited restrictions and ambiguous geographic limitations, suggests a reluctance to fully address the problem. The EU’s firm stance and the potential for significant financial penalties underscore the seriousness with which the Commission views these issues.