Who's in control of AI? | Inside Story

Key Concepts

• Mythos: A highly advanced, restricted AI model developed by Anthropic, capable of identifying and exploiting critical cybersecurity vulnerabilities.
• Agentic AI: AI systems capable of autonomous action, decision-making, and coordination at superhuman speeds.
• Sovereign AI: The concept of nations building their own domestic AI infrastructure to reduce reliance on US or Chinese tech giants.
• AGI (Artificial General Intelligence): AI that possesses the ability to understand, learn, and apply knowledge across a wide variety of tasks at a level equal to or exceeding human capability.
• Regulatory Capture: The phenomenon where regulatory frameworks are influenced or controlled by the industries they are intended to oversee.

1. Main Topics and Key Points

• The Mythos Breach: Anthropic recently reported unauthorized access to its "Mythos" model. While no malicious activity was confirmed, the incident highlighted the extreme risks associated with models that can autonomously navigate and exploit global banking and utility infrastructure.
• The AI Arms Race: The industry is characterized by a fierce competition between US-based giants (Anthropic, OpenAI, Google) and Chinese developers. This competition is driving rapid, often unchecked, development.
• Global Governance Divide:
  • United States: Current policy favors minimal regulation to maintain global dominance, with the Trump administration actively discouraging state-level restrictions.
  • China: Proposes a global AI cooperation organization (headquartered in Shanghai) and has implemented strict domestic regulations requiring the registration of all Large Language Models (LLMs).
  • European Union: Implementing the world’s first comprehensive AI law, though critics argue it may inadvertently stifle local innovation while being easily absorbed by large multinationals.

2. Real-World Applications and Risks

• Cybersecurity: Mythos can identify flaws in 27-year-old legacy software used in critical infrastructure. While this allows for "patching," it simultaneously provides a blueprint for unprecedented cyberattacks.
• Economic and Social Impact: Experts warn of rising utility costs due to the massive energy consumption of data centers (projected to rival the energy usage of entire nations like India within a decade) and the potential for AI to exacerbate economic inequality.

3. Methodologies and Frameworks

• The "Sandbox" Approach: Anthropic keeps Mythos in a restricted, controlled environment, providing access only to select partners (e.g., JP Morgan Chase, Apple) to allow them to fortify their defenses before a wider release.
• Open Source vs. Closed Models: China is increasingly utilizing open-source models to gain influence in the Global South, contrasting with the "closed" proprietary models favored by US firms.

4. Key Arguments and Perspectives

• The "Doomsday" vs. "Practical" Debate: While some fear AGI-driven existential threats, experts like Adrien Monk argue that the immediate, tangible risks—such as privacy erosion, data theft, and the loss of human agency—are more pressing.
• The "Shakespearean Drama": Sam Altman (OpenAI) describes the industry as being in a state of intense, high-stakes competition where the pursuit of AGI creates a "power dynamic" that leads to erratic behavior among leadership.
• Democratic Accountability: Romesh Shrenasan argues that AI development is currently removed from democratic oversight, functioning as a "powerhouse force" controlled by a few CEOs rather than the public.

5. Notable Quotes

• Sam Altman (via report): "The fear and anxiety about AI is justified. We're in the process of witnessing the largest change to society in a long time and perhaps ever."
• Anthropic CEO (via report): Described their models as a "country of geniuses in a data center... a set of AI agents that are more capable than most humans at most things."
• Adrien Monk: "We've seen with the European approach that actually what it sometimes ends up doing is creating more problems for its own companies... we're talking about the problems of 2021, not 2026."

6. Synthesis and Conclusion

The rapid evolution of AI, particularly "Agentic AI," has outpaced current regulatory frameworks. The consensus among the panel is that a singular, non-proliferation-style treaty is unlikely to succeed due to the speed of innovation and geopolitical tensions. Instead, the future likely involves a fragmented landscape where nations pursue "Sovereign AI" to maintain control over their own digital infrastructure. The primary challenge remains balancing the immense potential for innovation and cyber-defense against the risks of economic instability, environmental strain, and the concentration of power in the hands of a few private corporations.