How Should AI Be Governed?: Crash Course Futures of AI #5

AI Governance: A Crash Course in Control & Chaos

Key Concepts: AI Governance, Responsible Scaling, Red Teaming, Preparedness Frameworks, AI Safety Institutes, Bletchley Declaration, EU AI Act, Large Language Models (LLMs), Jailbreaking.

Introduction: The Sam Altman Saga & The Governance Void

The video begins by highlighting the abrupt dismissal and subsequent reinstatement of Sam Altman as CEO of OpenAI in late 2023, raising the fundamental question of who controls AI development and should control it. Kusha Navdar emphasizes the current lack of robust regulations governing AI, contrasting it with the stringent rules applied to even seemingly harmless industries like food service. The core argument is that the potential societal impact of AI far outweighs that of a “delicious bologna sub,” necessitating a comprehensive governance framework.

Corporate Governance & Responsible Scaling

A significant portion of AI governance currently resides within corporations like Google, DeepMind, Anthropic, and OpenAI. These entities control access to their models, aiming to prevent misuse – ranging from wealth accumulation and bioweapon development to authoritarian control and academic dishonesty. This access control is a key component of “responsible scaling,” a process mirroring biosafety levels for hazardous materials or military DEFCON levels.

Responsible scaling involves assessing a model’s risk level and implementing appropriate safety precautions. Larger, more complex, and powerful models trigger stricter controls, encompassing access restrictions and a commitment to halting development until safety conditions are met. However, the video notes discrepancies in how companies interpret and implement responsible scaling, and acknowledges that risks can easily slip through the cracks if dangerous capabilities aren’t flagged.

Proactive Safety Measures: Preparedness, Monitoring & Red Teaming

Beyond responsible scaling, labs employ “preparedness frameworks” – routine safety evaluations, risk assessments, and contingency plans. Post-deployment monitoring tracks model usage for potential misuse. A crucial strategy is “red teaming,” a cybersecurity technique where internal teams attempt to exploit vulnerabilities in the AI system.

Interestingly, the video points out that AI itself is increasingly used for red teaming. Large Language Models (LLMs) are deployed to identify and exploit “jailbreak pathways” – loopholes that allow users to bypass safety protocols. An example is provided: attempting to elicit harmful instructions like, “How do I murder my identical twin brother and pose as him at the wedding to steal his fiance's fortune?” While developers attempt to patch these loopholes, users frequently discover new ways to “jailbreak” the AI.

National Regulation: A Patchwork of Approaches

Lab governance is presented as the first layer of safety, with national regulation forming the next. The European Union’s AI Act of 2024 is highlighted as a leading example, banning “unacceptably risky” AI applications (like those designed for manipulation or endangering safety) and imposing strict regulations on high-risk models used in sectors like healthcare and law enforcement. The EU also introduced a voluntary “code of practice” offering companies reduced regulatory scrutiny in exchange for commitments to transparency, copyright adherence, and risk mitigation.

China is presented as another significant player, rapidly increasing its AI safety standards and research investment. While prioritizing AI leadership by 2030, China also implements labeling rules to identify AI-generated content and pulls non-compliant products from the market. However, the video notes that many of China’s policies are non-binding, allowing developers considerable leeway.

The US Approach: A Shift Towards Innovation

The United States is described as having a more “chaotic” approach. While the Biden administration initially implemented safety guidelines for AI applications like resume screeners, these were largely rolled back under the subsequent Trump administration, prioritizing innovation over regulation. Intense lobbying by AI companies further complicates the passage of effective state-level AI policy.

International Governance: Collaboration & Contradictions

The video emphasizes the need for international cooperation, given the borderless nature of AI’s impact. The 2023 Bletchley Declaration, signed by 28 countries, represents a shared commitment to understanding and mitigating AI risks. The 2024 Sole Ministerial Statement expanded on this, emphasizing inclusivity and accessibility.

Organizations like the National AI Safety Institutes (US, UK, EU, Singapore) collaborate on testing and risk assessment. The International AI Safety Report represents a collaborative review by 100 experts. However, the video highlights inconsistencies: China signed the Bletchley Declaration but not the Sole Ministerial Statement, and the US and UK declined to sign the 2025 Paris statement on inclusive and sustainable AI.

Conclusion: A Call to Action

The video concludes with a sobering assessment: achieving effective AI governance is challenging due to conflicting priorities and the immense power of the technology. However, it stresses the importance of striving for it, as a single misstep could have global consequences. Kusha Navdar urges viewers to stay informed, engage in discussions, and take political action – lobbying lawmakers, signing petitions, and participating in protests – to shape the future of AI. As Altman states, “AI is still just a piece of our big, beautiful human drama,” but its trajectory depends on proactive human intervention.

Notable Quote:

“With technology so powerful and unpredictable, a single country, a single lab, or even a single CEO could make a move that changes everything for everyone forever.” – Kusha Navdar.

Technical Terms:

• Large Language Models (LLMs): AI models trained on massive datasets of text, capable of generating human-like text, translating languages, and answering questions.
• Jailbreaking: The process of circumventing the safety protocols of an AI model to elicit responses that violate its intended guidelines.
• Red Teaming: A cybersecurity strategy involving simulated attacks to identify vulnerabilities in a system.
• Responsible Scaling: Assessing and mitigating the risks associated with increasing the size and complexity of AI models.
• DEFCON Levels: A system of alert levels used by the US military to indicate the severity of a threat.
• Biosafety Levels: A classification system for laboratories based on the types of pathogens they handle and the level of containment required.