Course Overview - AI Security

Key Concepts:

• AI Security
• AI Life Cycle Vulnerabilities (Training Data Poisoning, Prompt Injection, Jailbreaks, Hallucinations, Adversarial Examples, Inference Time Threats)
• AI System Architectures (including Multi-Agent Setups)
• Large Language Models (LLMs)
• Deep Fakes
• Resilience
• Modern Threats

AI Security: An Overview

The increasing power and pervasiveness of AI in organizations and daily workflows elevates the importance of mitigating security risks. Understanding these risks and defenses is crucial. Neil Dwani, co-academic director of the Stanford online advanced cyber security program, introduces an AI security course designed to explore potential risks in the design, build, and maintenance of AI systems. The course aims to equip participants with the knowledge to make AI systems more secure and resilient against modern threats.

Vulnerabilities Across the AI Life Cycle

The course will cover the evaluation of vulnerabilities throughout the AI life cycle. Specific threats to be examined include:

• Training Data Poisoning: Attacks that compromise the integrity of the data used to train AI models, leading to biased or malicious outputs.
• Prompt Injection: Exploiting vulnerabilities in LLMs by crafting prompts that manipulate the model's behavior to bypass intended restrictions or reveal sensitive information.
• Jailbreaks: Techniques used to circumvent safety mechanisms and ethical guidelines implemented in AI systems, allowing users to generate harmful or inappropriate content.
• Hallucinations: Instances where AI models generate outputs that are factually incorrect, nonsensical, or unrelated to the input, potentially leading to misinformation or unreliable decision-making.
• Adversarial Examples: Inputs specifically designed to cause AI models to make incorrect predictions or classifications, often imperceptible to humans but capable of disrupting critical systems.
• Inference Time Threats: Attacks that occur during the deployment and use of AI models, such as model inversion or membership inference, which can expose sensitive data or compromise model integrity.

AI System Architectures and Breaches

The course will delve into the architectures of modern AI systems, including multi-agent setups. It will also examine high-profile breaches involving large language models and deep fakes. The course poses the question of how far these technologies can be pushed.

Expert Guidance and Skill Development

The course is guided by world-renowned faculty and industry experts from leading companies. It aims to equip participants with the knowledge and skills needed to safeguard their organizations. The course addresses the future of AI and the need to build around it.

Conclusion

The AI security course aims to provide participants with the critical skills needed to fortify AI systems and empower their resilience against evolving threats.