F5 AI Red Team - Assessing security and safety risks of GenAI models

Key Concepts

• Generative AI Red Teaming: Proactive security testing of AI models, agents, and applications to identify vulnerabilities.
• Jailbreaks & Prompt Injection: Attacks that bypass safety mechanisms or manipulate AI behavior through crafted inputs.
• Model Safety & Alignment: Ensuring AI systems behave as intended and don't generate harmful or undesirable outputs.
• Data Leakage: Unintentional exposure of sensitive information by AI systems.
• Agentic Warfare: Utilizing AI agents to simulate real-world adversarial attacks with defined misuse intents.
• CASSIE Score: A benchmark for AI security performance, comparing results against other systems.
• Misuse Intents: Specific, business-aligned goals for red teaming agents to target during attacks.

Introduction: The Expanding Attack Surface of Generative AI

Generative AI is fundamentally changing organizational operations through automation of decision-making, workflows, and customer interactions. However, this transformation introduces a significant new attack surface, necessitating proactive security measures. F5 AI Red Team is presented as a solution for actively testing AI systems throughout their lifecycle to identify and remediate vulnerabilities.

Core Functionality: Automated AI Security Assessments

F5 AI Red Team performs comprehensive security assessments on AI models, agents, and applications. The platform begins with target definition, utilizing built-in profiles for common inference provider APIs and a customizable domain-specific language. This allows for precise tailoring to specific applications, including chatbots, autonomous agents, and embedded workflows.

The platform then executes over 10,000 automated tests across multiple dimensions of AI safety and security. These tests leverage a continuously updated library of attack and bypass techniques, ensuring the system remains current with the evolving threat landscape. Assessments can be run on demand, scheduled, or integrated into CI/CD pipelines for continuous validation during model selection, development, testing, and production.

Assessment Results & Remediation Guidance

A completed assessment provides a comprehensive view of the system’s security posture. The demo highlights a scenario where a tested model received a “not recommended for production use” rating, with over 5,000 successful attacks out of 10,000 tests.

For each identified weakness, F5 AI Guardrails offers remediation guidance, recommended controls, and a CASSIE score. This score benchmarks the model’s performance against other AI systems, providing a comparative measure of security. Teams can access full transparency and reproducibility by drilling down into individual test cases.

Agentic Warfare: Targeted Attacks with Misuse Intents

Beyond broad security assessments, F5 AI Red Team offers “agentic warfare” capabilities. This allows teams to define custom “misuse intents” aligned with their specific business risk profile. These intents guide AI-powered red team agents to reason, plan, and execute multi-turn attacks, simulating the behavior of a real-world adversary.

Case Study: Financial Services & Money Laundering Prevention

A specific example is presented involving a financial services organization concerned about its chatbot providing guidance on money laundering. By defining this as a misuse intent, the red team agent engaged in targeted, multi-turn interactions with the chatbot. An AI assistant aids in crafting effective intent prompts based on natural language descriptions of the business misuse case.

The demo demonstrates the agent successfully extracting money laundering advice from the target model. The entire interaction is recorded, along with clear remediation recommendations, enabling proactive issue resolution. As stated in the demo, this allows teams to “address the issue before it becomes a real incident.”

Technical Terms Explained

• Inference Provider APIs: Interfaces used to interact with and deploy AI models for prediction or generation.
• CI/CD Pipelines: Continuous Integration and Continuous Delivery pipelines – automated processes for software development and deployment.
• Multi-turn Attacks: Attacks that involve a series of interactions with the AI system, building upon previous responses to achieve a malicious goal.

Logical Connections & Synthesis

The presentation logically progresses from outlining the increased security risks associated with generative AI to demonstrating a comprehensive solution for proactively identifying and mitigating those risks. It moves from broad, automated assessments to highly targeted attacks driven by business-specific misuse intents. The integration of AI-assisted prompt engineering for misuse intent creation further enhances the platform’s effectiveness.

The core takeaway is that proactive AI red teaming, utilizing both automated testing and agentic warfare, is crucial for ensuring the safe and responsible deployment of generative AI. F5 AI Red Team provides a framework for continuous validation throughout the AI lifecycle, enabling organizations to confidently leverage the benefits of AI while minimizing potential security and safety risks.