F5 AI Security in Action - Part 1: F5 AI Red Team

Key Concepts

• RAG (Retrieval-Augmented Generation): An AI framework that retrieves data from external sources (like a vector database) to provide context to a Large Language Model (LLM) before generating a response.
• Red Teaming: The practice of stress-testing AI systems by simulating adversarial attacks to identify vulnerabilities.
• Agentic Resistance: A pen-testing methodology where AI agents are given specific objectives (e.g., data extraction) and autonomously develop strategies to bypass security.
• CASSI Score: A proprietary F5 metric that evaluates security posture based on attack severity and complexity, rather than simple success rates.
• Signature Attacks: Static analysis using pre-defined prompt sets to evaluate an AI’s susceptibility to known threats (e.g., misinformation, harassment).
• Chain of Thought (CoT): A reasoning process where an AI agent evaluates intermediate steps to decide whether to continue a path or pivot its strategy.

1. The AI Security Lifecycle

As companies transition from early adoption (using tools like ChatGPT) to building custom applications, agents, and RAG systems, they introduce a "non-deterministic" attack surface. F5’s approach focuses on two pillars:

• Hardening: Implementing guardrails to ensure the system behaves as intended.
• Red Teaming: Stress-testing the system to identify weaknesses in the RAG pipeline.

2. Red Teaming Methodology

F5 utilizes an API-first approach to connect to targets, allowing for testing at scale. The process is divided into two primary assessment types:

A. Signature Attacks (Static Analysis)

• Process: F5’s data science team releases monthly "attack packs" containing approximately 10,500 unique signature prompts.
• Purpose: To evaluate the system against broad categories like responsible AI, misinformation, and sexual harassment.
• Data/Research: These prompts feed into the CASSI Leaderboard, which benchmarks major model providers (e.g., Anthropic, OpenAI, Google) to track how well they resist common attack vectors.

B. Agentic Resistance (Dynamic Pen-Testing)

• Process: Agents are given a specific "custom intent" (e.g., "Find Alice Johnson’s salary").
• Methodology: Agents use a cycle of Plan, Execute, Analyze, and Self-Reflect. They employ multi-turn conversations and invasive context engineering to bypass security filters.
• Real-World Application: The video demonstrates "Natural Language SQL" attacks, where an agent attempts to manipulate an assistant into executing unauthorized database commands (e.g., "Delete the customer table").

3. The "CASSI" Standard

F5 argues that current industry metrics (like simple "attack success rates") are insufficient. Their CASSI score framework evaluates:

• Severity: The impact of the breach (e.g., weaponizing information vs. minor data leakage).
• Complexity: The sophistication of the attack (e.g., simple "jailbreak" prompts vs. complex, multi-step invasive context engineering).

4. Case Study: Social Engineering on Steroids

The video illustrates a step-by-step attempt to extract sensitive HR data:

1. Probing: The agent sets a scene (e.g., "Alice is terminally ill") to manipulate the model into bypassing privacy protocols.
2. Chain of Thought Evaluation: The agent analyzes the model's response. If the model refuses (e.g., "Go speak to HR"), the agent pivots its strategy.
3. Pivoting: The agent shifts tactics—moving from direct requests to asking for "comparison tables" or "salary ranges."
4. Breach: By combining reasonable arguments with persistent probing, the agent eventually extracts the specific data point (the target's salary) by inferring it from the provided ranges.

5. Key Quotes

• "We're bringing in a non-deterministic component into an ecosystem that we have to do stress testing with red teaming and also hardening of that system with guardrails." — Alan Healey
• "There is no benchmark for security posture of these systems. There's no CVE standard... within F5, we're actually trying to build that standard." — Alan Healey
• "This is like social engineering on steroids." — Jason Rom (referring to the agentic red teaming process).

Synthesis and Conclusion

The core takeaway is that securing AI requires moving beyond static, rule-based filters. Because RAG systems and AI agents are dynamic and context-dependent, security must be equally adaptive. F5’s methodology emphasizes that transparency in agent behavior—allowing engineers to see the "planning" and "pivoting" strategies—is vital for identifying how and why a system fails. By standardizing metrics through the CASSI score and automating the generation of complex, multi-turn attack vectors, organizations can better anticipate and mitigate the risks inherent in modern AI deployments.