AI & Cybersecurity: Neil Daswani Interviews Heather Adkins

Key Concepts

• Cybersecurity Resilience: The ability of an organization to withstand and recover from cyberattacks.
• Information Security: The practice of protecting information by mitigating information risks.
• Data Breach Notification Law: Legislation requiring organizations to inform individuals when their personal data has been compromised.
• Misconfigurations: Incorrect or suboptimal settings in software or systems that can create vulnerabilities.
• Abstract Layers: The different levels of abstraction in computing, from hardware to high-level software, where developers often work without full visibility of underlying complexities.
• AI (Artificial Intelligence) & Machine Learning (ML): Advanced computational techniques used for pattern recognition, prediction, and decision-making.
• Generative AI: A type of AI that can create new content, such as text, images, or code.
• Spam Filter: A system designed to detect and block unwanted email (spam).
• App Behavior Classification: Using AI to analyze the actions and characteristics of applications to identify malicious ones.
• Malware Detection: Identifying and preventing harmful software.
• VirusTotal: A Google-owned online service that analyzes suspicious files and URLs to detect types of malware.
• Gemini: Google's large language model (LLM).
• Reverse Engineering: The process of analyzing a system to understand its design, architecture, or code.
• Defender's Dilemma: The challenge where defenders must secure every possible entry point, while attackers only need to find one vulnerability.
• Dwell Time: The amount of time an attacker remains undetected within a compromised network.
• Agentic AI / Agentic Frameworks: AI systems composed of multiple agents that can communicate, collaborate, and perform tasks autonomously.
• Wire Speed Detection: Detecting threats at the speed of network communication, rather than human response time.
• Turing Undecidable Problem: A problem for which no algorithm can provide a correct "yes" or "no" answer for all possible inputs. Identifying malicious programs is an example.
• Backdoors: Covert methods of bypassing normal authentication or encryption in a computer system.
• Buffer Overflow: A common software vulnerability where a program writes data to a buffer beyond its allocated size, potentially overwriting adjacent memory.
• Memory Unsafe Language: Programming languages (e.g., C, C++) that allow direct memory access, making them susceptible to memory-related vulnerabilities like buffer overflows.
• SAIF (Secure AI Framework): Google's framework for building and protecting AI systems.
• Prompt Injection: A type of attack on LLMs where malicious input (a "prompt") manipulates the model into performing unintended actions or revealing sensitive information.
• SQL Injection / XML Injection: Similar injection attacks targeting databases (SQL) or XML parsers.
• Authentication & Authorization: Verifying identity (authentication) and granting appropriate permissions (authorization).
• Data Movement / Data Integrity: Ensuring that data remains accurate, consistent, and protected as it is transferred or processed.
• Multi-Context Protocol (MCP): A protocol for AI agents to communicate and manage context across different interactions.
• Agent-to-Agent Protocol: A protocol enabling communication and interoperability between different AI agents.
• Secure Enclaves / Confidential Computing: Hardware-based security features that protect data in use by isolating it in a trusted execution environment, preventing unauthorized access even from the operating system or hypervisor.
• COBOL Transformation: Modernizing legacy COBOL codebases using AI assistance.
• STEM (Science, Technology, Engineering, Mathematics): Academic disciplines crucial for technological advancement.
• Cognitive Scaffolding: The fundamental human knowledge and skills that underpin and enable the effective use and development of AI.
• Frontier Model Forum (FMF): A coalition of leading AI companies focused on ensuring the safe and responsible development of frontier AI models.
• Coalition for Secure AI: An initiative focused on developing playbooks and best practices for safely adopting AI in various organizations.
• Supervised Learning: An ML approach where the model is trained on labeled data (input-output pairs).
• Unsupervised Learning: An ML approach where the model finds patterns in unlabeled data.
• Synthetic Data: Artificially generated data that mimics real-world data characteristics but does not contain actual sensitive information.
• Threat Modeling: A structured approach to identifying potential threats and vulnerabilities in a system.
• Continuous Learning: The ongoing process of acquiring new knowledge and skills, especially critical in rapidly evolving fields like AI and cybersecurity.
• Collaboration Skills: The ability to work effectively with others to achieve common goals.

---

Introduction and Heather Adkins' Background

Heather Adkins, Head of Google's Office of Cybersecurity Resilience and Deputy Chair of CISA's Cyber Safety Review Board, with over 25 years of experience, shared her journey into cybersecurity. She initially studied marine biology but was drawn to a Unix lab, finding its command-line interface intriguing. An unpaid internship led to a pivotal experience: a system hack. This "sleuthing adventure" highlighted that systems are not inherently secure by default, sparking her passion for solving this fundamental problem.

Early Cybersecurity Challenges and the Evolution of "Cyber"

Neil Daswani noted the evolution of "cyber" from a derided term to a common one. He also highlighted that before California's 2003 data breach notification law, many hacks likely occurred undetected. From a computer science perspective, software systems are complex, with numerous configuration parameters; even a few misconfigurations can lead to initial compromises and escalating incidents. The internet's opaque building blocks, each with potential flaws, mean that experts in specific components can uncover novel vulnerabilities. Developers often work at abstract layers, unaware of the hundreds of underlying layers, making the problem fascinatingly complex. Common breach reasons include misconfigurations, software vulnerabilities, account takeovers, malware, third-party compromise, and unencrypted data.

AI and Machine Learning in Google's Cybersecurity (Past & Present)

Google has utilized AI and machine learning for cybersecurity applications for over 20 years, long before the public prominence of generative AI. While applying advanced data processing and ML to security's "data problem" has historically been challenging, Google has achieved significant success in several areas:

• Gmail Spam Filter: Over 20+ years, this filter has evolved to automatically filter out 99.9% of fraud, phishing, and malware attacks using increasingly sophisticated ML models and statistical analysis. Generative AI is further enhancing these capabilities.
• App Behavior Classification: For platforms like Android and Chrome, millions of apps and Chrome Web extensions are processed through AI classifiers. These classifiers understand app behavior to filter out malicious content, maintaining open platforms while ensuring user safety.
• Malware Detection: Adware, exemplified by the 2006 Clickbot A incident, has grown into a major industry producing highly sophisticated, stealthy malware. Traditional supervised and unsupervised learning models struggle with malware detection due to a lack of sufficient "hack data" for training, often leading to false positives (e.g., new employees' network behavior mimicking threat actors).
• Generative AI for Malware Analysis: Google's VirusTotal, the world's largest malware database, now integrates with large language models like Gemini. Gemini can analyze malware (written in machine language) by reverse engineering binaries or reading scripts, providing analysts with insights in seconds, a task that previously took human analysts weeks. This dramatically reduces detection and response times.

Advancements in AI for Better Defense (Future Vision)

AI is poised to fundamentally alter the "defender's dilemma," where defenders must be perfect, and attackers only need one flaw. Rob Joyce's quote, "Hackers win because they know your networks better than you do," highlights this challenge. AI will enable defenders to process data at unprecedented scales and sophistication, allowing them to spot the "full chain of attack" easily and quickly. This will drastically reduce the average dwell time (attacker presence in networks), currently 10-200 days, to mere hours or minutes, significantly shrinking the attacker's window of success.

The vision includes agentic AI in the enterprise: systems of AI agents that communicate and perform tasks autonomously. For businesses, this means querying AI for complex reports (e.g., "top customers and complaints") in seconds, freeing human experts to solve problems. In defense, security professionals could receive morning summaries of critical events and automated actions taken (e.g., a CEO phishing attempt detected and blocked, a CFO's account locked to prevent fraudulent wire transfers). While acknowledging the need to build trust in these systems, the goal is to achieve "wire speed" detection and response, fundamentally changing the battlefield.

Emerging Threats and Computational Limits

Heather Adkins noted that many core cybersecurity problems were identified in papers from 1968-1972, suggesting a persistent nature to these challenges. Attack motivations (money, espionage, hacktivism, curiosity) remain constant, but the speed and sophistication of attacks will increase due to automation. The rise of an "agentic hacker" is anticipated, where open-source platforms could allow users to command an AI to "hack this company" and gain access in days.

However, there are inherent computational limits:

• Turing Undecidable Problem: Identifying malicious programs is a Turing undecidable problem, meaning no algorithm can achieve 100% accuracy in all cases.
• Backdoors: Backdoors, like those in SolarWinds or XZ Utils, are still often detected manually. Ken Thompson's Turing Award lecture famously demonstrated how a compiler could be backdoored to inject malicious code into every compiled program, making it indiscernible.

Despite these limits, AI offers solutions:

• Reasoning about Code: Large language models (LLMs) are expected to reason about code with increasing sophistication.
• Vulnerability Detection: AI assistants could spot stubborn vulnerabilities like buffer overflows in complex, memory-unsafe legacy code before production.
• Backdoor Detection: LLMs could compare code versions, highlight differences, and facilitate human inspection, potentially addressing the compiler backdoor problem.
• Practical Improvement: While 100% detection is impossible, AI can lead to "practically much, much, much better detection rates" and aid in backdoor detection.

Building Secure AI Systems (SAIF, Secure Agents, Protocols)

Google developed the SAIF (Secure AI Framework) for structured thinking in building and protecting AI technology. Key elements include:

• Protecting Training Data: Ensuring data integrity to prevent models from learning and regurgitating incorrect or unsafe information (e.g., from tampered data).
• Model Integrity: Protecting the model's supply chain throughout its lifecycle.
• Preventing Classes of Attacks: Addressing threats like prompt injection, where malicious prompts manipulate LLMs into unintended actions (e.g., revealing sensitive data, ignoring legitimate user commands). This is analogous to historical SQL or XML injection attacks.
• AI for Prevention: Identifying and deploying AI effectively for attack prevention.

For building secure AI agents, which perform actions rather than just answer questions, challenges include agent takeover and credential management. A research paper co-authored by Christoph Kern outlines critical considerations:

• Authentication and Authorization: Essential for agent access and actions.
• Data Movement: Preserving original data settings (e.g., protection, privacy) when agents copy, transform, or move data, a complex problem requiring new frameworks.

Protocols for agent communication are crucial:

• Multi-Context Protocol (MCP): Anthropic's protocol for agents to speak to each other, addressing authorization for data access, movement, and transformation.
• Google's Agent-to-Agent Protocol: A complementary protocol focusing on interoperability across enterprise cloud data silos and providers, aiming for standardization to enable more productive and secure agent interactions.

The convergence of secure enclaves (confidential computing) in cloud infrastructure with these protocols holds promise. Secure enclaves protect sensitive data in use by isolating it, making it accessible only to secure co-processors. This could enable AI agents to operate on data more securely than current human-operated systems. LLMs' language understanding can also accelerate the adoption of such technologies by translating between different protocols and assisting in transforming legacy infrastructure (e.g., rewriting COBOL code), as demonstrated by Morgan Stanley.

Diversity in Cybersecurity (Women in Tech)

According to a recent ISC squared study, only 22% of cybersecurity professionals are women, an improvement from 4% when Heather Adkins started her career, but still with significant room for growth. This is viewed as both a pipeline problem (getting young girls interested in STEM) and a retention problem.

Advice for women pursuing a career in cybersecurity:

• Persistence: It's a challenging but rewarding field with a wide variety of problems.
• Specialization: Like medicine, cybersecurity has many specializations (e.g., cryptographers, detection specialists). Finding a niche that aligns with one's interests can lead to longevity and satisfaction.
• Meaningful Work: Focus on tasks that provide a sense of accomplishment and personal growth.
• Community and Collaboration: Build a strong network of peers for support, guidance, and collaboration, which fosters a sense of belonging and helps overcome challenges.
• Courage: Essential for navigating the field.

Historical Perspective and the Future of Education

Comparing the current era to medieval times, Heather Adkins noted that Petrarch termed the Middle Ages "dark" due to a decline in education and knowledge, leading to incomplete written records. Today, everything is recorded, a stark contrast. However, the rate of technological change echoes the transition from the Middle Ages to the Renaissance, emphasizing the scientific method. Humans must retain this core scientific thinking (spotting problems, hypothesizing, testing) even as AI accelerates the process.

To prevent a "next dark ages," which historically resulted from the erosion of education, it's crucial to:

• Maintain Cognitive Scaffolding: Continue focusing on STEM education to ensure humans retain fundamental knowledge and skills, rather than becoming overly reliant on machines.
• Leverage AI for Education: AI can support customized, mass-customized education, offering personalized learning experiences (e.g., conversational agents for language learning, tailored math instruction). This can make learning more engaging and accessible, preventing a loss of human cognitive capabilities.

Beyond LLMs and Essential Skills for Cybersecurity Professionals

LLMs are powerful but not the sole or always the appropriate tool. Highly specific AI models, like AlphaGo for Go or DeepMind's protein folding model, achieve extraordinary results in narrow domains. In cybersecurity, traditional supervised and unsupervised models often struggle with a lack of "real hack" training data, making it difficult to finely tune them for accurate threat detection. However, they succeed in data-rich areas like spam classification (using abundant email and synthetic data) and app behavior analysis.

For future cybersecurity professionals, essential skills include:

• Open Mindset and Continuous Learning: The AI paradigm will shift rapidly, requiring constant curiosity and adaptation to new technologies and workflows.
• STEM Skills (Cognitive Scaffolding): Software engineering, coding, threat modeling, and secure code principles remain vital for underpinning and improving AI systems. As Sir Demis Hassabis noted, this "cognitive scaffolding" is indispensable.
• Collaboration Skills: Machines cannot replace human teamwork. The ability to collaborate effectively in teams will be crucial for solving complex problems.
• Imagination and Innovation: The human capacity to connect dots, identify problems, and envision novel solutions ("a-ha moments") will continue to drive progress, as LLMs do not possess this spark of imagination.

The ability to code remains critical, even with AI-generated code, to "trust but verify" its correctness and security. The future of software development will involve humans building "scaffolding" around AI, such as scrutineer engineers or fuzzing models, to ensure trust and functionality, rather than simply having machines write all the code.

Conclusion

Heather Adkins provided invaluable insights into the transformative potential of AI in cybersecurity, emphasizing its role in enhancing defense capabilities, addressing long-standing challenges, and shaping the future of the field. She underscored the importance of responsible AI development, continuous human learning, collaboration, and maintaining fundamental STEM skills to navigate this rapidly evolving landscape.