Why a Red-Team Pass Isn't Enough: Testing AI Beyond the Point-in-Time Scan

Why a Red-Team Pass Isn't Enough: Testing AI Beyond the Point-in-Time Scan

A single security scan is no longer sufficient to secure Large Language Models (LLMs) and autonomous AI agents. In this deep dive, Jamison Utter (Field CISO, A10 Networks) and Priyanka Mullan explore why modern AI cybersecurity requires continuous, multi-turn, adaptive red teaming rather than static point-in-time penetration testing.

Key Discussion Points & Takeaways:
▫️ Logic Layer vs. Traditional Penetration Testing: Traditional pen testing checks for specific flaws and vulnerabilities (APIs, access controls), but AI red teaming must evaluate how the model interprets context, follows logic, and makes decisions.
▫️ The "Willing Attacker" Problem & Obliteration: Safety-aligned attacker models often refuse to generate realistic test scenarios. Obliteration strips refusal codes and safety constraints from the testing model so it can run true, uninhibited attack iterations.
▫️ Multi-Turn & Agentic Attack Vectors: Single-turn tests only show if a specific command fails. Dynamic multi-turn attacks allow an attacker agent to adapt, morph prompts based on responses, and build context to uncover non-deterministic vulnerabilities.
▫️ Securing Autonomous AI Agents: When an AI agent connects to external tools, databases, and APIs, testing must follow the full workflow. Strict authorization outside the model, DLP controls, and human-in-the-loop oversight prevent indirect prompt injections and unauthorized actions.
▫️ Continuous Red Teaming as a Lifecycle: Because LLMs are non-deterministic and change behavior whenever models, tools, or contexts update, AI security testing must be an ongoing, continuous process—not a one-time check before launch.

Learn more about securing AI infrastructure and agent workflows at: https://bit.ly/4srUI1W

Timestamps

0:00 - Introduction to AI Red Teaming with Jamison Utter & Priyanka Mullan

0:15 - Red Teaming Beyond the Model: Traditional Pen Testing vs. AI Logic Testing

1:42 - The Willing Attacker Problem & Model Obliteration

3:36 - Why One Pass Isn't Enough: Single-Turn vs. Multi-Turn Testing 5:40 - Adaptive Red Team Tactics & Multimodal Exploits

7:07 - Hallucinations & The Need for Human/Multi-Model Review

8:28 - Securing Autonomous AI Agents & Downstream Workflows

10:52 - Authorization, DLP Controls, and Human-in-the-Loop Oversight 12:00 - Kill Switches & Tool Server Controls for Agentic AI

12:43 - Continuous Red Teaming across the AI Lifecycle

15:38 - Final Thoughts & Conclusion

#airedteaming #cybersecurity #a10networks #llmsecurity #aisecurity #agenticai #penetrationtesting #promptinjection #continuoussecurity #informationsecurity