What Frontier Models Can Actually Do in a SOC: Open-source Benchmark for Agentic SecOps Capabilities
Maxime Lamothe-Brassard, founder and CEO of LimaCharlie, sought answers on AI’s current capabilities in the SecOps space. Plenty of benchmarks exist to test AI's knowledge of cybersecurity, but none test whether a model actually does the work. There's a significant difference between an AI that can answer trivia questions about CVEs and one that can pick up an alert, investigate it, and produce an incident report.That gap matters more now than ever.