CrowdStrike, in collaboration with Meta, has unveiled CyberSOCEval, a new suite of open-source benchmarks designed to measure the effectiveness of AI systems in real-world security operations. This initiative blends Meta’s CyberSecEval framework with CrowdStrike’s threat intelligence and AI-driven security expertise, creating a foundation for evaluating large language models (LLMs) within the security operations center (SOC).
Security teams are inundated with alerts and face increasingly sophisticated threats. While AI and LLMs are emerging as game-changers for automating workflows and accelerating detection, many organizations struggle to determine which solutions deliver measurable value. The absence of reliable standards has left security leaders uncertain about performance and readiness against active adversaries.
Cyber Technology Insights : Smarter Security: Akamai and Seraphic Team Up to Simplify SSE with Secure Enterprise Browsing
CyberSOCEval addresses this gap by offering benchmarks that simulate high-pressure, real-world SOC conditions. The framework tests AI performance across core functions including incident response, malware investigation, and threat intelligence analysis. By assessing LLMs against a mix of real adversary tactics and expert-crafted reasoning challenges, organizations gain the ability to validate operational readiness, identify high-value use cases, and optimize ROI. Meanwhile, model developers benefit from a roadmap to enhance functionality and deliver AI tools that align with SOC needs.
“At Meta, we believe in the power of open-source AI to advance security for everyone,” said Vincent Gonguet, Director of Product, GenAI at Superintelligence Labs, Meta. “Our work with CrowdStrike introduces a benchmark suite that reflects real-world defense scenarios. By making this open to the community, we can accelerate progress and strengthen defenses against advanced and AI-driven threats.”
Cyber Technology Insights : Proofpoint Unveils First Agentic AI Solution for Human Communications Intelligence
“When CrowdStrike and Meta join forces, it’s not just collaboration—it’s about shaping the future of cybersecurity in the AI era,” added Daniel Bernard, Chief Business Officer at CrowdStrike. “By combining our adversary intelligence and AI-native security expertise with Meta’s research and data scale, we’re setting new standards for how AI in the SOC should function. The result is greater defender confidence and stronger resilience against adversaries.”
With CyberSOCEval, CrowdStrike and Meta aim to empower the cybersecurity industry to confidently adopt AI solutions, ensuring they meet the demands of modern threat landscapes and deliver tangible value in protecting organizations worldwide.
Cyber Technology Insights : Cyber Skills Shortage Forces 64 Percent of EMEA Organisations into Risky Security Shortcuts
To participate in our interviews, please write to our CyberTech Media Room at sudipto@intentamplify.com
