
The definitive source for cybersecurity LLM performance.
Compare models across
Comprehensive evaluation across 10 cybersecurity domains
1 benchmark
7 benchmarks
6 benchmarks
2 benchmarks
4 benchmarks
12 benchmarks
7 benchmarks
5 benchmarks
0 benchmarks
2 benchmarks
Latest cybersecurity LLM evaluation datasets
Benchmark measuring operational stealth in autonomous offensive-security agents: whether an agent can complete an intrusion task without leaking credentials, tripping detection, causing third-party harm, or leaving forensic traces
Open multi-range benchmark measuring whether frontier AI systems can carry out realistic end-to-end attacks, spanning web exploitation and post-exploitation lateral movement across live multi-host cyber ranges
Large-scale benchmark evaluating AI agents across the full vulnerability lifecycle on real open-source code: discovering the vulnerability, generating a crashing proof-of-concept, and producing a patch that holds without breaking the test suite
Benchmark testing whether LLMs can automate a real-world threat-research workflow end to end, covering article triage and prioritization, deep search for related sources, and threat-intelligence drafting, scored with analyst-centric rather than text-similarity metrics
Statement-level benchmark for real-world C/C++ vulnerability detection with rich program context.
A Comprehensive Evaluation Framework and Benchmarks for LLMs in Security Vulnerability Identification and Reasoning
Help build the most comprehensive cybersecurity LLM benchmark database. Submit your evaluation results or support the project.