Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Back to Benchmarks

AgentCyberRange

Penetration Testing
Published June 12, 2026
View PaperView CodeView Dataset
Benchmark Overview

About this benchmark

Open multi-range benchmark measuring whether frontier AI systems can carry out realistic end-to-end attacks, spanning web exploitation and post-exploitation lateral movement across live multi-host cyber ranges

Dataset

266 samples

266 benchmark instances: 110 vulnerabilities across 15 real web applications for the web-exploitation stage, plus 8 cyber ranges containing 156 internal hosts for the post-exploitation stage. Enterprise components such as wikis, CI/CD systems and AI applications are included. Three difficulty levels supply progressively more information, from Level-0 (target URL or entry-point IP only) to Level-2 (vulnerability types or concrete CVEs).

5 tasks

Web ExploitationPost ExploitationLateral MovementPrivilege EscalationReconnaissance

Metrics

Results are ranked by the first listed metric; direction is shown for every metric below.

Web exploitation Pass@1 ↑Web exploitation Pass@3 (avg) ↑Web exploitation Pass@3 (max) ↑Post exploitation Pass@1 ↑Post exploitation Pass@3 (avg) ↑Post exploitation Pass@3 (max) ↑
Model Results
Ranked by Web exploitation Pass@1 · higher is better
RankModelWeb exploitation Pass@1Web exploitation Pass@3 (avg)Web exploitation Pass@3 (max)Post exploitation Pass@1Post exploitation Pass@3 (avg)Post exploitation Pass@3 (max)Evaluated ByDateSource
1st
GPT-5.5
gpt-5-5-codex • OpenAI
19.1%16.1%28.2%31.7%31.7%43.9%AgentCyberRange authors (Fudan University)June 12, 2026Tables 4 and 5: Level-0 web exploitation and post exploitation success rates
2nd
Claude Opus 4.7
claude-opus-4-7 • Anthropic
16.4%14.6%26.4%12.2%15.0%21.9%AgentCyberRange authors (Fudan University)June 12, 2026Tables 4 and 5: Level-0 web exploitation and post exploitation success rates
3rd
GLM-5.1
glm-5-1 • Zhipu AI
11.8%8.2%15.4%17.1%11.4%19.5%AgentCyberRange authors (Fudan University)June 12, 2026Tables 4 and 5: Level-0 web exploitation and post exploitation success rates
#4
Qwen 3.7 Max
qwen3.7-max • Alibaba
10.9%12.4%20.9%19.5%13.0%19.5%AgentCyberRange authors (Fudan University)June 12, 2026Tables 4 and 5: Level-0 web exploitation and post exploitation success rates
#5
DeepSeek V4 Pro
deepseek-v4-pro • DeepSeek
10.0%8.2%18.2%9.8%12.2%19.5%AgentCyberRange authors (Fudan University)June 12, 2026Tables 4 and 5: Level-0 web exploitation and post exploitation success rates
#6
Kimi K2.6
kimi-k2-6 • Open-weight
3.6%3.0%8.2%12.2%5.7%12.2%AgentCyberRange authors (Fudan University)June 12, 2026Tables 4 and 5: Level-0 web exploitation and post exploitation success rates
Cyber LLM Benchmark Hub

Cyber LLM Benchmark Hub

Benchmarking frontier models across cybersecurity tasks.

BenchmarksContactAbout

© 2026 Cyber LLM Benchmark Hub