Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Cyber LLM Benchmark Hub Logo
Cyber LLM Benchmark Hub
  • Home
  • Benchmarks
  • Contact
  • About
Support
Back to Benchmarks

StealthBench

Penetration Testing
Published July 28, 2026
View PaperView Code
Benchmark Overview

About this benchmark

Benchmark measuring operational stealth in autonomous offensive-security agents: whether an agent can complete an intrusion task without leaking credentials, tripping detection, causing third-party harm, or leaving forensic traces

Dataset

14 samples

14 dockerized offensive-security scenarios derived from real bug-bounty and red-team engagements, yielding 771 evaluated agent trajectories. Each run is scored both for task success and across six OPSEC tradecraft dimensions, so an agent can solve a task yet still fail stealth review.

6 tasks

Credential HandlingDestructive Operation AvoidanceDetection EvasionTelemetry CleanlinessArtifact SeparationRequest Discipline

Metrics

Results are ranked by the first listed metric; direction is shown for every metric below.

Safe success rate ↑Stealth@Solve ↑Reckless solve rate ↓Cost per safe solve ↓Evaluated runs ↑
Model Results
Ranked by Safe success rate · higher is better
RankModelSafe success rateStealth@SolveReckless solve rateCost per safe solveEvaluated runsEvaluated ByDateSource
1st
Claude Opus 4.8
claude-opus-4-8 • Anthropic
53.8%65.6%27.4%$1.08117StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
2nd
Kimi K3
kimi-k3 • Moonshot AI
52.5%57.8%39.4%$0.5599StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
3rd
GLM-5.2
glm-5-2 • Zhipu AI
50.4%55.8%36.1%$0.54119StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
#4
Qwen 3.7 Plus
qwen3.7-plus • Alibaba
41.1%61.1%24.1%$0.60112StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
#5
Nemotron 3 Ultra
nemotron-3-ultra • NVIDIA
40.0%82.2%7.1%$0.7170StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
#6
DeepSeek V4 Pro
deepseek-v4-pro • DeepSeek
35.7%60.1%22.9%$0.8770StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
#7
GPT-5.6 Sol
gpt-5-6-sol • OpenAI
34.8%54.0%29.0%$2.0569StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
#8
Trinity Large Thinking
trinity-large-thinking • Arcee
34.8%84.3%6.1%$0.90115StealthBench authorsJuly 28, 2026Rank / model: Runs, Solved, Safe success, Stealth@Solve, Reckless, $ / safe
Cyber LLM Benchmark Hub

Cyber LLM Benchmark Hub

Benchmarking frontier models across cybersecurity tasks.

BenchmarksContactAbout

© 2026 Cyber LLM Benchmark Hub