CyBench
CyBench is a suite of Capture-the-Flag (CTF) challenges measuring agentic cyber attack capabilities. It evaluates dual-use cybersecurity knowledge and measures the 'unguided success rate', where agents complete tasks end-to-end without guidance on appropriate subtasks.
Progress Over Time
Interactive timeline showing model performance evolution on CyBench
State-of-the-art frontier
Open
Proprietary
CyBench Leaderboard
2 models
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Anthropic | — | — | $25.00 / $125.00 | ||
| 2 | — | 256K | $3.00 / $15.00 |
Notice missing or incorrect data?
FAQ
Common questions about CyBench
CyBench is a suite of Capture-the-Flag (CTF) challenges measuring agentic cyber attack capabilities. It evaluates dual-use cybersecurity knowledge and measures the 'unguided success rate', where agents complete tasks end-to-end without guidance on appropriate subtasks.
The CyBench paper is available at https://arxiv.org/abs/2408.08926. This paper provides detailed information about the benchmark methodology, dataset creation, and evaluation criteria.
The CyBench leaderboard ranks 2 AI models based on their performance on this benchmark. Currently, Claude Mythos Preview by Anthropic leads with a score of 1.000. The average score across all models is 0.695.
The highest CyBench score is 1.000, achieved by Claude Mythos Preview from Anthropic.
2 models have been evaluated on the CyBench benchmark, with 0 verified results and 2 self-reported results.
CyBench is categorized under agents, code, and safety. The benchmark evaluates text models.