- Organizations
- Laude Institute
Laude Institute: AI benchmarks & research
Non-profit lab that runs Terminal-Bench with Stanford, plus the Terminus reference agent.
Maintained by Laude Institute
9 benchmarks- Terminal-Bench
Terminal-Bench is a benchmark for testing AI agents in real terminal environments. It evaluates how well agents can handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, security tasks, data science workflows, and cybersecurity vulnerabilities. The benchmark consists of a dataset of ~100 hand-crafted, human-verified tasks and an execution harness that connects language models to a terminal sandbox.
- Leader
Claude Sonnet 4.550.0%
- Models scored
- 25
- Terminal-Bench 2.0
Terminal-Bench 2.0 is an updated benchmark for testing AI agents' tool use ability to operate a computer via terminal. It evaluates how well models can handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, security tasks, data science workflows, and cybersecurity vulnerabilities.
- Leader
GPT-5.582.7%
- Models scored
- 53
- Terminal-Bench 2.1
Terminal-Bench 2.1 is an updated release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal. It evaluates how well models handle real-world, end-to-end tasks autonomously, including compiling code, training models, setting up servers, system administration, data science workflows, and security tasks.
- Leader
SWE-292.8%- Models scored
- 56
- Terminal-Bench 3.0
Terminal-Bench 3.0 is a release of the Terminal-Bench benchmark that tests AI agents' ability to operate a computer via the terminal on real-world, end-to-end tasks.
- Leader
DeepSeek-V4.1-Flash30.0%
- Models scored
- 4
- Terminal-Bench 4.0
Terminal-Bench 4.0 evaluates AI agents on real-world work in terminal and command-line containerized environments across 66 tasks, with emphasis on science-adjacent and frontier engineering problems. Relative to earlier Terminal-Bench releases, 4.0 increases timeouts and adaptively raises RAM/CPU on selected tasks to reduce harness and resource confounds.
- Leader
Claude Sonnet 5.570.6%
- Models scored
- 37
- Terminal-Bench Hard
Terminal-Bench Hard is a harder terminal-agent benchmark variant evaluated with the Terminus-2 harness in Cohere's Command A+ and North Mini Code releases.
- Leader
North Mini Code 1.031.1%- Models scored
- 2
- Terminal-Bench-Science 0.1
Terminal-Bench-Science 0.1 is a Stanford-led community benchmark of 70 tasks drawn from scientific research workflows across the life, physical, earth, mathematical, and engineering sciences. Tasks are authored and reviewed by scientists and researchers; scores are typically reported as accuracy with relatively large standard error due to strongly bimodal per-task outcomes.
- Leader
Claude Mythos 5.275.3%
- Models scored
- 13
- Terminus
Terminal-Bench is a benchmark for testing AI agents in real terminal environments, evaluating how well agents can handle real-world, end-to-end tasks autonomously. The benchmark includes tasks spanning coding, system administration, security, data science, model training, file operations, version control, and web development. Terminus is the neutral test-bed agent designed to work with Terminal-Bench, operating purely through tmux sessions without dedicated tools.
- Leader
Kimi K2 Instruct25.0%
- Models scored
- 1
- Terminal-Bench 4.0-AA
Primary maintainer Artificial Analysis
Artificial Analysis reported accuracy on terminal tasks with executable final-container-state verifiers.
- Leader
Gemini 4 Carbon65.2%
- Models scored
- 5