AutomationBench
Progress Over Time
Interactive timeline showing model performance evolution on AutomationBench
AutomationBench Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  DeepSeek | 763B | 1.0M | $0.30 / $1.20 | ||
| 2 |  Mistral AI | 1.1T | 1.0M | $0.70 / $2.10 | ||
| 3 | DeepSeek | 763B | 1.0M | $0.22 / $0.66 | ||
| 4 |  Google | — | — | — | ||
| 5 | Google | — | 1.0M | $2.00 / $10.00 | ||
| 6 |  Meta | — | 1.0M | $0.10 / $0.20 | ||
| 7 |  Zhipu AI | 320B | 1.0M | $0.15 / $0.50 | ||
| 8 | Zhipu AI | 753B | 1.0M | $1.20 / $4.00 | ||
| 9 |  Anthropic | — | 1.0M | $2.00 / $10.00 | ||
| 10 |  OpenAI | — | 1.1M | $10.00 / $50.00 | ||
| 11 |  Liquid AI | 2B | — | — | ||
| 12 | Anthropic | — | 1.0M | $4.00 / $20.00 | ||
| 13 |  Reflection AI | 501B | 1.0M | $0.40 / $1.20 | ||
| 14 |  Tencent | 770B | — | — | ||
| 15 | DeepSeek | 1.6T | 1.0M | $1.30 / $2.60 | ||
| 16 | Anthropic | — | 1.0M | $10.00 / $50.00 | ||
| 17 |  Moonshot AI | 2.8T | 1.0M | $2.85 / $14.25 | ||
| 18 | Google | — | 1.0M | $0.75 / $3.75 | ||
| 19 |  Alibaba Cloud / Qwen Team | 2.4T | 1.0M | $1.65 / $4.95 | ||
| 20 | Anthropic | — | 1.0M | $5.00 / $25.00 | ||
| 21 | DeepSeek | — | 1.0M | $0.44 / $1.32 | ||
| 22 |  Institute of Foundation Models | 375B | — | — | ||
| 23 | DeepSeek | 304B | 1.0M | $0.06 / $0.18 | ||
| 24 | OpenAI | — | 1.1M | $5.00 / $30.00 | ||
| 25 | Anthropic | — | 1.0M | $10.00 / $50.00 | ||
| 26 | OpenAI | — | 1.1M | $2.00 / $12.00 | ||
| 27 | OpenAI | — | 1.1M | $0.20 / $1.20 | ||
| 28 | Anthropic | — | 1.0M | $2.00 / $10.00 |
Sub-benchmarks
AutomationBench-AA
AutomationBench-AA is Artificial Analysis's independently run version of AutomationBench, covering 657 real-world SaaS workflow tasks across 40 simulated applications (e.g. Gmail, Slack, Salesforce, HubSpot). It scores the share of objectives an agent completes without violating business guardrails, using a private held-out task set.
AutomationBench v1.0.6
AutomationBench v1.0.6 evaluates agent performance on multi-step workflow automation tasks.
What is AutomationBench?
AutomationBench is a tool-use benchmark that evaluates AI agents on automating real-world workflows, testing their ability to orchestrate tools and complete multi-step automation tasks.
AutomationBench is a text benchmark evaluating models on reasoning, agents, and tool calling tasks. LLM Stats tracks 28 models on this benchmark, scored on a 0–1 scale. The current average is 0.4, with the leader at 0.6.
Compare leaders on the best AI for reasoning, best AI for agents and best AI for tool calling leaderboards.
Current leaders
DeepSeek-V4.1-Flash-1017 from DeepSeek currently leads the AutomationBench leaderboard with a score of 0.605 across 28 evaluated AI models.
FAQ
Common questions about the AutomationBench benchmark and leaderboard.