SWE-Bench Pro
Progress Over Time
Interactive timeline showing model performance evolution on SWE-Bench Pro
SWE-Bench Pro Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Anthropic | — | 1.0M | $10.00 / $50.00 | ||
| 2 | Anthropic | — | — | — | ||
| 3 | Anthropic | — | 1.0M | $5.00 / $25.00 | ||
| 4 |  Alibaba Cloud / Qwen Team | 2.4T | 1.0M | $1.65 / $4.95 | ||
| 5 |  Tencent | 770B | — | — | ||
| 6 |  xAI | — | 500K | $2.00 / $6.00 | ||
| 7 |  OpenAI | — | 1.1M | $5.00 / $30.00 | ||
| 8 | Anthropic | — | 1.0M | $5.00 / $25.00 | ||
| 9 | OpenAI | — | 1.1M | $2.00 / $12.00 | ||
| 10 | Anthropic | — | 1.0M | $2.00 / $10.00 | ||
| 11 | OpenAI | — | 1.1M | $0.20 / $1.20 | ||
| 12 | Alibaba Cloud / Qwen Team | 125B | — | — | ||
| 12 | Alibaba Cloud / Qwen Team | 125B | 1.0M | $0.15 / $0.47 | ||
| 14 |  Zhipu AI | 753B | 1.0M | $0.75 / $2.40 | ||
| 15 | Alibaba Cloud / Qwen Team | 28B | 262K | $0.40 / $3.00 | ||
| 16 |  Meta | — | 1.0M | $1.25 / $4.25 | ||
| 17 | Alibaba Cloud / Qwen Team | — | 1.0M | $1.25 / $3.75 | ||
| 18 |  Shanghai AI Laboratory | 744B | — | — | ||
| 19 |  Poolside | 118B | 1.0M | $0.10 / $0.20 | ||
| 20 |  MiniMax | 428B | 1.0M | $0.28 / $1.10 | ||
| 21 |  Google | — | 1.0M | $1.50 / $7.50 | ||
| 22 |  Moonshot AI | 1.0T | 262K | $0.75 / $3.50 | ||
| 22 | OpenAI | — | 1.1M | $5.00 / $30.00 | ||
| 24 | Zhipu AI | 754B | 203K | $1.05 / $3.50 | ||
| 25 | Tencent | 295B | 262K | $0.14 / $0.58 | ||
| 26 | OpenAI | — | 1.0M | $2.50 / $15.00 | ||
| 27 | Alibaba Cloud / Qwen Team | — | — | — | ||
| 28 |  ByteDance | — | — | — | ||
| 29 |  Xiaomi | 1.0T | 1.0M | $0.43 / $0.87 | ||
| 30 | ByteDance | — | — | — | ||
| 31 | OpenAI | — | 400K | $1.75 / $14.00 | ||
| 32 |  InclusionAI | 124B | 131K | $0.06 / $0.18 | ||
| 32 | Alibaba Cloud / Qwen Team | — | 1.0M | $0.50 / $3.00 | ||
| 34 | OpenAI | — | 400K | $1.75 / $14.00 | ||
| 35 |  StepFun | 198B | 262K | $0.20 / $1.15 | ||
| 36 | MiniMax | — | 205K | $0.30 / $1.20 | ||
| 37 | Xiaomi | 311B | 1.0M | $0.17 / $0.34 | ||
| 38 | Thinking Machines Lab | 276B | 524K | $0.30 / $1.20 | ||
| 39 |  DeepSeek | 1.6T | 1.0M | $1.30 / $2.60 | ||
| 39 | MiniMax | 230B | 1.0M | $0.30 / $1.20 | ||
| 41 | Google | — | 1.0M | $1.50 / $9.00 | ||
| 42 | OpenAI | — | 400K | $0.75 / $4.50 | ||
| 43 | Google | — | 1.0M | $2.00 / $12.00 | ||
| 43 | Google | — | 1.0M | $0.30 / $2.50 | ||
| 45 | Alibaba Cloud / Qwen Team | 28B | 262K | $0.32 / $3.20 | ||
| 46 |  Microsoft | 1.0T | — | — | ||
| 47 | DeepSeek | 284B | 1.0M | $0.09 / $0.18 | ||
| 48 | OpenAI | — | 400K | $0.20 / $1.25 | ||
| 48 | Meta | — | — | — | ||
| 50 | DeepSeek | 284B | 1.0M | $0.09 / $0.18 |
What is SWE-Bench Pro?
SWE-Bench Pro is an advanced version of SWE-Bench that evaluates language models on complex, real-world software engineering tasks requiring extended reasoning and multi-step problem solving.
SWE-Bench Pro is a text benchmark evaluating models on reasoning, agents, and code tasks. LLM Stats tracks 58 models on this benchmark, scored on a 0–1 scale. The current average is 0.6, with the leader at 0.8.
Compare leaders on the best AI for reasoning, best AI for agents and best AI for code leaderboards.
Current leaders
Claude Fable 5 from Anthropic currently leads the SWE-Bench Pro leaderboard with a score of 0.800 across 58 evaluated AI models.
FAQ
Common questions about the SWE-Bench Pro benchmark and leaderboard.