SWE Atlas - Codebase QnA
Progress Over Time
Interactive timeline showing model performance evolution on SWE Atlas - Codebase QnA
SWE Atlas - Codebase QnA Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 | Mistral AI | 1.1T | 1.0M | $0.70 / $2.10 | ||
| 1 | Meta | — | 1.0M | $0.10 / $0.20 | ||
| 3 | Meituan | 1.6T | 1.0M | $0.30 / $1.20 | ||
| 4 | Meta | 816B | 1.0M | $1.25 / $4.25 | ||
| 5 | Cognition | 276B | — | — | ||
| 6 | Institute of Foundation Models | 375B | — | — | ||
| 7 | Poolside | 118B | 1.0M | $0.10 / $0.20 | ||
| 8 | MiniMax | 428B | 1.0M | $0.28 / $1.10 | ||
| 9 | Reflection AI | 501B | 1.0M | $0.40 / $1.20 |
What is SWE Atlas - Codebase QnA?
SWE Atlas - Codebase QnA evaluates a model's ability to answer questions about real codebases, measuring repository-level comprehension and the ability to reason about code structure, behavior, and intent across an entire project.
SWE Atlas - Codebase QnA is a text benchmark evaluating models on agents and code tasks. LLM Stats tracks 9 models on this benchmark, scored on a 0–1 scale. The current average is 0.5, with the leader at 0.6.
Compare leaders on the best AI for agents and best AI for code leaderboards.
Current leaders
Mistral Large 4 from Mistral AI currently leads the SWE Atlas - Codebase QnA leaderboard with a score of 0.594 across 9 evaluated AI models.
FAQ
Common questions about the SWE Atlas - Codebase QnA benchmark and leaderboard.