NL2Repo
Progress Over Time
Interactive timeline showing model performance evolution on NL2Repo
State-of-the-art frontier
Open
Proprietary
NL2Repo Leaderboard
12 models
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Zhipu AI | 753B | 1.0M | $0.95 / $3.00 | ||
| 2 |  Alibaba Cloud / Qwen Team | — | 1.0M | $1.25 / $3.75 | ||
| 3 |  ByteDance | — | — | — | ||
| 4 |  Tencent | 295B | — | — | ||
| 5 | ByteDance | — | — | — | ||
| 6 | Zhipu AI | 754B | 200K | $1.40 / $4.40 | ||
| 7 |  MiniMax | — | 1.0M | $0.30 / $1.20 | ||
| 8 | Alibaba Cloud / Qwen Team | — | 1.0M | $0.32 / $1.28 | ||
| 9 | MiniMax | — | 205K | $0.30 / $1.20 | ||
| 10 | Alibaba Cloud / Qwen Team | — | 1.0M | $0.50 / $3.00 | ||
| 11 | Alibaba Cloud / Qwen Team | 28B | 262K | $0.60 / $3.60 | ||
| 12 | Alibaba Cloud / Qwen Team | 35B | — | — |
Notice missing or incorrect data?
What is NL2Repo?
NL2Repo evaluates long-horizon coding capabilities including repository-level understanding, where models must generate or modify code across entire repositories from natural language specifications.
NL2Repo is a text benchmark evaluating models on agents and code tasks. LLM Stats tracks 12 models on this benchmark, scored on a 0–1 scale. The current average is 0.4, with the leader at 0.5.
Compare leaders on the best AI for agents and best AI for code leaderboards.
Current leaders
GLM-5.2 from Zhipu AI currently leads the NL2Repo leaderboard with a score of 0.489 across 12 evaluated AI models.
FAQ
Common questions about the NL2Repo benchmark and leaderboard.