Benchmarks/code/LiveCodeBench

LiveCodeBench

LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.

Paper

Progress Over Time

Interactive timeline showing model performance evolution on LiveCodeBench

State-of-the-art frontier
Open
Proprietary

LiveCodeBench Leaderboard

68 models • 0 verified
ContextCostLicense
1
DeepSeek
0.833685B——
2
MiniMax
MiniMax M2
MiniMax
0.830230B1.0M
$0.30
$1.20
3
Meituan
0.828560B128K
$0.30
$1.20
4
NVIDIA
0.812120B262K
$0.10
$0.50
5
xAI
0.804—128K
$0.30
$0.50
6
xAI
0.800—2.0M
$0.20
$0.50
7
xAI
0.794———
7
Meituan
0.794560B128K
$0.30
$1.20
7
xAI
0.794—128K
$3.00
$15.00
10
xAI
0.790———
11
MiniMax
0.780230B1.0M
$0.30
$1.20
12
DeepSeek
0.741685B——
13
DeepSeek
0.733671B131K
$0.50
$2.15
14
Zhipu AI
GLM-4.5
Zhipu AI
0.729355B131K
$0.40
$1.60
15
NVIDIA
0.7119B——
16
Zhipu AI
GLM-4.5-Air
Zhipu AI
0.707106B——
16
Alibaba Cloud / Qwen Team
Qwen3 235B A22B
Alibaba Cloud / Qwen Team
0.707235B128K
$0.10
$0.10
18
Google
0.690—1.0M
$1.25
$10.00
19
Inception
Mercury 2
Inception
0.670—128K
$0.25
$0.75
20
NVIDIA
0.663253B——
21
Alibaba Cloud / Qwen Team
Qwen3 32B
Alibaba Cloud / Qwen Team
0.65733B128K
$0.10
$0.30
22
MiniMax
0.650456B1.0M
$0.55
$2.20
23
Mistral AI
0.64614B262K
$0.20
$0.20
24
Mistral AI
Mistral Small 4New
Mistral AI
0.636119B256K
$0.15
$0.60
25
Alibaba Cloud / Qwen Team
QwQ-32B
Alibaba Cloud / Qwen Team
0.63433B——
26
Alibaba Cloud / Qwen Team
Qwen3 30B A3B
Alibaba Cloud / Qwen Team
0.62631B128K
$0.10
$0.30
27
MiniMax
0.623456B——
28
Mistral AI
0.6168B262K
$0.15
$0.15
29
DeepSeek
0.57571B128K
$0.10
$0.40
30
DeepSeek
0.57233B128K
$0.12
$0.18
31
DeepSeek
0.564671B164K
$0.27
$1.00
32
Alibaba Cloud / Qwen Team
Qwen2.5 72B Instruct
Alibaba Cloud / Qwen Team
0.55573B131K
$0.35
$0.40
33
Mistral AI
0.5483B131K
$0.10
$0.10
34
Microsoft
0.53814B——
35
Moonshot AI
0.5371.0T——
36
Microsoft
0.53114B——
36
DeepSeek
0.53115B——
38
Mistral AI
0.51324B——
39
Mistral AI
0.50324B——
40
Alibaba Cloud / Qwen Team
QwQ-32B-Preview
Alibaba Cloud / Qwen Team
0.50033B33K
$0.15
$0.60
40
DeepSeek
0.500671B——
42
DeepSeek
0.492671B164K
$0.28
$1.14
43
Meituan
0.480560B128K
$0.30
$1.20
44
Meta
0.434400B1.0M
$0.17
$0.60
45
DeepSeek
0.3968B——
46
DeepSeek
DeepSeek-V3
DeepSeek
0.376671B131K
$0.27
$1.10
46
DeepSeek
0.3768B——
48
Google
0.351—1.0M
$0.10
$0.40
49
Mistral AI
0.344675B——
49
Mistral AI
0.344675B——
Showing 1-50 of 68
1 / 2
Notice missing or incorrect data?Start an Issue discussion→

FAQ

Common questions about LiveCodeBench

LiveCodeBench is a holistic and contamination-free evaluation benchmark for large language models for code. It continuously collects new problems from programming contests (LeetCode, AtCoder, CodeForces) and evaluates four different scenarios: code generation, self-repair, code execution, and test output prediction. Problems are annotated with release dates to enable evaluation on unseen problems released after a model's training cutoff.
The LiveCodeBench paper is available at https://arxiv.org/abs/2403.07974. This paper provides detailed information about the benchmark methodology, dataset creation, and evaluation criteria.
The LiveCodeBench leaderboard ranks 68 AI models based on their performance on this benchmark. Currently, DeepSeek-V3.2 (Thinking) by DeepSeek leads with a score of 0.833. The average score across all models is 0.514.
The highest LiveCodeBench score is 0.833, achieved by DeepSeek-V3.2 (Thinking) from DeepSeek.
68 models have been evaluated on the LiveCodeBench benchmark, with 0 verified results and 68 self-reported results.
LiveCodeBench is categorized under code, general, and reasoning. The benchmark evaluates text models.