MMLU-Pro

A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.

MMLU-Pro Leaderboard

AllOpenProprietary

115 models

#	Model	Score	Size	Context	Cost	License
1	Qwen3.6 Plus Alibaba Cloud / Qwen Team	0.885	—	—	—
2	MiniMax M2.1 MiniMax	0.880	230B	1.0M	$0.30 / $1.20
3	Qwen3.5-397B-A17B Alibaba Cloud / Qwen Team	0.878	397B	262K	$0.60 / $3.60
4	Kimi K2.5 Moonshot AI	0.871	1.0T	262K	$0.60 / $3.00
5	ERNIE 5.0 Baidu	0.870	—	—	—
6	Qwen3.5-122B-A10B Alibaba Cloud / Qwen Team	0.867	122B	262K	$0.40 / $3.20
7	Qwen3.5-27B Alibaba Cloud / Qwen Team	0.861	27B	262K	$0.30 / $2.40
8	Qwen3.5-35B-A3B Alibaba Cloud / Qwen Team	0.853	35B	262K	$0.25 / $2.00
9	Gemma 4 31B Google	0.852	31B	262K	$0.14 / $0.40
10	DeepSeek-V3.2 DeepSeek	0.850	685B	164K	$0.26 / $0.38
10	DeepSeek-V3.2 (Thinking) DeepSeek	0.850	685B	—	—
10	DeepSeek-R1-0528 DeepSeek	0.850	671B	131K	$0.55 / $2.19
10	DeepSeek-V3.2-Exp DeepSeek	0.850	685B	—	—
14	MiMo-V2-Flash Xiaomi	0.849	309B	256K	$0.10 / $0.30
15	GLM-4.5 Zhipu AI	0.846	355B	—	—
15	Kimi K2-Thinking-0905 Moonshot AI	0.846	1.0T	—	—
17	Qwen3-235B-A22B-Thinking-2507 Alibaba Cloud / Qwen Team	0.844	235B	262K	$0.30 / $3.00
18	GLM-4.7 Zhipu AI	0.843	358B	205K	$0.60 / $2.20
19	K-EXAONE-236B-A23B LG AI Research	0.838	236B	33K	$0.60 / $1.00
19	Qwen3 VL 235B A22B Thinking Alibaba Cloud / Qwen Team	0.838	236B	262K	$0.45 / $3.49
21	Nemotron 3 Super (120B A12B) NVIDIA	0.837	120B	—	—
22	DeepSeek-V3.1 DeepSeek	0.837	671B	164K	$0.27 / $1.00
23	Qwen3-235B-A22B-Instruct-2507 Alibaba Cloud / Qwen Team	0.830	235B	262K	$0.15 / $0.80
24	Qwen3-Next-80B-A3B-Thinking Alibaba Cloud / Qwen Team	0.827	80B	66K	$0.15 / $1.50
25	LongCat-Flash-Chat Meituan	0.827	560B	128K	$0.30 / $1.20
26	LongCat-Flash-Thinking Meituan	0.826	560B	—	—
26	Gemma 4 26B-A4B Google	0.826	25B	262K	$0.13 / $0.40
28	Qwen3.5-9B Alibaba Cloud / Qwen Team	0.825	9B	—	—
28	Kimi K2 0905 Moonshot AI	0.825	1.0T	262K	$0.60 / $2.50
30	Qwen3 VL 32B Thinking Alibaba Cloud / Qwen Team	0.821	33B	—	—
31	MiniMax M2 MiniMax	0.820	230B	1.0M	$0.30 / $1.20
32	Qwen3 VL 235B A22B Instruct Alibaba Cloud / Qwen Team	0.818	236B	262K	$0.30 / $1.50
33	Sarvam-105B Sarvam AI	0.817	105B	—	—
34	GLM-4.5-Air Zhipu AI	0.814	106B	—	—
35	DeepSeek-V3 0324 DeepSeek	0.812	671B	164K	$0.28 / $1.14
36	MiniMax M1 80K MiniMax	0.811	456B	1.0M	$0.55 / $2.20
36	Kimi K2 Instruct Moonshot AI	0.811	1.0T	200K	$0.50 / $0.50
36	Kimi K2-Instruct-0905 Moonshot AI	0.811	1.0T	—	—
39	GPT OSS 120B High OpenAI	0.807	117B	131K	$0.10 / $0.50
40	Qwen3-Next-80B-A3B-Instruct Alibaba Cloud / Qwen Team	0.806	80B	66K	$0.15 / $1.50
40	MiniMax M1 40K MiniMax	0.806	456B	—	—
42	Qwen3 VL 30B A3B Thinking Alibaba Cloud / Qwen Team	0.805	31B	262K	$0.20 / $1.00
42	Llama 4 Maverick Meta	0.805	400B	1.0M	$0.17 / $0.85
44	Sarvam-30B Sarvam AI	0.800	30B	—	—
45	Qwen3.5-4B Alibaba Cloud / Qwen Team	0.791	4B	—	—
46	Qwen3 VL 32B Instruct Alibaba Cloud / Qwen Team	0.786	33B	—	—
47	Nemotron 3 Nano (30B A3B) NVIDIA	0.783	32B	262K	$0.06 / $0.24
48	LongCat-Flash-Lite Meituan	0.783	69B	256K	$0.10 / $0.40
49	Mistral Small 4 Mistral AI	0.780	119B	256K	$0.15 / $0.60
50	Qwen3 VL 30B A3B Instruct Alibaba Cloud / Qwen Team	0.778	31B	262K	$0.20 / $0.70

1–50 of 115

Prev1/3Next