MMLU-Pro

A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.

MMLU-Pro Leaderboard

AllOpenProprietary

116 models

#	Model	Score	Size	Context	Cost	License
1	Qwen3.6 Plus Alibaba Cloud / Qwen Team	0.885	—	—	—
2	MiniMax M2.1 MiniMax	0.880	230B	1.0M	$0.30 / $1.20
3	Qwen3.5-397B-A17B Alibaba Cloud / Qwen Team	0.878	397B	262K	$0.60 / $3.60
4	Kimi K2.5 Moonshot AI	0.871	1.0T	262K	$0.60 / $3.00
5	ERNIE 5.0 Baidu	0.870	—	—	—
6	Qwen3.5-122B-A10B Alibaba Cloud / Qwen Team	0.867	122B	262K	$0.40 / $3.20
7	Qwen3.5-27B Alibaba Cloud / Qwen Team	0.861	27B	262K	$0.30 / $2.40
8	Qwen3.5-35B-A3B Alibaba Cloud / Qwen Team	0.853	35B	262K	$0.25 / $2.00
9	Gemma 4 31B Google	0.852	31B	262K	$0.14 / $0.40
9	Qwen3.6-35B-A3BNew Alibaba Cloud / Qwen Team	0.852	35B	—	—
11	DeepSeek-V3.2 (Thinking) DeepSeek	0.850	685B	—	—
11	DeepSeek-R1-0528 DeepSeek	0.850	671B	131K	$0.55 / $2.19
11	DeepSeek-V3.2 DeepSeek	0.850	685B	164K	$0.26 / $0.38
11	DeepSeek-V3.2-Exp DeepSeek	0.850	685B	—	—
15	MiMo-V2-Flash Xiaomi	0.849	309B	256K	$0.10 / $0.30
16	GLM-4.5 Zhipu AI	0.846	355B	—	—
16	Kimi K2-Thinking-0905 Moonshot AI	0.846	1.0T	—	—
18	Qwen3-235B-A22B-Thinking-2507 Alibaba Cloud / Qwen Team	0.844	235B	262K	$0.30 / $3.00
19	GLM-4.7 Zhipu AI	0.843	358B	205K	$0.60 / $2.20
20	K-EXAONE-236B-A23B LG AI Research	0.838	236B	33K	$0.60 / $1.00
20	Qwen3 VL 235B A22B Thinking Alibaba Cloud / Qwen Team	0.838	236B	262K	$0.45 / $3.49
22	Nemotron 3 Super (120B A12B) NVIDIA	0.837	120B	—	—
23	DeepSeek-V3.1 DeepSeek	0.837	671B	164K	$0.27 / $1.00
24	Qwen3-235B-A22B-Instruct-2507 Alibaba Cloud / Qwen Team	0.830	235B	262K	$0.15 / $0.80
25	Qwen3-Next-80B-A3B-Thinking Alibaba Cloud / Qwen Team	0.827	80B	66K	$0.15 / $1.50
26	LongCat-Flash-Chat Meituan	0.827	560B	128K	$0.30 / $1.20
27	LongCat-Flash-Thinking Meituan	0.826	560B	—	—
27	Gemma 4 26B-A4B Google	0.826	25B	262K	$0.13 / $0.40
29	Kimi K2 0905 Moonshot AI	0.825	1.0T	262K	$0.60 / $2.50
29	Qwen3.5-9B Alibaba Cloud / Qwen Team	0.825	9B	—	—
31	Qwen3 VL 32B Thinking Alibaba Cloud / Qwen Team	0.821	33B	—	—
32	MiniMax M2 MiniMax	0.820	230B	1.0M	$0.30 / $1.20
33	Qwen3 VL 235B A22B Instruct Alibaba Cloud / Qwen Team	0.818	236B	262K	$0.30 / $1.50
34	Sarvam-105B Sarvam AI	0.817	105B	—	—
35	GLM-4.5-Air Zhipu AI	0.814	106B	—	—
36	DeepSeek-V3 0324 DeepSeek	0.812	671B	164K	$0.28 / $1.14
37	Kimi K2-Instruct-0905 Moonshot AI	0.811	1.0T	—	—
37	MiniMax M1 80K MiniMax	0.811	456B	1.0M	$0.55 / $2.20
37	Kimi K2 Instruct Moonshot AI	0.811	1.0T	200K	$0.50 / $0.50
40	GPT OSS 120B High OpenAI	0.807	117B	131K	$0.10 / $0.50
41	Qwen3-Next-80B-A3B-Instruct Alibaba Cloud / Qwen Team	0.806	80B	66K	$0.15 / $1.50
41	MiniMax M1 40K MiniMax	0.806	456B	—	—
43	Llama 4 Maverick Meta	0.805	400B	1.0M	$0.17 / $0.60
43	Qwen3 VL 30B A3B Thinking Alibaba Cloud / Qwen Team	0.805	31B	262K	$0.20 / $1.00
45	Sarvam-30B Sarvam AI	0.800	30B	—	—
46	Qwen3.5-4B Alibaba Cloud / Qwen Team	0.791	4B	—	—
47	Qwen3 VL 32B Instruct Alibaba Cloud / Qwen Team	0.786	33B	—	—
48	Nemotron 3 Nano (30B A3B) NVIDIA	0.783	32B	262K	$0.06 / $0.24
49	LongCat-Flash-Lite Meituan	0.783	69B	256K	$0.10 / $0.40
50	Mistral Small 4 Mistral AI	0.780	119B	256K	$0.15 / $0.60

1–50 of 116

Prev1/3Next