CMMLU
Progress Over Time
Interactive timeline showing model performance evolution on CMMLU
CMMLU Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Xiaomi | 1.0T | 1.0M | $0.43 / $0.87 | ||
| 2 |  Alibaba Cloud / Qwen Team | 72B | — | — | ||
| 3 |  Meituan | 560B | — | — | ||
| 4 | Meituan | 69B | 256K | $0.10 / $0.40 | ||
| 5 |  OpenBMB | 9B | — | — | ||
| 6 |  Baidu | 21B | — | — |
What is CMMLU?
CMMLU (Chinese Massive Multitask Language Understanding) is a comprehensive Chinese benchmark that evaluates the knowledge and reasoning capabilities of large language models across 67 different subject topics. The benchmark covers natural sciences, social sciences, engineering, and humanities with multiple-choice questions ranging from basic to advanced professional levels.
CMMLU is a text benchmark evaluating models on language, reasoning, and general tasks. LLM Stats tracks 6 models on this benchmark, scored on a 0–1 scale. The current average is 0.8, with the leader at 0.9.
Compare leaders on the best AI for language, best AI for reasoning and best AI for general leaderboards.
Current leaders
MiMo-V2.5-Pro from Xiaomi currently leads the CMMLU leaderboard with a score of 0.902 across 6 evaluated AI models.
FAQ
Common questions about the CMMLU benchmark and leaderboard.