CLUEWSC
Progress Over Time
Interactive timeline showing model performance evolution on CLUEWSC
CLUEWSC Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Moonshot AI | — | — | — | ||
| 2 |  DeepSeek | 671B | — | — | ||
| 3 |  Baidu | 21B | — | — |
What is CLUEWSC?
CLUEWSC2020 is the Chinese version of the Winograd Schema Challenge, part of the CLUE benchmark. It focuses on pronoun disambiguation and coreference resolution, requiring models to determine which noun a pronoun refers to in a sentence. The dataset contains 1,244 training samples and 304 development samples extracted from contemporary Chinese literature.
CLUEWSC is a text benchmark evaluating models on language and reasoning tasks. LLM Stats tracks 3 models on this benchmark, scored on a 0–1 scale. The current average is 0.8, with the leader at 0.9.
Compare leaders on the best AI for language and best AI for reasoning leaderboards.
Current leaders
Kimi-k1.5 from Moonshot AI currently leads the CLUEWSC leaderboard with a score of 0.914 across 3 evaluated AI models.
Source paper
- Title
- CLUE: A Chinese Language Understanding Evaluation Benchmark
- Authors
- Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, and 28 others
- Published
- arXiv
- 2004.05986
Abstract
The advent of natural language understanding (NLU) benchmarks for English, such as GLUE and SuperGLUE allows new NLU models to be evaluated across a diverse set of tasks. These comprehensive benchmarks have facilitated a broad range of research and applications in natural language processing (NLP). The problem, however, is that most such benchmarks are limited to English, which has made it difficult to replicate many of the successes in English NLU for other languages. To help remedy this issue, we introduce the first large-scale Chinese Language Understanding Evaluation (CLUE) benchmark. CLUE is an open-ended, community-driven project that brings together 9 tasks spanning several well-established single-sentence/sentence-pair classification tasks, as well as machine reading comprehension, all on original Chinese text. To establish results on these tasks, we report scores using an exhaustive set of current state-of-the-art pre-trained Chinese models (9 in total). We also introduce a number of supplementary datasets and additional tools to help facilitate further progress on Chinese NLU. Our benchmark is released at https://www.CLUEbenchmarks.com
FAQ
Common questions about the CLUEWSC benchmark and leaderboard.