TriviaQA
Progress Over Time
Interactive timeline showing model performance evolution on TriviaQA
TriviaQA Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Moonshot AI | 1.0T | — | — | ||
| 2 |  Google | 27B | — | — | ||
| 3 |  Xiaomi | 1.0T | 1.0M | $0.43 / $0.87 | ||
| 4 |  Mistral AI | 24B | — | — | ||
| 4 | Mistral AI | 24B | — | — | ||
| 6 | Mistral AI | 24B | — | — | ||
| 7 | 8B | — | — | |||
| 8 | Google | 9B | — | — | ||
| 9 | Mistral AI | 675B | — | — | ||
| 9 | Mistral AI | 14B | — | — | ||
| 11 | Mistral AI | 12B | — | — | ||
| 12 | 2B | — | — | |||
| 12 | Google | 8B | — | — | ||
| 14 | Mistral AI | 8B | — | — | ||
| 15 | Mistral AI | 8B | — | — | ||
| 16 | Google | 8B | — | — | ||
| 16 | 2B | — | — | |||
| 18 | Mistral AI | 3B | — | — |
What is TriviaQA?
A large-scale reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents (six per question on average) that provide high quality distant supervision for answering the questions. The dataset features relatively complex, compositional questions with considerable syntactic and lexical variability, requiring cross-sentence reasoning to find answers.
TriviaQA is a text benchmark evaluating models on reasoning and general tasks. LLM Stats tracks 18 models on this benchmark, scored on a 0–1 scale. The current average is 0.7, with the leader at 0.9.
Compare leaders on the best AI for reasoning and best AI for general leaderboards.
Current leaders
Kimi K2 Base from Moonshot AI currently leads the TriviaQA leaderboard with a score of 0.851 across 18 evaluated AI models.
Source paper
- Title
- TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
- Authors
- Mandar Joshi, Eunsol Choi, Daniel S. Weld, Luke Zettlemoyer
- Published
- arXiv
- 1705.03551
Abstract
We present TriviaQA, a challenging reading comprehension dataset containing over 650K question-answer-evidence triples. TriviaQA includes 95K question-answer pairs authored by trivia enthusiasts and independently gathered evidence documents, six per question on average, that provide high quality distant supervision for answering the questions. We show that, in comparison to other recently introduced large-scale datasets, TriviaQA (1) has relatively complex, compositional questions, (2) has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and (3) requires more cross sentence reasoning to find answers. We also present two baseline algorithms: a feature-based classifier and a state-of-the-art neural network, that performs well on SQuAD reading comprehension. Neither approach comes close to human performance (23% and 40% vs. 80%), suggesting that TriviaQA is a challenging testbed that is worth significant future study. Data and code available at -- http://nlp.cs.washington.edu/triviaqa/
FAQ
Common questions about the TriviaQA benchmark and leaderboard.