CRAG
Progress Over Time
Interactive timeline showing model performance evolution on CRAG
CRAG Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Amazon | — | — | — | ||
| 2 | Amazon | — | — | — | ||
| 3 | Amazon | — | — | — |
What is CRAG?
CRAG (Comprehensive RAG Benchmark) is a factual question answering benchmark consisting of 4,409 question-answer pairs across 5 domains (finance, sports, music, movie, open domain) and 8 question categories. The benchmark includes mock APIs to simulate web and Knowledge Graph search, designed to represent the diverse and dynamic nature of real-world QA tasks with temporal dynamism ranging from years to seconds. It evaluates retrieval-augmented generation systems for trustworthy question answering.
CRAG is a text benchmark evaluating models on reasoning, search, finance, and economics tasks. LLM Stats tracks 3 models on this benchmark, scored on a 0–1 scale. The current average is 0.5, with the leader at 0.5.
Compare leaders on the best AI for reasoning, best AI for search, best AI for finance and best AI for economics leaderboards.
Current leaders
Nova Pro from Amazon currently leads the CRAG leaderboard with a score of 0.503 across 3 evaluated AI models.
FAQ
Common questions about the CRAG benchmark and leaderboard.