DocVQA
Progress Over Time
Interactive timeline showing model performance evolution on DocVQA
DocVQA Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Liquid AI | 2B | — | — | ||
| 2 |  Alibaba Cloud / Qwen Team | 72B | — | — | ||
| 3 | Alibaba Cloud / Qwen Team | 8B | — | — | ||
| 4 |  Anthropic | — | — | — | ||
| 4 | Alibaba Cloud / Qwen Team | 7B | — | — | ||
| 6 |  Mistral AI | 24B | 128K | $0.07 / $0.20 | ||
| 7 | Alibaba Cloud / Qwen Team | 34B | — | — | ||
| 8 |  Meta | 109B | 328K | $0.10 / $0.30 | ||
| 8 | Meta | 400B | 1.0M | $0.20 / $0.80 | ||
| 10 |  xAI | — | — | — | ||
| 11 |  Amazon | — | — | — | ||
| 12 |  DeepSeek | 27B | — | — | ||
| 12 | Mistral AI | 124B | — | — | ||
| 14 |  Microsoft | 6B | — | — | ||
| 14 | xAI | — | — | — | ||
| 16 |  OpenAI | — | 128K | $2.50 / $10.00 | ||
| 17 | Amazon | — | — | — | ||
| 18 | DeepSeek | 16B | — | — | ||
| 19 | 2B | — | — | |||
| 20 | Liquid AI | 3B | — | — | ||
| 21 | Mistral AI | 12B | — | — | ||
| 22 | 90B | — | — | |||
| 23 | DeepSeek | 3B | — | — | ||
| 24 | 11B | — | — | |||
| 25 |  Google | 12B | 131K | $0.05 / $0.15 | ||
| 26 | Google | 27B | 131K | $0.08 / $0.16 | ||
| 27 | xAI | — | — | — | ||
| 27 | xAI | — | — | — | ||
| 29 | Google | 4B | 131K | $0.05 / $0.10 |
What is DocVQA?
A dataset for Visual Question Answering on document images containing 50,000 questions defined on 12,000+ document images. The benchmark tests AI's ability to understand document structure and content, requiring models to comprehend document layout and perform information retrieval to answer questions about document images.
DocVQA is a multimodal benchmark evaluating models on image to text, multimodal, and vision tasks. LLM Stats tracks 29 models on this benchmark, scored on a 0–1 scale. The current average is 0.9, with the leader at 1.0.
Compare leaders on the best AI for image to text, best AI for multimodal and best AI for vision leaderboards.
Current leaders
LFM3-VL-2B from Liquid AI currently leads the DocVQA leaderboard with a score of 0.967 across 29 evaluated AI models.
Source paper
- Title
- DocVQA: A Dataset for VQA on Document Images
- Authors
- Minesh Mathew, Dimosthenis Karatzas, C. V. Jawahar
- Published
- arXiv
- 2007.00398
Abstract
We present a new dataset for Visual Question Answering (VQA) on document images called DocVQA. The dataset consists of 50,000 questions defined on 12,000+ document images. Detailed analysis of the dataset in comparison with similar datasets for VQA and reading comprehension is presented. We report several baseline results by adopting existing VQA and reading comprehension models. Although the existing models perform reasonably well on certain types of questions, there is large performance gap compared to human performance (94.36% accuracy). The models need to improve specifically on questions where understanding structure of the document is crucial. The dataset, code and leaderboard are available at docvqa.org
FAQ
Common questions about the DocVQA benchmark and leaderboard.