InfoVQAtest
Progress Over Time
Interactive timeline showing model performance evolution on InfoVQAtest
InfoVQAtest Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Moonshot AI | 1.0T | — | — | ||
| 2 |  Alibaba Cloud / Qwen Team | 236B | — | — | ||
| 3 | Alibaba Cloud / Qwen Team | 33B | — | — | ||
| 3 | Alibaba Cloud / Qwen Team | 236B | — | — | ||
| 5 | Alibaba Cloud / Qwen Team | 33B | — | — | ||
| 6 | Alibaba Cloud / Qwen Team | 9B | — | — | ||
| 6 | Alibaba Cloud / Qwen Team | 31B | — | — | ||
| 8 | Alibaba Cloud / Qwen Team | 73B | — | — | ||
| 9 | Alibaba Cloud / Qwen Team | 9B | — | — | ||
| 10 | Alibaba Cloud / Qwen Team | 4B | 262K | $0.10 / $1.00 | ||
| 11 | Alibaba Cloud / Qwen Team | 31B | — | — | ||
| 12 | Alibaba Cloud / Qwen Team | 4B | 262K | $0.10 / $0.60 |
What is InfoVQAtest?
InfoVQA test set with infographic images requiring joint reasoning over document layout, textual content, graphical elements, and data visualizations with elementary reasoning and arithmetic skills
InfoVQAtest is a multimodal benchmark evaluating models on multimodal and vision tasks. LLM Stats tracks 12 models on this benchmark, scored on a 0–1 scale. The current average is 0.9, with the leader at 0.9.
Compare leaders on the best AI for multimodal and best AI for vision leaderboards.
Current leaders
Kimi K2.5 from Moonshot AI currently leads the InfoVQAtest leaderboard with a score of 0.926 across 12 evaluated AI models.
Source paper
- Title
- InfographicVQA
- Authors
- Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, and 2 others
- Published
- arXiv
- 2104.12756
Abstract
Infographics are documents designed to effectively communicate information using a combination of textual, graphical and visual elements. In this work, we explore the automatic understanding of infographic images by using Visual Question Answering technique.To this end, we present InfographicVQA, a new dataset that comprises a diverse collection of infographics along with natural language questions and answers annotations. The collected questions require methods to jointly reason over the document layout, textual content, graphical elements, and data visualizations. We curate the dataset with emphasis on questions that require elementary reasoning and basic arithmetic skills. Finally, we evaluate two strong baselines based on state of the art multi-modal VQA models, and establish baseline performance for the new task. The dataset, code and leaderboard will be made available at http://docvqa.org
FAQ
Common questions about the InfoVQAtest benchmark and leaderboard.