InfographicsQA
Progress Over Time
Interactive timeline showing model performance evolution on InfographicsQA
InfographicsQA Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 | 90B | — | — |
What is InfographicsQA?
InfographicVQA dataset with 5,485 infographic images and over 30,000 questions requiring joint reasoning over document layout, textual content, graphical elements, and data visualizations with elementary reasoning and arithmetic skills
InfographicsQA is a multimodal benchmark evaluating models on multimodal and vision tasks. LLM Stats tracks 1 models on this benchmark, scored on a 0–1 scale. The current average is 0.6, with the leader at 0.6.
Compare leaders on the best AI for multimodal and best AI for vision leaderboards.
Current leaders
Llama 3.2 90B Instruct from Meta currently leads the InfographicsQA leaderboard with a score of 0.568 across 1 evaluated AI models.
Source paper
- Title
- InfographicVQA
- Authors
- Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, and 2 others
- Published
- arXiv
- 2104.12756
Abstract
Infographics are documents designed to effectively communicate information using a combination of textual, graphical and visual elements. In this work, we explore the automatic understanding of infographic images by using Visual Question Answering technique.To this end, we present InfographicVQA, a new dataset that comprises a diverse collection of infographics along with natural language questions and answers annotations. The collected questions require methods to jointly reason over the document layout, textual content, graphical elements, and data visualizations. We curate the dataset with emphasis on questions that require elementary reasoning and basic arithmetic skills. Finally, we evaluate two strong baselines based on state of the art multi-modal VQA models, and establish baseline performance for the new task. The dataset, code and leaderboard will be made available at http://docvqa.org
FAQ
Common questions about the InfographicsQA benchmark and leaderboard.