MathVista-Mini
MathVista-Mini is a smaller version of the MathVista benchmark that evaluates mathematical reasoning in visual contexts. It consists of examples derived from multimodal datasets involving mathematics, combining challenges from diverse mathematical and visual tasks to assess foundation models' ability to solve problems requiring both visual understanding and mathematical reasoning.
Kimi K2.5 from Moonshot AI currently leads the MathVista-Mini leaderboard with a score of 0.901 across 23 evaluated AI models.
What MathVista-Mini measures
MathVista-Mini is a multimodal benchmark that evaluates large language models on math, multimodal, and vision tasks. LLM Stats tracks 23 models on this benchmark, with a maximum possible score of 1. Current average across reported models is 0.8, with the leader reaching 0.9.
Compare leaders on the best AI for math, best AI for multimodal and best AI for vision leaderboards.
Publication
- Paper
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts
- Authors
- Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, and 6 others
- Published
- arXiv
- 2310.02255
Abstract
Large Language Models (LLMs) and Large Multimodal Models (LMMs) exhibit impressive problem-solving skills in many tasks and domains, but their ability in mathematical reasoning in visual contexts has not been systematically studied. To bridge this gap, we present MathVista, a benchmark designed to combine challenges from diverse mathematical and visual tasks. It consists of 6,141 examples, derived from 28 existing multimodal datasets involving mathematics and 3 newly created datasets (i.e., IQTest, FunctionQA, and PaperQA). Completing these tasks requires fine-grained, deep visual understanding and compositional reasoning, which all state-of-the-art foundation models find challenging. With MathVista, we have conducted a comprehensive, quantitative evaluation of 12 prominent foundation models. The best-performing GPT-4V model achieves an overall accuracy of 49.9%, substantially outperforming Bard, the second-best performer, by 15.1%. Our in-depth analysis reveals that the superiority of GPT-4V is mainly attributed to its enhanced visual perception and mathematical reasoning. However, GPT-4V still falls short of human performance by 10.4%, as it often struggles to understand complex figures and perform rigorous reasoning. This significant gap underscores the critical role that MathVista will play in the development of general-purpose AI agents capable of tackling mathematically intensive and visually rich real-world tasks. We further explore the new ability of self-verification, the application of self-consistency, and the interactive chatbot capabilities of GPT-4V, highlighting its promising potential for future research. The project is available at https://mathvista.github.io/.
Kimi K2.5 leads with 90.1%, followed by 
Qwen3.5-27B at 87.8% and Qwen3.5-122B-A10B at 87.4%.
Progress Over Time
Interactive timeline showing model performance evolution on MathVista-Mini
MathVista-Mini Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 | Moonshot AI | 1.0T | — | — | ||
| 2 |  Alibaba Cloud / Qwen Team | 27B | 262K | $0.30 / $2.40 | ||
| 3 | Alibaba Cloud / Qwen Team | 122B | 262K | $0.40 / $3.20 | ||
| 3 | Alibaba Cloud / Qwen Team | 28B | 262K | $0.60 / $3.60 | ||
| 5 | Alibaba Cloud / Qwen Team | 35B | — | — | ||
| 6 | Alibaba Cloud / Qwen Team | 35B | 262K | $0.25 / $2.00 | ||
| 7 | Alibaba Cloud / Qwen Team | 33B | — | — | ||
| 8 | Alibaba Cloud / Qwen Team | 236B | — | — | ||
| 9 | Alibaba Cloud / Qwen Team | 236B | 262K | $0.30 / $1.50 | ||
| 10 | Alibaba Cloud / Qwen Team | 33B | — | — | ||
| 11 | Alibaba Cloud / Qwen Team | 31B | — | — | ||
| 12 | Alibaba Cloud / Qwen Team | 9B | 262K | $0.18 / $2.09 | ||
| 13 | Alibaba Cloud / Qwen Team | 31B | — | — | ||
| 14 | Alibaba Cloud / Qwen Team | 4B | 262K | $0.10 / $1.00 | ||
| 15 | Alibaba Cloud / Qwen Team | 9B | 262K | $0.08 / $0.50 | ||
| 16 | Alibaba Cloud / Qwen Team | 72B | — | — | ||
| 17 | Alibaba Cloud / Qwen Team | 34B | — | — | ||
| 18 | Alibaba Cloud / Qwen Team | 4B | 262K | $0.10 / $0.60 | ||
| 19 | Alibaba Cloud / Qwen Team | 73B | — | — | ||
| 20 | Alibaba Cloud / Qwen Team | 8B | — | — | ||
| 21 |  Google | 27B | — | — | ||
| 22 | Google | 12B | — | — | ||
| 23 | Google | 4B | — | — |
FAQ
Common questions about MathVista-Mini.
More evaluations to explore
Related benchmarks in the same category
A more robust and challenging multi-task language understanding benchmark that extends MMLU by expanding multiple-choice options from 4 to 10, eliminating trivial questions, and focusing on reasoning-intensive tasks. Features over 12,000 curated questions across 14 domains and causes a 16-33% accuracy drop compared to original MMLU.
All 30 problems from the 2025 American Invitational Mathematics Examination (AIME I and AIME II), testing olympiad-level mathematical reasoning with integer answers from 000-999. Used as an AI benchmark to evaluate large language models' ability to solve complex mathematical problems requiring multi-step logical deductions and structured symbolic reasoning.
Massive Multitask Language Understanding benchmark testing knowledge across 57 diverse subjects including STEM, humanities, social sciences, and professional domains
Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions
MATH dataset contains 12,500 challenging competition mathematics problems from AMC 10, AMC 12, AIME, and other mathematics competitions. Each problem includes full step-by-step solutions and spans multiple difficulty levels (1-5) across seven mathematical subjects including Prealgebra, Algebra, Number Theory, Counting and Probability, Geometry, Intermediate Algebra, and Precalculus.
MMMU (Massive Multi-discipline Multimodal Understanding) is a benchmark designed to evaluate multimodal models on college-level subject knowledge and deliberate reasoning. Contains 11.5K meticulously collected multimodal questions from college exams, quizzes, and textbooks, covering six core disciplines: Art & Design, Business, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering across 30 subjects and 183 subfields.