Chartography
Progress Over Time
Interactive timeline showing model performance evolution on Chartography
Chartography Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Anthropic | — | 1.0M | $2.00 / $10.00 | ||
| 2 | Anthropic | — | 1.0M | $4.00 / $20.00 | ||
| 3 | Anthropic | — | 1.0M | $0.10 / $0.50 | ||
| 4 |  DeepSeek | 763B | 1.0M | $0.30 / $1.20 | ||
| 5 | DeepSeek | 763B | 1.0M | $0.22 / $0.66 | ||
| 6 |  Zhipu AI | 320B | 1.0M | $0.15 / $0.50 |
Sub-benchmarks
What is Chartography?
Chartography evaluates professional chart understanding with 100 expert-written tasks drawn from real charts across science, engineering, finance, manufacturing, supply chain, and healthcare.
Chartography is a multimodal benchmark evaluating models on multimodal, reasoning, and vision tasks. LLM Stats tracks 6 models on this benchmark, scored on a 0–1 scale. The current average is 0.8, with the leader at 0.9.
Compare leaders on the best AI for multimodal, best AI for reasoning and best AI for vision leaderboards.
Current leaders
Claude Sonnet 5.5 from Anthropic currently leads the Chartography leaderboard with a score of 0.902 across 6 evaluated AI models.
Source paper
- Title
- Chartography: A Benchmark for Professional Chart Understanding
- Authors
- Suhaas Garre, Chris Mutty, Sushant Mehta, Edwin Chen
- Published
- arXiv
- 2608.10677
Abstract
Professionals across medicine, engineering, finance, manufacturing, and the sciences often make consequential decisions from charts. Existing chart benchmarks do not sufficiently measure this ability: they are dominated by bar, line, and pie formats, rely on shorter reasoning chains, and are nearing saturation, with frontier models already scoring 80-90%. We introduce Chartography, a benchmark of 100 tasks that pair charts drawn from professional practice, in domain-specific formats that standard chart benchmarks rarely include, with questions written by professionals who read these charts for a living and independently verified by three additional experts. In an evaluation of 30 frontier-model configurations (20 scored trials per task), the best configuration reaches only 45.0% mean pass@1; the remainder span 9.0-39.5%. Failures concentrate in visual perception: models can miss nuanced features, misread values along sparsely labeled axes, mishandle projected 3D geometry, and violate domain conventions encoded in the chart. We release all tasks, images, provenance metadata, and evaluation code.
FAQ
Common questions about the Chartography benchmark and leaderboard.