- Organizations
- Artificial Analysis
Artificial Analysis: AI benchmarks & research
Independent AI model evaluator. Maintains the Artificial Analysis Intelligence Index and its own runs of third-party benchmarks.
Maintained by Artificial Analysis
23 benchmarks- AA-Briefcase
AA-Briefcase is an Artificial Analysis evaluation of AI systems on professional knowledge-work tasks, reported as an Elo score.
- Leader
Grok 4.61577
- Models scored
- 5
- AA-Briefcase v1.1
AA-Briefcase v1.1 evaluates multi-hour professional office work and reports Elo scores. Kept separate from unversioned AA-Briefcase results.
- Leader
Claude Opus 5.51822
- Models scored
- 7
- AA Controlled Voice Arena
Artificial Analysis Controlled Voice Arena ranks text-to-speech models with Elo ratings from blind pairwise human preference votes using the same set of cloned voices across models (English US and UK).
- Leader
Sonic 3.61119- Models scored
- 1
- AA-Index
No official academic documentation found for this benchmark. Extensive research through ArXiv, IEEE/ACL/NeurIPS papers, and university research sites yielded no peer-reviewed sources for an 'aa-index' benchmark. This entry requires verification from official academic sources.
- Leader
GLM-4.567.7%
- Models scored
- 4
- AA Intelligence Index v4.3
Version 4.3 of the Artificial Analysis Intelligence Index. Scores are stored as fractions of 100.
- Leader
Muse Wonder Full56.0%
- Models scored
- 3
- AA-LCR
Agent Arena Long Context Reasoning benchmark
- Leader
Solar Mini 483.3%- Models scored
- 32
- AA-Omniscience Index
AA-Omniscience Index is Artificial Analysis's knowledge-reliability metric. It rewards correct answers, penalizes hallucinations, and does not penalize abstention. Scores range from -100 to 100, where 0 means as many correct as incorrect answers.
- Leader
Grok 4.5126
- Models scored
- 6
- AA Provider Voice Arena
Artificial Analysis Provider Voice Arena ranks text-to-speech models with Elo ratings from blind pairwise human preference votes, using each provider's own native voices.
- Leader
Sonic 3.61288- Models scored
- 1
- Artificial Analysis
Artificial Analysis benchmark evaluates AI models across quality, speed, and pricing dimensions, providing a composite assessment of model capabilities for real-world usage.
- Leader
GPT-5.6 Sol59.0%
- Models scored
- 14
- AutomationBench-AA
AutomationBench-AA is Artificial Analysis's independently run version of AutomationBench, covering 657 real-world SaaS workflow tasks across 40 simulated applications (e.g. Gmail, Slack, Salesforce, HubSpot). It scores the share of objectives an agent completes without violating business guardrails, using a private held-out task set.
- Leader
Kimi K3.1 Pro73.3%
- Models scored
- 6
- Big Bench Audio
Big Bench Audio is an audio reasoning benchmark adapted from a subset of Big Bench Hard, with text questions converted to spoken audio. It evaluates the reasoning ability of speech-to-speech and audio language models on tasks delivered as audio input, with accuracy scored by an independent evaluation (Artificial Analysis).
- Leader
Grok Voice 3.098.5%
- Models scored
- 2
- Coding Agent Index
Coding Agent Index is a combined coding-agent score (as reported by Mistral / Artificial Analysis) summarizing performance across software-engineering agent evaluations such as DeepSWE, SWE-Atlas QnA, and Terminal-Bench.
- Leader
Mistral Large 448.0%
- Models scored
- 1
- GDPval-AA
GDPval-AA evaluates AI agents on economically valuable professional knowledge-work tasks and reports performance as an Elo score.
- Leader
Claude Fable 5.11853
- Models scored
- 18
- GDPval-AA 2.1
Version 2.1 of the Artificial Analysis GDPval evaluation of professional knowledge work, reported as an Elo rating.
- Leader
Kimi K3.1 Pro1846
- Models scored
- 7
- Leader
Gemini 3.7 Flash90.7%
- Models scored
- 5
- OmniScience
OmniScience is a broad scientific knowledge and reasoning benchmark that measures both answer accuracy and non-hallucination (calibrated abstention) across science domains.
- Leader
Nemotron 3 Ultra (550B A55B)78.7%
- Models scored
- 10
- OmniScience (non-hallucination rate)
OmniScience variant that reports the non-hallucination rate, defined as one minus the hallucination rate.
- Leader
K2 Horizon 375B-A23B74.7%- Models scored
- 4
- Pronunciation Robustness
Artificial Analysis Pronunciation Robustness evaluates how often text-to-speech models correctly read hard words in English sentences (numbers, codes, brand names, and words whose reading depends on context).
- Leader
Fish Audio Drama 393.2%- Models scored
- 1
- Pronunciation Robustness (Context)
Artificial Analysis Pronunciation Robustness subcategory: words whose reading depends on context.
- Leader
Fish Audio Drama 395.3%- Models scored
- 1
- Pronunciation Robustness (Exact Sequences)
Artificial Analysis Pronunciation Robustness subcategory: exact sequences such as codes, paths, and emails.
- Leader
Fish Audio Drama 386.3%- Models scored
- 1
- Pronunciation Robustness (Names)
Artificial Analysis Pronunciation Robustness subcategory: names of brands, places and technical terms.
- Leader
Fish Audio Drama 395.9%- Models scored
- 1
- Pronunciation Robustness (Shorthand)
Artificial Analysis Pronunciation Robustness subcategory: shorthand including numbers, dates, units, and notation.
- Leader
Fish Audio Drama 393.6%- Models scored
- 1
- Terminal-Bench 4.0-AA
With Laude Institute
Artificial Analysis reported accuracy on terminal tasks with executable final-container-state verifiers.
- Leader
Gemini 4 Carbon65.2%
- Models scored
- 5