All benchmarks

AutomationBench-AA

Paper

Progress Over Time

Interactive timeline showing model performance evolution on AutomationBench-AA

State-of-the-art frontier
Open
Proprietary

AutomationBench-AA Leaderboard

1 models
ContextCostLicense
1
xAI
Grok 4.5New
xAI
—500K$2.00 / $6.00
Notice missing or incorrect data?
About this benchmark

What is AutomationBench-AA?

AutomationBench-AA is Artificial Analysis's independently run version of AutomationBench, covering 657 real-world SaaS workflow tasks across 40 simulated applications (e.g. Gmail, Slack, Salesforce, HubSpot). It scores the share of objectives an agent completes without violating business guardrails, using a private held-out task set.

AutomationBench-AA is a text benchmark evaluating models on reasoning, agents, and tool calling tasks. LLM Stats tracks 1 models on this benchmark, scored on a 0–1 scale. The current average is 0.5, with the leader at 0.5.

Compare leaders on the best AI for reasoning, best AI for agents and best AI for tool calling leaderboards.

Current leaders

Grok 4.5 from xAI currently leads the AutomationBench-AA leaderboard with a score of 0.514 across 1 evaluated AI models.

1Grok 4.5xAI51.4%

FAQ

Common questions about the AutomationBench-AA benchmark and leaderboard.

What is the AutomationBench-AA benchmark?

AutomationBench-AA is Artificial Analysis's independently run version of AutomationBench, covering 657 real-world SaaS workflow tasks across 40 simulated applications (e.g. Gmail, Slack, Salesforce, HubSpot). It scores the share of objectives an agent completes without violating business guardrails, using a private held-out task set.

What is the AutomationBench-AA leaderboard?

The AutomationBench-AA leaderboard ranks 1 AI models based on their performance on this benchmark. Currently, Grok 4.5 by xAI leads with a score of 0.514. The average score across all models is 0.514.

What is the highest AutomationBench-AA score?

The highest AutomationBench-AA score is 0.514, achieved by Grok 4.5 from xAI.

How many models are evaluated on AutomationBench-AA?

1 models have been evaluated on the AutomationBench-AA benchmark, with 0 verified results and 0 self-reported results.

Where can I find the AutomationBench-AA paper?

The AutomationBench-AA paper is available at https://artificialanalysis.ai. The paper details the methodology, dataset construction, and evaluation criteria.

What categories does AutomationBench-AA cover?

AutomationBench-AA is categorized under reasoning, agents, and tool calling. The benchmark evaluates text models.

What's the difference between AutomationBench-AA and AutomationBench?

AutomationBench-AA is a variant of AutomationBench. See the AutomationBench leaderboard for the broader benchmark and per-model comparison.

Which model offers the best value on AutomationBench-AA?

Among models scoring within 10% of the leader, Grok 4.5 from xAI is the cheapest, at $2.00 per million input tokens with a score of 0.514.

How recent are the AutomationBench-AA leaderboard results?

The AutomationBench-AA leaderboard was last updated in July 2026 and currently includes 1 evaluated models.