The AI arena is free today

Open Superagent

B3 AI Security Benchmark

PaperImplementation

Progress Over Time

Interactive timeline showing model performance evolution on B3 AI Security Benchmark

State-of-the-art frontier
Open
Proprietary

B3 AI Security Benchmark Leaderboard

1 models
ContextCostLicense
1
Mistral AI
Mistral AI
1.1T1.0M$0.70 / $2.10
Notice missing or incorrect data?
About this benchmark

What is B3 AI Security Benchmark?

B3 AI Security Benchmark evaluates agent-backbone robustness against contextual prompt-injection attacks collected in Lakera's Agent Breaker Challenge.

B3 AI Security Benchmark is a text benchmark evaluating models on safety and agents tasks. LLM Stats tracks 1 models on this benchmark, scored on a 0–1 scale. The current average is 1.0, with the leader at 1.0.

Compare leaders on the best AI for safety and best AI for agents leaderboards.

Current leaders

Mistral Large 4 from Mistral AI currently leads the B3 AI Security Benchmark leaderboard with a score of 0.971 across 1 evaluated AI models.

1Mistral Large 4Mistral AI97.1%

Source paper

Title
Breaking Agent Backbones: Evaluating the Security of Backbone LLMs in AI Agents
Authors
Julia Bazinska, Max Mathys, Francesco Casucci, Mateo Rojas-Carulla, and 3 others
Published
Abstract

AI agents powered by large language models (LLMs) are being deployed at scale, yet we lack a systematic understanding of how the choice of backbone LLM affects agent security. The non-deterministic sequential nature of AI agents complicates security modeling, while the integration of traditional software with AI components entangles novel LLM vulnerabilities with conventional security risks. Existing frameworks only partially address these challenges as they either capture specific vulnerabilities only or require modeling of complete agents. To address these limitations, we introduce threat snapshots: a framework that isolates specific states in an agent's execution flow where LLM vulnerabilities manifest, enabling the systematic identification and categorization of security risks that propagate from the LLM to the agent level. We apply this framework to construct the $b^3$ benchmark, a security benchmark based on 194,331 unique crowdsourced adversarial attacks. We then evaluate 34 popular LLMs with it, revealing, among other insights, that enhanced reasoning capabilities improve security, while model size does not correlate with security. We release our benchmark, dataset, and evaluation code to facilitate widespread adoption by LLM providers and practitioners, offering guidance for agent developers and incentivizing model developers to prioritize backbone security improvements.

FAQ

Common questions about the B3 AI Security Benchmark benchmark and leaderboard.

What is the B3 AI Security Benchmark benchmark?

B3 AI Security Benchmark evaluates agent-backbone robustness against contextual prompt-injection attacks collected in Lakera's Agent Breaker Challenge.

What is the B3 AI Security Benchmark leaderboard?

The B3 AI Security Benchmark leaderboard ranks 1 AI models based on their performance on this benchmark. Currently, Mistral Large 4 by Mistral AI leads with a score of 0.971. The average score across all models is 0.971.

What is the highest B3 AI Security Benchmark score?

The highest B3 AI Security Benchmark score is 0.971, achieved by Mistral Large 4 from Mistral AI.

How many models are evaluated on B3 AI Security Benchmark?

1 models have been evaluated on the B3 AI Security Benchmark benchmark, with 0 verified results and 1 self-reported results.

Where can I find the B3 AI Security Benchmark paper?

The B3 AI Security Benchmark paper is available at https://arxiv.org/abs/2510.22620. The paper details the methodology, dataset construction, and evaluation criteria.

Where can I find the B3 AI Security Benchmark dataset?

The B3 AI Security Benchmark dataset is available on HuggingFace at https://huggingface.co/datasets/Lakera/b3-agent-security-benchmark-weak.

What categories does B3 AI Security Benchmark cover?

B3 AI Security Benchmark is categorized under safety and agents. The benchmark evaluates text models.

Which model offers the best value on B3 AI Security Benchmark?

Among models scoring within 10% of the leader, Mistral Large 4 from Mistral AI is the cheapest, at $0.70 per million input tokens with a score of 0.971.

How recent are the B3 AI Security Benchmark leaderboard results?

The B3 AI Security Benchmark leaderboard was last updated in November 2026 and currently includes 1 evaluated models.