The AI arena is free today

Open Superagent

Claude Sonnet 4 vs Mistral Small 3.1 24B Instruct

Claude Sonnet 4 leads the LLM Stats Score 24.2 to 4.2.

Anthropic · Mistral AI · Updated for 2026

Which is better?

Claude Sonnet 4 leads the overall LLM Stats Score 24.2 to 4.2, ranking #168 overall.

In the 2 individual benchmarks reported for both models, Claude Sonnet 4 wins 2; this is a narrower head-to-head signal than the composite indexes.

Based on current LLM Stats indexes, shared benchmarks, pricing, and model metadata for 2026.

Choose Claude Sonnet 4

  • overall performance matters — it scores 24.2 and ranks #168 on LLM Stats
  • your work emphasizes reasoning — it leads those capability indexes
  • you value its reported benchmark strengths — it wins 2 of 2 exact shared results
  • you want the most recent training data — it shipped May 2025

Choose Mistral Small 3.1 24B Instruct

  • you need open weights you can self-host or fine-tune

At a glance

The differences that matter most.

Claude Sonnet 4
Anthropic
Mistral Small 3.1 24B Instruct
Mistral AI
Core performance indexes
LLM Stats Score
24.2
#168
4.2
#305
Reasoning
24.3
#161
4.2
#299
Coding
16.6
#128
9.7
#173
Cost, coverage & limits
Benchmark wins
2 of 2
0 of 2
Input price
$3.00 / M
— / M
Output price
$15.00 / M
— / M
Context window
200,000
—

Capability indexes

Additional strengths measured across groups of related public benchmarks

1 shared
Index
Claude Sonnet 4
Mistral Small 3.1 24B Instruct
Math
16.5#204
7.9#261
Conservative TrueSkill rating · higher is betterHow scores work

Individual benchmarks

8 reported for Claude Sonnet 4 · 9 for Mistral Small 3.1 24B Instruct

2 shared

Claude Sonnet 4 outperforms in 2 benchmarks (GPQA, MMMU), while Mistral Small 3.1 24B Instruct is better at 0 benchmarks.

Claude Sonnet 4 significantly outperforms across most benchmarks.

Tue Sep 22 2026 • llm-stats.com

Human preference

Blind head-to-head votes and playground preference scores

Context Window

Maximum input and output token capacity

Only Claude Sonnet 4 specifies input context (200,000 tokens). Only Claude Sonnet 4 specifies output context (64,000 tokens).

Anthropic
Claude Sonnet 4
Input200,000 tokens
Output64,000 tokens
Mistral AI
Mistral Small 3.1 24B Instruct
Input- tokens
Output- tokens
Tue Sep 22 2026 • llm-stats.com

Input capabilities

Documented input modalities across available providers

Both Claude Sonnet 4 and Mistral Small 3.1 24B Instruct support multimodal inputs.

They are both capable of processing various types of data, offering versatility in application.

Claude Sonnet 4

Text
Images
Audio
Video

Mistral Small 3.1 24B Instruct

Text
Images
Audio
Video

License

Usage and distribution terms

Claude Sonnet 4 is licensed under a proprietary license, while Mistral Small 3.1 24B Instruct uses Apache 2.0.

License differences may affect how you can use these models in commercial or open-source projects.

Claude Sonnet 4

Proprietary

Closed source

Mistral Small 3.1 24B Instruct

Apache 2.0

Open weights

Release Timeline

When each model was launched

Claude Sonnet 4 was released on 2025-05-22, while Mistral Small 3.1 24B Instruct was released on 2025-03-17.

Claude Sonnet 4 is 2 months newer than Mistral Small 3.1 24B Instruct.

Claude Sonnet 4

May 22, 2025

1.3 years ago

2mo newer
Mistral Small 3.1 24B Instruct

Mar 17, 2025

1.5 years ago

Knowledge Cutoff

When training data ends

Neither model specifies a knowledge cutoff date.

Unable to compare the recency of their training data.

No cutoff dates available

Outputs Comparison

Notice missing or incorrect data?Start an Issue discussion→

Judge for yourself.

Run your own prompts against Claude Sonnet 4 and Mistral Small 3.1 24B Instruct side-by-side, then vote on the output you prefer.

Claude Sonnet 4
✓ Preferred
Mistral Small 3.1 24B Instruct
Open in Playground

FAQ

Common questions about Claude Sonnet 4 vs Mistral Small 3.1 24B Instruct.

Which is better, Claude Sonnet 4 or Mistral Small 3.1 24B Instruct?

Claude Sonnet 4 leads the LLM Stats Score 24.2 to 4.2. Claude Sonnet 4 is made by Anthropic and Mistral Small 3.1 24B Instruct is made by Mistral AI. The best choice depends on your use case — compare their capability indexes, individual benchmarks, pricing, and limits above.

How does Claude Sonnet 4 compare to Mistral Small 3.1 24B Instruct in benchmarks?

Claude Sonnet 4 scores MMMLU: 86.5%, TAU-bench Retail: 80.5%, GPQA: 75.4%, MMMU: 74.4%, SWE-Bench Verified: 72.7%. Mistral Small 3.1 24B Instruct scores HumanEval: 88.4%, MMLU: 80.6%, TriviaQA: 80.5%, MBPP: 74.7%, MATH: 69.3%.

What are the context window sizes for Claude Sonnet 4 and Mistral Small 3.1 24B Instruct?

Claude Sonnet 4 supports 200K tokens and Mistral Small 3.1 24B Instruct supports an unknown number of tokens. A larger context window lets you process longer documents, conversations, or codebases in a single request.

What are the main differences between Claude Sonnet 4 and Mistral Small 3.1 24B Instruct?

Key differences include LLM Stats Score (24.2 vs 4.2), licensing (Proprietary vs Apache 2.0). See the full comparison above for benchmark-by-benchmark results.

Who makes Claude Sonnet 4 and Mistral Small 3.1 24B Instruct?

Claude Sonnet 4 is developed by Anthropic and Mistral Small 3.1 24B Instruct is developed by Mistral AI.

Related comparisons