The AI arena is free today

Open Superagent

Gemini 1.5 Pro vs Mistral Small 3.1 24B Instruct

Gemini 1.5 Pro significantly outperforms across most benchmarks.

Google · Mistral AI · Updated for 2026

Which is better?

Gemini 1.5 Pro outperforms in 5 benchmarks (GPQA, MATH, MMLU, MMLU-Pro, MMMU), while Mistral Small 3.1 24B Instruct is better at 1 benchmark (HumanEval). Gemini 1.5 Pro significantly outperforms across most benchmarks.

Based on current benchmark, pricing, and model metadata for 2026.

Choose Gemini 1.5 Pro

  • you want the strongest raw capability — it leads on 5 of 6 shared benchmarks

Choose Mistral Small 3.1 24B Instruct

  • you want the most recent training data — it shipped Mar 2025
  • you need open weights you can self-host or fine-tune

At a glance

The differences that matter most.

Gemini 1.5 Pro
Google
Mistral Small 3.1 24B Instruct
Mistral AI
Benchmark wins
5 of 6
1 of 6
Input price
$2.50 / M
— / M
Output price
$10.00 / M
— / M
Context window
2,097,152
—
Released
May 2024
Mar 2025
License
Proprietary
Apache 2.0

Performance Benchmarks

Comparative analysis across standard metrics

6 benchmarks

Gemini 1.5 Pro outperforms in 5 benchmarks (GPQA, MATH, MMLU, MMLU-Pro, MMMU), while Mistral Small 3.1 24B Instruct is better at 1 benchmark (HumanEval).

Gemini 1.5 Pro significantly outperforms across most benchmarks.

Mon Aug 24 2026 • llm-stats.com

Arena Performance

Playground indexes and blind preference scores

Context Window

Maximum input and output token capacity

Only Gemini 1.5 Pro specifies input context (2,097,152 tokens). Only Gemini 1.5 Pro specifies output context (8,192 tokens).

Google
Gemini 1.5 Pro
Input2,097,152 tokens
Output8,192 tokens
Mistral AI
Mistral Small 3.1 24B Instruct
Input- tokens
Output- tokens
Mon Aug 24 2026 • llm-stats.com

Input Capabilities

Supported data types and modalities

Both Gemini 1.5 Pro and Mistral Small 3.1 24B Instruct support multimodal inputs.

They are both capable of processing various types of data, offering versatility in application.

Gemini 1.5 Pro

Text
Images
Audio
Video

Mistral Small 3.1 24B Instruct

Text
Images
Audio
Video

License

Usage and distribution terms

Gemini 1.5 Pro is licensed under a proprietary license, while Mistral Small 3.1 24B Instruct uses Apache 2.0.

License differences may affect how you can use these models in commercial or open-source projects.

Gemini 1.5 Pro

Proprietary

Closed source

Mistral Small 3.1 24B Instruct

Apache 2.0

Open weights

Release Timeline

When each model was launched

Gemini 1.5 Pro was released on 2024-05-01, while Mistral Small 3.1 24B Instruct was released on 2025-03-17.

Mistral Small 3.1 24B Instruct is 11 months newer than Gemini 1.5 Pro.

Gemini 1.5 Pro

May 1, 2024

2.3 years ago

Mistral Small 3.1 24B Instruct

Mar 17, 2025

1.4 years ago

10mo newer

Knowledge Cutoff

When training data ends

Gemini 1.5 Pro has a documented knowledge cutoff of 2023-11-01, while Mistral Small 3.1 24B Instruct's cutoff date is not specified.

We can confirm Gemini 1.5 Pro's training data extends to 2023-11-01, but cannot make a direct comparison without Mistral Small 3.1 24B Instruct's cutoff date.

Gemini 1.5 Pro

Nov 2023

Mistral Small 3.1 24B Instruct

—

Outputs Comparison

Notice missing or incorrect data?Start an Issue discussion→

Judge for yourself.

Run your own prompts against Gemini 1.5 Pro and Mistral Small 3.1 24B Instruct side-by-side, then vote on the output you prefer.

Gemini 1.5 Pro
✓ Preferred
Mistral Small 3.1 24B Instruct
Open in Playground

FAQ

Common questions about Gemini 1.5 Pro vs Mistral Small 3.1 24B Instruct.

Which is better, Gemini 1.5 Pro or Mistral Small 3.1 24B Instruct?

Gemini 1.5 Pro significantly outperforms across most benchmarks. Gemini 1.5 Pro is made by Google and Mistral Small 3.1 24B Instruct is made by Mistral AI. The best choice depends on your use case — compare their benchmark scores, pricing, and capabilities above.

How does Gemini 1.5 Pro compare to Mistral Small 3.1 24B Instruct in benchmarks?

Gemini 1.5 Pro scores XSTest: 98.8%, FLEURS: 93.3%, HellaSwag: 93.3%, GSM8k: 90.8%, BIG-Bench Hard: 89.2%. Mistral Small 3.1 24B Instruct scores HumanEval: 88.4%, MMLU: 80.6%, TriviaQA: 80.5%, MBPP: 74.7%, MATH: 69.3%.

What are the context window sizes for Gemini 1.5 Pro and Mistral Small 3.1 24B Instruct?

Gemini 1.5 Pro supports 2.1M tokens and Mistral Small 3.1 24B Instruct supports an unknown number of tokens. A larger context window lets you process longer documents, conversations, or codebases in a single request.

What are the main differences between Gemini 1.5 Pro and Mistral Small 3.1 24B Instruct?

Key differences include licensing (Proprietary vs Apache 2.0). See the full comparison above for benchmark-by-benchmark results.

Who makes Gemini 1.5 Pro and Mistral Small 3.1 24B Instruct?

Gemini 1.5 Pro is developed by Google and Mistral Small 3.1 24B Instruct is developed by Mistral AI.

Related comparisons