Model Comparison

Gemma 2 27B vs Claude 3 Haiku

Claude 3 Haiku has a slight edge in benchmark performance.

Want to compare interactively?Try the playground

Performance Benchmarks

Comparative analysis across standard metrics

6 benchmarks

Gemma 2 27B outperforms in 2 benchmarks (HellaSwag, MATH), while Claude 3 Haiku is better at 3 benchmarks (ARC-C, GSM8k, HumanEval).

Claude 3 Haiku has a slight edge in benchmark performance.

Tue Jun 02 2026 • llm-stats.com

Arena Performance

Human preference votes

Context Window

Maximum input and output token capacity

Only Claude 3 Haiku specifies input context (200,000 tokens). Only Claude 3 Haiku specifies output context (200,000 tokens).

Google
Gemma 2 27B
Input- tokens
Output- tokens
Anthropic
Claude 3 Haiku
Input200,000 tokens
Output200,000 tokens
Tue Jun 02 2026 • llm-stats.com

Input Capabilities

Supported data types and modalities

Claude 3 Haiku supports multimodal inputs, whereas Gemma 2 27B does not.

Claude 3 Haiku can handle both text and other forms of data like images, making it suitable for multimodal applications.

Gemma 2 27B

Text
Images
Audio
Video

Claude 3 Haiku

Text
Images
Audio
Video

License

Usage and distribution terms

Gemma 2 27B is licensed under Gemma, while Claude 3 Haiku uses a proprietary license.

License differences may affect how you can use these models in commercial or open-source projects.

Gemma 2 27B

Gemma

Open weights

Claude 3 Haiku

Proprietary

Closed source

Release Timeline

When each model was launched

Gemma 2 27B was released on 2024-06-27, while Claude 3 Haiku was released on 2024-03-13.

Gemma 2 27B is 4 months newer than Claude 3 Haiku.

Gemma 2 27B

Jun 27, 2024

1.9 years ago

3mo newer
Claude 3 Haiku

Mar 13, 2024

2.2 years ago

Knowledge Cutoff

When training data ends

Neither model specifies a knowledge cutoff date.

Unable to compare the recency of their training data.

No cutoff dates available

Outputs Comparison

Notice missing or incorrect data?Start an Issue discussion→

Key Takeaways

Google
Has open weights
Higher HellaSwag score (86.4% vs 85.9%)
Higher MATH score (42.3% vs 38.9%)
Anthropic
Larger context window (200,000 tokens)
Supports multimodal inputs
Higher ARC-C score (89.2% vs 71.4%)
Higher GSM8k score (88.9% vs 74.0%)
Higher HumanEval score (75.9% vs 51.8%)

Detailed Comparison

AI Model Comparison Table
Feature
Google
Gemma 2 27B
Anthropic
Claude 3 Haiku

FAQ

Common questions about Gemma 2 27B vs Claude 3 Haiku.

Which is better, Gemma 2 27B or Claude 3 Haiku?

Claude 3 Haiku has a slight edge in benchmark performance. Gemma 2 27B is made by Google and Claude 3 Haiku is made by Anthropic. The best choice depends on your use case — compare their benchmark scores, pricing, and capabilities above.

How does Gemma 2 27B compare to Claude 3 Haiku in benchmarks?

Gemma 2 27B scores ARC-E: 88.6%, HellaSwag: 86.4%, BoolQ: 84.8%, TriviaQA: 83.7%, Winogrande: 83.7%. Claude 3 Haiku scores ARC-C: 89.2%, GSM8k: 88.9%, HellaSwag: 85.9%, DROP: 78.4%, HumanEval: 75.9%.

What are the context window sizes for Gemma 2 27B and Claude 3 Haiku?

Gemma 2 27B supports an unknown number of tokens and Claude 3 Haiku supports 200K tokens. A larger context window lets you process longer documents, conversations, or codebases in a single request.

What are the main differences between Gemma 2 27B and Claude 3 Haiku?

Key differences include multimodal support (no vs yes), licensing (Gemma vs Proprietary). See the full comparison above for benchmark-by-benchmark results.

Who makes Gemma 2 27B and Claude 3 Haiku?

Gemma 2 27B is developed by Google and Claude 3 Haiku is developed by Anthropic.