The AI arena is free today

Open Superagent
AI OrganizationMaintains 3 benchmarks

Aider: AI benchmarks & research

Open-source AI pair-programming tool. Maintains the Aider polyglot coding leaderboard.

Maintained by Aider

3 benchmarks
  • Aider

    Aider is a comprehensive code editing benchmark based on 133 practice exercises from Exercism's Python repository, designed to evaluate AI models' ability to translate natural language coding requests into executable code that passes unit tests. The benchmark measures end-to-end code editing capabilities, including GPT's ability to edit existing code and format code changes for automated saving to local files. The Aider Polyglot variant extends this evaluation across 225 challenging exercises spanning C++, Go, Java, JavaScript, Python, and Rust, making it a standard benchmark for assessing multilingual code editing performance in AI research.

    Leader
    DeepSeek-V2.572.2%
    Models scored
    4
  • Aider-Polyglot

    A coding benchmark that evaluates LLMs on 225 challenging Exercism programming exercises across C++, Go, Java, JavaScript, Python, and Rust. Models receive two attempts to solve each problem, with test error feedback provided after the first attempt if it fails. The benchmark measures both initial problem-solving ability and capacity to edit code based on error feedback, providing an end-to-end evaluation of code generation and editing capabilities across multiple programming languages.

    Leader
    GPT-588.0%
    Models scored
    22
  • Aider-Polyglot Edit

    A challenging multi-language coding benchmark that evaluates models' code editing abilities across C++, Go, Java, JavaScript, Python, and Rust. Contains 225 of Exercism's most difficult programming problems, selected as problems that were solved by 3 or fewer out of 7 top coding models. The benchmark focuses on code editing tasks and measures both correctness of solutions and proper edit format usage. Designed to re-calibrate evaluation scales so top models score between 5-50%.

    Leader
    DeepSeek-V379.7%
    Models scored
    10

In one line

Aider maintains Aider (counts toward the LLM Stats Score, 4 models scored), Aider-Polyglot (counts toward the LLM Stats Score, 22 models scored) and Aider-Polyglot Edit (counts toward the LLM Stats Score, 10 models scored). It ships no models of its own on LLM Stats.

FAQ

Common questions about Aider.

What is Aider?

Open-source AI pair-programming tool. Maintains the Aider polyglot coding leaderboard. On LLM Stats it maintains 3 benchmarks, led by Aider-Polyglot, Aider-Polyglot Edit and Aider. It ships no models of its own; each score on its benchmarks carries its own provenance label on the model page.

Which Aider benchmarks count toward the LLM Stats Score?

Aider-Polyglot, Aider-Polyglot Edit and Aider count toward the LLM Stats Score. Only benchmarks admitted to the core set move the main leaderboard; the rest are shown for transparency.

Which models lead Aider's benchmarks?

GPT-5 leads Aider-Polyglot at 88.0% (22 models scored), DeepSeek-V3 leads Aider-Polyglot Edit at 79.7% (10 models scored) and DeepSeek-V2.5 leads Aider at 72.2% (4 models scored). Each benchmark page lists the full ranking and how the results were produced.

Where are Aider's official benchmark results?

Aider's official site is https://aider.chat. Each benchmark page on LLM Stats links to that benchmark's own paper or leaderboard, and shows the scores next to every other benchmark so models can be compared on one page.