- Organizations
- Together AI
Together AI: AI benchmarks & research
AI cloud company. Maintains DeepSWE with Agentica.
Maintained by Together AI
3 benchmarks- DeepSWE
DeepSWE is a software engineering agent benchmark evaluated with the mini-swe-agent harness, where each task is solved in an isolated container with no internet access. It measures an agent's ability to autonomously resolve real-world coding issues end to end.
- Leader
Kimi K3.1 Pro75.5%
- Models scored
- 14
- DeepSWE 1.0
DeepSWE 1.0 pass@1 benchmark as reported by Artificial Analysis using provider-specific harness runs.
- Leader
Grok 4.562.0%
- Models scored
- 2
- DeepSWE 1.1
DeepSWE 1.1 evaluates software engineering agents using the mini-swe-agent harness.
- Leader
Muse Wonder Full79.0%
- Models scored
- 58