PHYBench
What is PHYBench?
PHYBench is a benchmark of real-world physics problems spanning mechanics, electromagnetism, thermodynamics, optics, and modern physics, designed to evaluate physical perception and multi-step quantitative reasoning in large language models.
PHYBench is a text benchmark evaluating models on physics, reasoning, and science tasks. LLM Stats tracks 1 models on this benchmark, scored on a 0–1 scale. The current average is 0.8, with the leader at 0.8.
Compare leaders on the best AI for physics, best AI for reasoning and best AI for science leaderboards.
Current leaders
Hy3 from Tencent currently leads the PHYBench leaderboard with a score of 0.774 across 1 evaluated AI models.
Source paper
- Title
- PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Models
- Authors
- Shi Qiu, Shaoyang Guo, Zhuo-Yang Song, Yunbo Sun, and 50 others
- Published
- arXiv
- 2504.16074
Abstract
Current benchmarks for evaluating the reasoning capabilities of Large Language Models (LLMs) face significant limitations: task oversimplification, data contamination, and flawed evaluation items. These deficiencies necessitate more rigorous assessment methods. To address these limitations, we introduce PHYBench, a benchmark of 500 original physics problems ranging from high school to Physics Olympiad difficulty. PHYBench addresses data contamination through original content and employs a systematic curation pipeline to eliminate flawed items. Evaluations show that PHYBench activates more tokens and provides stronger differentiation between reasoning models compared to other baselines like AIME 2024, OlympiadBench and GPQA. Even the best-performing model, Gemini 2.5 Pro, achieves only 36.9% accuracy compared to human experts' 61.9%. To further enhance evaluation precision, we introduce the Expression Edit Distance (EED) Score for mathematical expression assessment, which improves sample efficiency by 204% over binary scoring. Moreover, PHYBench effectively elicits multi-step and multi-condition reasoning, providing a platform for examining models' reasoning robustness, preferences, and deficiencies. The benchmark results and dataset are publicly available at https://www.phybench.cn/.
FAQ
Common questions about the PHYBench benchmark and leaderboard.