- Organizations
- M-A-P
M-A-P: AI benchmarks & research
Multimodal Art Projection open research community. Maintains SuperGPQA and OmniBench.
Maintained by M-A-P
3 benchmarks- OmniBench
A novel multimodal benchmark designed to evaluate large language models' ability to recognize, interpret, and reason across visual, acoustic, and textual inputs simultaneously. Comprises 1,142 question-answer pairs covering 8 task categories from basic perception to complex inference, with a unique constraint that accurate responses require integrated understanding of all three modalities.
- Leader
Qwen2.5-Omni-7B56.1%- Models scored
- 1
- OmniBench Music
Music component of OmniBench, a comprehensive benchmark for evaluating omni-language models' ability to recognize, interpret, and reason across visual, acoustic, and textual inputs simultaneously. The music category includes various compositions and performances that require integrated understanding across text, image, and audio modalities.
- Leader
Qwen2.5-Omni-7B52.8%- Models scored
- 1
- SuperGPQA
SuperGPQA is a comprehensive benchmark that evaluates large language models across 285 graduate-level academic disciplines. The benchmark contains 25,957 questions covering 13 broad disciplinary areas including Engineering, Medicine, Science, and Law, with specialized fields in light industry, agriculture, and service-oriented domains. It employs a Human-LLM collaborative filtering mechanism with over 80 expert annotators to create challenging questions that assess graduate-level knowledge and reasoning capabilities.
- Leader
Qwen3.7 Max73.6%- Models scored
- 37