Center for AI Safety: AI benchmarks & research

Weapons of Mass Destruction (WMDP) is a multiple-choice benchmark on dual-use biology, chemistry, and cyber knowledge. It measures a model's capacity to enable malicious actors to design, synthesize, acquire, or use chemical, biological, radiological, or nuclear (CBRN) weapons.

Biology subset of WMDP (Weapons of Mass Destruction Proxy). Multiple-choice questions measuring dual-use biology knowledge relevant to enabling malicious actors to design, synthesize, acquire, or use biological weapons.

Chemistry subset of WMDP (Weapons of Mass Destruction Proxy). Multiple-choice questions measuring dual-use chemistry knowledge relevant to enabling malicious actors to design, synthesize, acquire, or use chemical weapons.

Cybersecurity subset of WMDP (Weapons of Mass Destruction Proxy). Multiple-choice questions measuring dual-use cyber knowledge relevant to enabling malicious actors to engage in offensive cyber operations.

Primary maintainer Scale AI

Humanity's Last Exam (HLE) is a multi-modal academic benchmark with 2,500 questions across mathematics, humanities, and natural sciences, designed to test LLM capabilities at the frontier of human knowledge with unambiguous, verifiable solutions

Primary maintainer Scale AI

Text-only Humanity's Last Exam variant evaluated without tool use.

Primary maintainer Scale AI

Full multimodal Humanity's Last Exam evaluated with tool use enabled, distinct from its text-only subset.

Primary maintainer Scale AI

Text-only Humanity's Last Exam variant evaluated with tool use enabled.

Primary maintainer Scale AI

MASK is a collection of 1000 questions measuring whether models faithfully report their beliefs when pressured to lie. It operationalizes deception as the rate at which the model lies, i.e., knowingly making false statements intended to be received as true. Lower dishonesty rates indicate better honesty.