Social IQa
The first large-scale benchmark for commonsense reasoning about social situations. Contains 38,000 multiple choice questions probing emotional and social intelligence in everyday situations, testing commonsense understanding of social interactions and theory of mind reasoning about the implied emotions and behavior of others.
Progress Over Time
Interactive timeline showing model performance evolution on Social IQa
State-of-the-art frontier
Open
Proprietary
Social IQa Leaderboard
9 models • 0 verified
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
1 |  Microsoft | 60B | — | — | ||
2 | Microsoft | 4B | 128K | $0.10 $0.10 | ||
3 | Microsoft | 4B | — | — | ||
4 |  Google | 27B | — | — | ||
5 | Google | 9B | — | — | ||
6 | Google | 8B | — | — | ||
6 | 2B | — | — | |||
8 | 2B | — | — | |||
8 | Google | 8B | — | — |
Notice missing or incorrect data?
FAQ
Common questions about Social IQa
The first large-scale benchmark for commonsense reasoning about social situations. Contains 38,000 multiple choice questions probing emotional and social intelligence in everyday situations, testing commonsense understanding of social interactions and theory of mind reasoning about the implied emotions and behavior of others.
The Social IQa paper is available at https://arxiv.org/abs/1904.09728. This paper provides detailed information about the benchmark methodology, dataset creation, and evaluation criteria.
The Social IQa leaderboard ranks 9 AI models based on their performance on this benchmark. Currently, Phi-3.5-MoE-instruct by Microsoft leads with a score of 0.780. The average score across all models is 0.589.
The highest Social IQa score is 0.780, achieved by Phi-3.5-MoE-instruct from Microsoft.
9 models have been evaluated on the Social IQa benchmark, with 0 verified results and 9 self-reported results.
Social IQa is categorized under creativity, psychology, and reasoning. The benchmark evaluates text models.