FLEURS
Progress Over Time
Interactive timeline showing model performance evolution on FLEURS
FLEURS Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 |  Alibaba Cloud / Qwen Team | 7B | — | — | ||
| 2 |  Google | — | — | — | ||
| 3 | Google | — | — | — | ||
| 4 | Google | 12B | — | — | ||
| 5 | Google | — | — | — | ||
| 6 | Google | 8B | — | — |
What is FLEURS?
Few-shot Learning Evaluation of Universal Representations of Speech - a parallel speech dataset in 102 languages built on FLoRes-101 with approximately 12 hours of speech supervision per language for tasks including ASR, speech language identification, translation and retrieval. Scores are shown as speech recognition accuracy (1 - word error rate), so higher is better.
FLEURS is a audio benchmark evaluating models on language and speech to text tasks. LLM Stats tracks 6 models on this benchmark, scored on a 0–1 scale. The current average is 0.9, with the leader at 1.0.
Compare leaders on the best AI for language and best AI for speech to text leaderboards.
Current leaders
Qwen2.5-Omni-7B from Alibaba Cloud / Qwen Team currently leads the FLEURS leaderboard with a score of 0.959 across 6 evaluated AI models.
Source paper
- Title
- FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech
- Authors
- Alexis Conneau, Min Ma, Simran Khanuja, Yu Zhang, and 5 others
- Published
- arXiv
- 2205.12446
Abstract
We introduce FLEURS, the Few-shot Learning Evaluation of Universal Representations of Speech benchmark. FLEURS is an n-way parallel speech dataset in 102 languages built on top of the machine translation FLoRes-101 benchmark, with approximately 12 hours of speech supervision per language. FLEURS can be used for a variety of speech tasks, including Automatic Speech Recognition (ASR), Speech Language Identification (Speech LangID), Translation and Retrieval. In this paper, we provide baselines for the tasks based on multilingual pre-trained models like mSLAM. The goal of FLEURS is to enable speech technology in more languages and catalyze research in low-resource speech understanding.
FAQ
Common questions about the FLEURS benchmark and leaderboard.