Open-rewrite
Progress Over Time
Interactive timeline showing model performance evolution on Open-rewrite
Open-rewrite Leaderboard
| Context | Cost | License | ||||
|---|---|---|---|---|---|---|
| 1 | 3B | — | — |
What is Open-rewrite?
OpenRewriteEval is a benchmark for evaluating open-ended rewriting of long-form texts, covering a wide variety of rewriting types expressed through natural language instructions including formality, expansion, conciseness, paraphrasing, and tone and style transfer.
Open-rewrite is a text benchmark evaluating models on language and writing tasks. LLM Stats tracks 1 models on this benchmark, scored on a 0–1 scale. The current average is 0.4, with the leader at 0.4.
Compare leaders on the best AI for language and best AI for writing leaderboards.
Current leaders
Llama 3.2 3B Instruct from Meta currently leads the Open-rewrite leaderboard with a score of 0.401 across 1 evaluated AI models.
Source paper
- Title
- RewriteLM: An Instruction-Tuned Large Language Model for Text Rewriting
- Authors
- Lei Shu, Liangchen Luo, Jayakumar Hoskere, Yun Zhu, and 4 others
- Published
- arXiv
- 2305.15685
Abstract
Large Language Models (LLMs) have demonstrated impressive capabilities in creative tasks such as storytelling and E-mail generation. However, as LLMs are primarily trained on final text results rather than intermediate revisions, it might be challenging for them to perform text rewriting tasks. Most studies in the rewriting tasks focus on a particular transformation type within the boundaries of single sentences. In this work, we develop new strategies for instruction tuning and reinforcement learning to better align LLMs for cross-sentence rewriting tasks using diverse wording and structures expressed through natural languages including 1) generating rewriting instruction data from Wiki edits and public corpus through instruction generation and chain-of-thought prompting; 2) collecting comparison data for reward model training through a new ranking function. To facilitate this research, we introduce OpenRewriteEval, a novel benchmark covers a wide variety of rewriting types expressed through natural language instructions. Our results show significant improvements over a variety of baselines. The public repository is available on GitHub under Google Research (https://github.com/google-research/google-research/tree/master/rewritelm).
FAQ
Common questions about the Open-rewrite benchmark and leaderboard.