| title | Loop the Loopies! |
|---|---|
| authors | Zitian Gao, Yilong Chen, Yihao Xiao, Xinyu Yang, Ran Tao, Joey Zhou, Bryan Dai |
| date | 2026-07-17 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2607.16051 |
| affiliation | IQuest |
| upvotes | 51 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | We present Loopie, the most powerful looped Transformer to date. The Loopie series consists of two Mixture-of-Experts (MoE) models: a 20B-parameter model with 2B active parameters and a 6Bparameter model with 0.6B active parameters. Looped Transformers have long faced a challenge: given an N-fold increase in pre-training compute, increasing the parameter count by a factor of N usually outperforms looping a model N times. Loopie addresses this challenge. Extensive ablation studies, including comparisons with a vanilla 30B-A3B model, show that Loopie substantially outperforms vanilla Transformer baselines trained with the same compute budget. Our novel post-training pipeline equips Loopie with strong reasoning abilities. At the 2025 IMO and IPhO, Loopie achieves gold-medal performance without tools. |
cite
@misc{gao2026loop,
title={Loop the Loopies!},
author={Zitian Gao and Yilong Chen and Yihao Xiao and Xinyu Yang and Ran Tao and Joey Zhou and Bryan Dai},
year={2026},
eprint={2607.16051},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2607.16051}
}