titleK2-Horizon: A Frontier Fleet of Open Source Language Models
authorsK2 Team, Zhengzhong Liu, Haonan Li, Xuezhe Ma, Alexander Moreno, Subham Sekhar Sahoo, Rupesh Kumar Srivastava, Mikhail Yurochkin, Hadi Abdine, Yash Akhauri, Adharsh Babu, David Bellamy, Jalaj Bhandari, Dani Bouch, Wail Bouhedja, Weili Cao, Ahmad Chamma, Po-Chun Chang, Junlin Chen, Lingjie Chen, Zhoujun Cheng, Kyle Chickering, Ryandito Diandaru, Evan Dufraisse, Chaitanya Dwivedi, Bilal El Bouardi, Desai Fan, Richard Fan, Chengqian Gao, Rifo Ahmad Genadi, Jonathan Geuter, Charles Guille-Escuret, Willis Guo, Dhawal Gupta, Seungwook Han, Tingxu Han, Shibo Hao, Besher Hassan, Guowei He, Ming Shan Hee, Joel Hestness, Yu Huang, Yuan Huang, Haolong Jia, Linghao Jin, Junu Kim, Lingjing Kong, Nurkhan Laiyk, Andrej Leban, Liqun Ma, John Maggs, Amr Mohamed, Mukhesh Narra, Omkar Pangarkar, Aldo Pareja, Khiem Pham, Varad Pimpalkhute, Nikhil Ranjan, Elizaveta Rebrova, Hector Ren, Shaurya Rohatgi, Guokan Shang, Jianshu She, Chufan Shi, Aaryamonvikram Singh, Daria Soboleva, Kartik Sreenivasan, Suqi Sun, Yuekai Sun, Bowen Tan, Liping Tang, Shengkun Tang, Muzi Tao, Mallikarjuna Tupakula, Micah Villmow, Hongyi Wang, Huijuan Wang, Renxi Wang, Yongxin Wang, Yuqi Wang, Zhenting Wang, Shicheng Wen, Yingquan Wu, Anze Xie, Andrew Xing, Matthew Yang, Shuanghai Yu, Kerem Zaman, Zenan Zhai, Hongli Zhan, Xuezhou Zhang, Anshun Asher Zheng, Zhe Zhou, Richard Zhuang, Eric P. Xing
date2026-11-16
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.21003
affiliationInstitute of Foundation Models
upvotes84 (Hugging Face Daily Papers, 2026-11-19)
linksabs pdf hf
cite
relatedUnlocking Lossless Speedups in LLMs via Discrete Diffusion
Towards Looped Models Done Right, Part II: Rethinking at Fixed Points
Towards Looped Models Done Right, Part I: Topology, Input Injection, Recurrent-State Design
LOCI: Spatial Linear Memory for Streaming World Models
Latent Feedback Is a Compute and Data Amplifier
abstractWe present K2-Horizon, a fleet of six fully open language models, from a 0.9B dense model to a 375B-A23B MoE, built for general-purpose reasoning and agentic work across math and science, software engineering, deep research, office automation, and beyond. The models lead or are competitive with strong open models of their size on agentic and coding benchmarks. On the modeling side, to strengthen reasoning, we introduce Mixture-of-Value Attention (MoVA), applying MoE to attention value projections, and softplus attention gating, a more flexible, theoretically motivated gate. Our 36B-A4B model uses both and nearly matches a 32B dense model trained on the same data, with one-seventh the active parameters. We also introduce Uno, lossless diffusion adapters that speed up decoding by up to 2.2x without a separate draft model. Our release follows the LLM360 definition of open source for transparency and reproducibility: the full development history, including intermediate checkpoints from every training stage, pre-training data, training logs, and all training, evaluation, and infrastructure code, notably our production-scale xLLM framework for dense and MoE training and the RL360 reinforcement-learning stack. Beyond the data itself, we release and document in detail every pipeline that built it, the part of model development that open releases most rarely disclose. These pipelines produced roughly 10T synthetic pre-training tokens, showing that synthetic data can be a major ingredient of pre-training at scale. Four of the models share one 22T-token pre-training corpus and recipe, revealing consistent scaling behavior and enabling controlled studies of scale and sparsity. All artifacts are available at https://ifm.ai/k2/

bibtex

@misc{team2026k,
  title={K2-Horizon: A Frontier Fleet of Open Source Language Models},
  author={K2 Team and Zhengzhong Liu and Haonan Li and Xuezhe Ma and Alexander Moreno and Subham Sekhar Sahoo and Rupesh Kumar Srivastava and Mikhail Yurochkin and Hadi Abdine and Yash Akhauri and Adharsh Babu and David Bellamy and Jalaj Bhandari and Dani Bouch and Wail Bouhedja and Weili Cao and Ahmad Chamma and Po-Chun Chang and Junlin Chen and Lingjie Chen and Zhoujun Cheng and Kyle Chickering and Ryandito Diandaru and Evan Dufraisse and Chaitanya Dwivedi and Bilal El Bouardi and Desai Fan and Richard Fan and Chengqian Gao and Rifo Ahmad Genadi and Jonathan Geuter and Charles Guille-Escuret and Willis Guo and Dhawal Gupta and Seungwook Han and Tingxu Han and Shibo Hao and Besher Hassan and Guowei He and Ming Shan Hee and Joel Hestness and Yu Huang and Yuan Huang and Haolong Jia and Linghao Jin and Junu Kim and Lingjing Kong and Nurkhan Laiyk and Andrej Leban and Liqun Ma and John Maggs and Amr Mohamed and Mukhesh Narra and Omkar Pangarkar and Aldo Pareja and Khiem Pham and Varad Pimpalkhute and Nikhil Ranjan and Elizaveta Rebrova and Hector Ren and Shaurya Rohatgi and Guokan Shang and Jianshu She and Chufan Shi and Aaryamonvikram Singh and Daria Soboleva and Kartik Sreenivasan and Suqi Sun and Yuekai Sun and Bowen Tan and Liping Tang and Shengkun Tang and Muzi Tao and Mallikarjuna Tupakula and Micah Villmow and Hongyi Wang and Huijuan Wang and Renxi Wang and Yongxin Wang and Yuqi Wang and Zhenting Wang and Shicheng Wen and Yingquan Wu and Anze Xie and Andrew Xing and Matthew Yang and Shuanghai Yu and Kerem Zaman and Zenan Zhai and Hongli Zhan and Xuezhou Zhang and Anshun Asher Zheng and Zhe Zhou and Richard Zhuang and Eric P. Xing},
  year={2026},
  eprint={2611.21003},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.21003}
}