| title | Lance: Unified Multimodal Modeling by Multi-Task Synergy |
|---|---|
| authors | Fengyi Fu, Mengqi Huang, Shaojin Wu, Yunsheng Jiang, Yufei Huo, Hao Li, Yinghang Song, Fei Ding, Jianzhu Guo, Qian He, Zheren Fu, Zhendong Mao, Yongdong Zhang |
| date | 2026-05-18 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.18678 |
| affiliation | bytedance-research |
| upvotes | 79 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | We present Lance, a lightweight native unified model supporting multimodal understanding, generation, and editing for both images and videos. Rather than relying on model capacity scaling or text-image-dominant designs, Lance explores a practical paradigm for unified multimodal modeling via collaborative multi-task training. It is grounded in two core principles: unified context modeling and decoupled capability pathways. Specifically, Lance is trained from scratch and employs a dual-stream mixture-of-experts architecture on shared interleaved multimodal sequences, enabling joint context learning while decoupling the pathways for understanding and generation. We further introduce modality-aware rotary positional encoding to mitigate interference among heterogeneous visual tokens and boost cross-task alignment. During training, Lance adopts a staged multi-task training paradigm with capability-oriented objectives and adaptive data scheduling to strengthen both semantic comprehension and visual generation performance. Experimental results demonstrate that Lance substantially outperforms existing open-source unified models in image and video generation, while retaining strong multimodal understanding capabilities. The homepage is available at https://lance-project.github.io. |
cite
@misc{fu2026lance,
title={Lance: Unified Multimodal Modeling by Multi-Task Synergy},
author={Fengyi Fu and Mengqi Huang and Shaojin Wu and Yunsheng Jiang and Yufei Huo and Hao Li and Yinghang Song and Fei Ding and Jianzhu Guo and Qian He and Zheren Fu and Zhendong Mao and Yongdong Zhang},
year={2026},
eprint={2605.18678},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.18678}
}