| title | Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation |
|---|---|
| authors | Qian Yu, Zongyang Qiu, Juncheng Yan, Hongliang Li, Qi Qin, Keqi Wang, Siqi Luo, Junkui Ju, Yapeng Mi, Tiantian Lu, Tian Liu, Pengyu Xie, Chenhao Ji, Minghui Zhang, Yiang Zhao, Siyu Yan, Haiying Xu, Yanbo Wang, Jianghan Shen, Yuandong Pu, Jiayi Lei, Borui Zhang, Jinxin Shi, Bo Zhang, Yuhang Zang, Shenglong Ye, Lixin Gu, Hongjie Zhang, Linghan Liu, Xinyuan Chen, Yuewen Cao, Lihe Zhang, Xiaohong Liu, Junjun He, Qiangwei Liu, Qi Zhang, Yu Cheng, Tianfan Xue, Hui Xiong, Chang Xu, Yu Qiao, Yihao Liu, Bin Fu |
| date | 2026-10-14 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.20818 |
| affiliation | Shanghai AI Laboratory |
| upvotes | 50 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Unified multimodal generation covers discrete, continuous, and hybrid paradigms. Discrete approaches with single-codebook visual tokenizers align naturally with text modeling but have limited representational capacity. Continuous and hybrid approaches retain richer visual representations, but they model text and images in heterogeneous spaces with different training objectives, potentially introducing optimization conflicts between modalities with misaligned representation spaces. These choices expose a central question: how to efficiently model high-capacity visual representations alongside textual content in a shared optimization space within a single backbone. We present Intern Lumina U2, to our knowledge the first diffusion language model to represent multi-codebook visual tokens in a fully discrete space. Each visual position is represented by a group of codes from complementary codebooks, increasing its representational capacity while preserving compatibility with discrete text modeling. At the input, the embeddings are concatenated at each position; at the output, a lightweight multi-codebook autoregressive head predicts the codes within each position sequentially, while the diffusion backbone processes all positions in parallel. Built on the unified 4D representation of AToken, Intern Lumina U2 supports image, video, and 3D inputs within a single backbone and achieves leading performance among unified models in visual understanding, image generation, and image editing. We further build a full-stack training, inference, and evaluation system on both Ascend and NVIDIA platforms to verify the cross-platform effectiveness of our model. We also improve the inference efficiency through a stability-aware sampling strategy, which combines confidence-plateau-guided unmasking approach with drift-triggered cache refreshing mechanisms, accelerating image generation by 4.45x. |
cite
@misc{yu2026intern,
title={Intern Lumina U2: A Multi-Codebook Diffusion Large Language Model for Omni-Visual Understanding and Image Generation},
author={Qian Yu and Zongyang Qiu and Juncheng Yan and Hongliang Li and Qi Qin and Keqi Wang and Siqi Luo and Junkui Ju and Yapeng Mi and Tiantian Lu and Tian Liu and Pengyu Xie and Chenhao Ji and Minghui Zhang and Yiang Zhao and Siyu Yan and Haiying Xu and Yanbo Wang and Jianghan Shen and Yuandong Pu and Jiayi Lei and Borui Zhang and Jinxin Shi and Bo Zhang and Yuhang Zang and Shenglong Ye and Lixin Gu and Hongjie Zhang and Linghan Liu and Xinyuan Chen and Yuewen Cao and Lihe Zhang and Xiaohong Liu and Junjun He and Qiangwei Liu and Qi Zhang and Yu Cheng and Tianfan Xue and Hui Xiong and Chang Xu and Yu Qiao and Yihao Liu and Bin Fu},
year={2026},
eprint={2610.20818},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.20818}
}