| title | A Language-Informed Loss for Visual Generation |
|---|---|
| authors | Suchir Agarwal, Keshigeyan Chandrasegaran, Ziyu Chen, Kyle Sargent, Juan Carlos Niebles, Li Fei-Fei, Jiajun Wu |
| date | 2026-10-16 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.23653 |
| affiliation | Stanford University |
| upvotes | 3 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | End-to-end visual generation with pixel-space loss has recently achieved competitive performance without the need for learned image tokenizers. In this work, we study whether language-informed supervision can provide a richer training signal than pixel loss for visual generation. We introduce LILIE (pronounced "lily"), a Language-Informed and Lossless Image Encoder. LILIE is an invertible image transform trained such that a vision-language model can recover captions from noised features. The encoder uses a normalizing flow transformer trained jointly with a vision-language model via next token prediction. We then use the encoder to extract a simple language-informed loss for text-to-image generation. We train JiT models with LILIE loss, reducing gFD by 14 points (66.84 to 53.01) against a pixel-only baseline on the GPIC dataset. Our code, encoders, and generative models are available at https://suchiragarwal.com/lilie. |
cite
@misc{agarwal2026language,
title={A Language-Informed Loss for Visual Generation},
author={Suchir Agarwal and Keshigeyan Chandrasegaran and Ziyu Chen and Kyle Sargent and Juan Carlos Niebles and Li Fei-Fei and Jiajun Wu},
year={2026},
eprint={2610.23653},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.23653}
}