titleCORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation
authorsTingyu Song, Mingxin Li, Yanzhao Zhang, Dingkun Long, Chu Liu, Pengjun Xie, Yilun Zhao, Shu Wu
date2026-09-03
venuearXiv (preprint, not peer reviewed)
idarXiv:2609.04083
affiliationAlibaba-NLP
upvotes13 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractMLLM-based embedding models remain limited in compositional retrieval, often failing to distinguish scenes containing the same concepts but different attribute-object bindings. Yet the same backbone can resolve such distinctions when used as a cross-attentive reranker, motivating us to distill its compositional judgments into the embedding model. We propose CORE, which synthesizes candidate lists spanning five compositional matching levels and introduces a Rank-KL objective that trains the embedding model to reproduce the reranker's fine-grained ranking. We further introduce a graded evaluation protocol and compare contrastive learning, pairwise CoSENT, and listwise Rank-KL under the same data and tuning budget. Our comparison shows that both CoSENT and Rank-KL use the multi-level supervision more effectively than contrastive learning, with Rank-KL achieving the strongest overall performance. Across three compositional reasoning benchmarks (COLA, SUGARCREPE++, NEGBENCH), CORE-RERANKER-8B achieves an 82.7% total average, outperforming Jina-Reranker by 10.7 points, while CORE-EMBED-8B achieves the best total average (0.666) among all evaluated embedding models. The improvements transfer to the MCMR benchmark without sacrificing retrieval performance on COCO and Flickr30K.

cite

@misc{song2026core,
  title={CORE: Improving Compositional Reasoning in MLLM Embedding via Reranker Distillation},
  author={Tingyu Song and Mingxin Li and Yanzhao Zhang and Dingkun Long and Chu Liu and Pengjun Xie and Yilun Zhao and Shu Wu},
  year={2026},
  eprint={2609.04083},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.04083}
}