| title | TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation |
|---|---|
| authors | Haoran Wang, Chaofan Ma, Ran Yi, Lizhuang Ma |
| date | 2026-08-17 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.16765 |
| affiliation | Shanghai Jiao Tong University |
| upvotes | 13 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Despite recent advances in unified multimodal models for multi-reference image generation, existing benchmarks remain organized around predefined task types (e.g., "subject composition"), which are ill-suited to this combinatorial setting and lead to fragmented coverage, uncontrolled complexity, and little diagnostic value. Recognizing that diverse multi-reference tasks share a common set of atomic operations, we adopt a capability-oriented perspective and formalize four operators: Anchor (f), Disentangle (g), Apply (oplus), and Compose (C). Any multi-reference prompt can then be represented as a compositional formula over these operators, whose structural complexity is quantified by the number of operator slots. Building on this formulation, we construct TRACE-Bench, comprising approximately 1,600 evaluation cases across slot counts 1--8, built from 631 formula templates and around 4,000 reference images spanning diverse artistic styles and real-world subjects. The formula structure directly drives an operator-aligned evaluation protocol for per-capability scoring and a diagnostic tree analysis for recursive failure localization. Evaluating 9 leading models reveals insights invisible to holistic scoring: the primary bottleneck lies in disentanglement (g) and attribute binding (oplus) rather than scene-level composition (C), with even the best model scoring only 0.74 on attribute fidelity. Project page: https://amuseum-whr.github.io/TraceBench |
cite
@misc{wang2026trace,
title={TRACE-Bench: Decomposing and Diagnosing Multi-Reference Image Generation},
author={Haoran Wang and Chaofan Ma and Ran Yi and Lizhuang Ma},
year={2026},
eprint={2608.16765},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.16765}
}