| title | AVC-Reward: Event-Centric Audio-Video Consistency Evaluation |
|---|---|
| authors | Yuruo Wu, Kaihang Pan, Kai Shen, Jiacheng Li, Shuyuan Tu, Jisheng Dang, Li Tuoyu, Juncheng Li, Siliang Tang |
| date | 2026-10-15 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.22321 |
| affiliation | Zhejiang University |
| upvotes | 14 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | As joint audio-video generation models become more capable, they can generate audio-visual scenes exhibiting three forms of complexity: multi-event composition, multi-layer audio mixture, and asymmetric cross-modal relations. However, current evaluations largely reuse metrics from earlier simple and unidirectional tasks. They typically evaluate an audio-video clip as a whole and rely on simple predefined relations, making them unsuitable for these increasingly complex scenes. We propose an event-centric evaluation framework that disentangles audio and visual events at the event level, analyzes each event independently, cross-validates relations across modalities, and aggregates the judgments into a coarse-to-fine, three-level consistency metric. The metric assesses global narrative harmony, event-level temporal synchronization, and fine-grained attribute alignment, providing a general formulation for audio-video consistency evaluation across diverse generation settings and cross-modal relations. We instantiate the metric as AVC-Reward, a trainable audio-video reward model. To evaluate the agreement between automatic judgments and human perception across generation paradigms and relation complexities, we construct AVC-Benchmark, which is primarily built from diverse positive and negative samples generated by various types of audio-video generation models. AVC-Reward achieves state-of-the-art alignment with human judgment among automatic methods, provides interpretable rationales that localize failures, and improves the audio-video consistency of a joint generation model when used as an online reinforcement-learning signal. Project page: https://tsuki0512.github.io/AVC-Reward.github.io/ |
cite
@misc{wu2026avc,
title={AVC-Reward: Event-Centric Audio-Video Consistency Evaluation},
author={Yuruo Wu and Kaihang Pan and Kai Shen and Jiacheng Li and Shuyuan Tu and Jisheng Dang and Li Tuoyu and Juncheng Li and Siliang Tang},
year={2026},
eprint={2610.22321},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.22321}
}