| title | LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation |
|---|---|
| authors | Rongxiang Zhang, Songhua Liu |
| date | 2026-07-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.00079 |
| affiliation | Shanghai Jiao Tong University SAI |
| upvotes | 15 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Long-form and real-time talking-head generation remains challenging due to a latency-quality trade-off: inefficient multi-step diffusion prohibits streaming generation, whereas real-time autoregressive approaches suffer from error accumulation and identity drift. To address this drawback, we propose LeapTalk, a novel framework that achieves stable and real-time talking-head generation with a single forward step, scaling to arbitrarily long videos. At the heart of our approach lies a single-step bridge distillation scheme. On the one hand, departing from the conventional noise-to-data paradigm, we introduce a data-to-data transport formulation based on a Brownian bridge. Anchored by a persistent reference, this strategy effectively mitigates identity drift and enhances long-term temporal stability. On the other hand, to enable smooth knowledge transfer from a pre-trained diffusion teacher to the student bridge model, we explore a heterogeneous distillation framework with an SNR-aligned time transformation Φ(τ), which bridges the functional discrepancy between the two models. Moreover, we propose an audio-driven classifier-free guidance mechanism to maintain fine-grained lip synchronization under extreme step reduction. Extensive experiments demonstrate that our method achieves high-fidelity and temporally consistent video generation with only 1 step at up to 200 FPS, significantly outperforming existing approaches in both efficiency and stability. Project Page: https://zhangrongxiang.github.io/leaptalk-page/ |
cite
@misc{zhang2026leaptalk,
title={LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation},
author={Rongxiang Zhang and Songhua Liu},
year={2026},
eprint={2608.00079},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.00079}
}