| title | Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion |
|---|---|
| authors | Kam Man Wu, Yu Zhang, Pengjun Fang, Mingfei Chen, Weijia Chen, Zeyue Tian, Qifeng Chen |
| date | 2026-10-25 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.34415 |
| affiliation | NoizAI |
| upvotes | 50 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Spatial audio authoring coordinates source decisions with soundfield synthesis. For editing, plans specify what to change and retain, while reference audio supplies further acoustic detail. We present AMBIT, a unified framework for native First-Order Ambisonics (FOA) generation and editing with three connected contributions. First, a shared Transfusion backbone couples autoregressive planning with rectified-flow rendering through executable ScenePlans that specify source content, activity, and 3D trajectories. For editing, the plan describes the full target scene, including retained sources; contrastive features ground the requested change, while full reference latents supply residual acoustic detail for preservation. Second, an FOA-aware continuous VAE combines shared-decoder omnidirectional recovery, asymmetric grouped KL, and spatial covariance matching to preserve content and inter-channel structure. Third, joint Transfusion on-policy self-distillation (OPSD) refines planning on student-generated prefixes and rendering at sampling states reached under student plans, with target-conditioned teachers supervising both modes of the shared student. Evaluations across speech, music, and sound show improved generation content and spatial quality, acoustic fidelity during instruction-guided editing, and unedited-source preservation over the compared baselines. Project page: https://wjm-george.github.io/AMBIT/ |
cite
@misc{wu2026spatial,
title={Spatial Audio Generation and Editing with Executable Plans and Self-Distilling Transfusion},
author={Kam Man Wu and Yu Zhang and Pengjun Fang and Mingfei Chen and Weijia Chen and Zeyue Tian and Qifeng Chen},
year={2026},
eprint={2610.34415},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.34415}
}