titleCoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models
authorsYingzhuo Deng, Chong Li, Bingxin Liu, Yi Shu, Tianyu Peng, Can Xie, Siyao Song, Shuo Ren, Jiajun Zhang
date2026-10-15
venuearXiv (preprint, not peer reviewed)
idarXiv:2610.21143
affiliationChinese Academic of Science Institute of Automation
upvotes12 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractSpoken Dialogue Models (SDMs) typically share a speech-token space for generation and synthesis, leading to long autoregressive sequences and costly inference. Lower-rate speech tokenizers shorten sequences but may introduce compression-quality trade-offs and costly replacement without breaking this coupling. To address this limitation, we propose CoSTA, a compatibility-preserving post-hoc framework that decouples generation and synthesis token spaces while retaining the original speech tokenizer and decoder. CoSTA extends the generation token space with compressed representations via mean-initialized vocabulary extension, pronunciation adaptation, and dialogue adaptation, followed by deterministic expansion back to the original speech-token space before synthesis. On bilingual speech-dialogue benchmarks, CoSTA achieves approximately 1.7times expanded speech-token generation speed, lower full-response latency, and higher throughput while preserving response quality. Analyses show more sustained interaction under limited budgets and a practical trade-off among compression, decoding efficiency, and quality. Further experiments demonstrate cross-architecture transfer and complementarity to low-rate tokenization.

cite

@misc{deng2026costa,
  title={CoSTA: Compressed Speech Token Adaptation for Efficient Spoken Dialogue Models},
  author={Yingzhuo Deng and Chong Li and Bingxin Liu and Yi Shu and Tianyu Peng and Can Xie and Siyao Song and Shuo Ren and Jiajun Zhang},
  year={2026},
  eprint={2610.21143},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.21143}
}