| title | Liberating LLM Capabilities in Full-Duplex Speech Models |
|---|---|
| authors | Luoyuan Zhang, Bokai Xu, Junbo Cui, Weiyue Sun, Yingjing Xu, Hanyu Liu, Yuan Yao |
| date | 2026-05-04 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.07547 |
| upvotes | 8 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Speech-based large language models are typically constrained to spoken replies, which limits their user-facing outputs to what can be verbalized and suppresses text-native capabilities such as code generation, structured analysis, and multi-step reasoning in realtime interaction, for tasks that require persistent, structured, and inspectable intermediate outputs. Existing work improves spoken reasoning or full-duplex turn-taking, but still treats text as a hidden intermediate state or a subordinate modality rather than a first-class output channel. We propose Listen-Write-Speak (LWS), a text-first tri-channel paradigm in which a single autoregressive LLM continuously listens to user audio, writes visible free-form text as its primary output, and speaks a realtime oral response in parallel under a shared causal attention context. This behavior is implemented entirely through a Token Schema, requiring no architectural modifications, and learned via a two-stage data pipeline that synthesizes per-second cognitive annotations consistent with the revealed input timeline. Empirically, LWS demonstrates strong full-duplex interaction on Full-Duplex-Bench, reaches 4.72 on VoiceBench AlpacaEval, achieves 92.6% writing-speaking consistency, and consistently outperforms its internal ablations on URO-Bench. These results suggest that visible writing can serve as a first-class output channel for speech interaction without sacrificing realtime responsiveness. The code and dataset are available on the project page: https://royalzhang.com/project/lws-page/. |
cite
@misc{zhang2026liberating,
title={Liberating LLM Capabilities in Full-Duplex Speech Models},
author={Luoyuan Zhang and Bokai Xu and Junbo Cui and Weiyue Sun and Yingjing Xu and Hanyu Liu and Yuan Yao},
year={2026},
eprint={2606.07547},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.07547}
}