| title | HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models |
|---|---|
| authors | Kyudan Jung, Hyunsin Park, Yoonhyung Lee, Jinhwan Park, Jinhyeok Yang, KiHyun Nam, Jaegul Choo, Jinkyu Lee |
| date | 2026-10-06 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.07727 |
| affiliation | Qualcomm AI Research |
| upvotes | 14 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | As human--AI interactions become more conversational, full-duplex speech language models capable of natural real-time dialogue are growing in importance. Beyond generating appropriate responses, these models must coordinate turn-taking, backchanneling, and floor management in real time. Reinforcement learning (RL) provides a way to refine these behaviors through direct feedback on interaction outcomes. However, existing RL methods either apply timing feedback to a token policy or optimize semantic content, leaving the joint improvement of timing and content unresolved. We introduce HiPLEX, an RL framework that factorizes a pretrained full-duplex text policy into a control policy that decides when to emit content and a conditional content policy that decides what to emit. The first factor selects among 'pad', 'epad', and 'con'. The second selects a token only when 'con' is chosen. This hierarchy describes conditional actions within each frame and uses the model's existing text head. We route timing advantages to the token-group factor through event-causal masks derived from generated speech episodes, and route an LLM-judge semantic advantage to the conditional content factor. Across three Moshi seeds on Full-Duplex-Bench v1, HiPLEX reduces takeover rates during natural user pauses and backchannel opportunities, and shortens post-interruption response latency relative to GRPO, while maintaining comparable judged interruption-response quality. On Moshi and PersonaPlex, HiPLEX better matches pooled human turn-timing and backchannel-rate marginals than GRPO. |
cite
@misc{jung2026hiplex,
title={HiPLEX: Hierarchical Policy Factorization for Full Duplex Speech Language Models},
author={Kyudan Jung and Hyunsin Park and Yoonhyung Lee and Jinhwan Park and Jinhyeok Yang and KiHyun Nam and Jaegul Choo and Jinkyu Lee},
year={2026},
eprint={2610.07727},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.07727}
}