| title | VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction |
|---|---|
| authors | Zhifei Xie, Jiaqi Lang, Ze An, Yifan Zhao, Dongchao Yang, Kai Li, Ziyang Ma, Mingbao Lin, Chunyan Miao, Shuicheng Yan |
| date | 2026-08-26 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.26005 |
| affiliation | Nanyang Technological University Singapore |
| upvotes | 133 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| primer | Memory for AI agents |
| abstract | Conversational systems, such as duplex speech language models (SLMs), still lack a streaming, accurate, and empathetic memory system as their soul. We introduce VoiceMem, a simple memory architecture with a parallel informational left brain, an emotional right brain, and streaming memory I/O mechanisms. We further build a complete pipeline for memory-aware SLM training, long-horizon evaluation, and decoupled deployment with interchangeable memory backends. Experiments and real-world deployment show three advantages: i) Accuracy: under top-5 retrieval, the left brain outperforms classical systems such as Mem0 at top-200 by nearly 30 points; ii) Emotional & Personal: the right brain, with short- and long-horizon affective attribution and dual-node persona modeling, achieves state-of-the-art performance across three persona benchmarks and improves the aggregate score by 4.29 points over the previous best system; and iii) Real-Time & Cheap: VoiceMem completes retrieval in 134 ms, well within standard VAD latency, adding no extra conversational delay while maintaining high accuracy and low cost. These results show that VoiceMem provides a practical memory foundation for real-time, personalized, and emotionally aware speech interaction. |
cite
@misc{xie2026voicemem,
title={VoiceMem: Streaming Dual-Brain Memory for Real-Time Interaction},
author={Zhifei Xie and Jiaqi Lang and Ze An and Yifan Zhao and Dongchao Yang and Kai Li and Ziyang Ma and Mingbao Lin and Chunyan Miao and Shuicheng Yan},
year={2026},
eprint={2608.26005},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.26005}
}