titleMemT-1.0: Towards Memory-Native Evolution
authorsZhiyu Li, Zeyu Zhang, Linyun Liu, Yiwei Bu, Chenyang Xi, Xichong Zhang, Yi Yu, Yu Yu, Zhichen Liu, Yuxiao Yi, Chunyu Li, Haoran Liu, Zhenkai Ding, Le Zhou, Juncheng Zhang, Boyu Chen, Zehao Lin, Guorui Li, Renyu Fu, Ji Li, Youyang Xia, Jiajun Shen, Harvey Li, Jingsong Zhou, Helan Hu, Tong Shen, Tianxiao Xu, Chenye Han, Weijie Li, Zhongquan Zhou, Hongming Xu, Yuekeng Li, Erenkai Zhu, Peng Yu, Wenhao Li, Jingyi Kang, Xixuan Hao, Xuzhou Zhu, Can Chen, Jiawei Liu, Chen Jiang, Tao Jiang, Fangwei Xiang, Zepeng Yao, Kai Chen, Guang Yang, Mingfei Lu, Chunhui Li, Zhengxuan Jiang, Bo Tang, Shiyu Zhan, Yihang Sun, Jie Chu, Wenjun Xiong, Zhenxiang Hua, Chen Lin, Detong Su, Dongyu Deng, Qiuping Ding, Guofeng Ding, Si Chen, Guohua Wen, Xu Chen, Zhi-Qin John Xu, Ying-Cong Chen, Ningyu Zhang, Haofen Wang, Muning Wen, Haotian Chen, Wentao Zhang, Xuanhe Zhou, Rita Zhang, Junchi Yan, Feiyu Xiong
date2026-10-19
venuearXiv (preprint, not peer reviewed)
idarXiv:2610.26825
affiliationMemTensor
upvotes142 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractMemory-native evolution requires making memory an intrinsic capability of foundation models, enabling them to continuously accumulate and utilize information. To this end, we introduce MemT-1.0, a new architecture of memory foundation models that internalize memory as part of model computation. Building on Metis, MemT-1.0 employs Gated DeltaNet-2 with a stable token-level memory update mechanism within each turn. We further introduce Mixture-of-Storage to scale native memory capacity. Following a deliberate backbone preparation, we develop a two-stage training pipeline consisting of continued pre-training and post-training, supported by 34B tokens of training data and efficient training and inference infrastructure. We release a model family comprising MemT-1.0-Nano and MemT-1.0-Flash, and conduct a comprehensive evaluation of memory and general capabilities. The results show that they outperform the parametric memory baselines on average and remain competitive with retrieval-augmented methods, while largely preserving general capabilities. At a history length of 128K tokens, MemT-1.0-Nano achieves approximately 1.86times end-to-end inference speedup and a 66times reduction in persistent storage relative to full context KV cache. By internalizing retrieved procedural experience, MemT-1.0 improves task success on agentic tasks, providing a practical step toward memory-native evolution.

cite

@misc{li2026memt,
  title={MemT-1.0: Towards Memory-Native Evolution},
  author={Zhiyu Li and Zeyu Zhang and Linyun Liu and Yiwei Bu and Chenyang Xi and Xichong Zhang and Yi Yu and Yu Yu and Zhichen Liu and Yuxiao Yi and Chunyu Li and Haoran Liu and Zhenkai Ding and Le Zhou and Juncheng Zhang and Boyu Chen and Zehao Lin and Guorui Li and Renyu Fu and Ji Li and Youyang Xia and Jiajun Shen and Harvey Li and Jingsong Zhou and Helan Hu and Tong Shen and Tianxiao Xu and Chenye Han and Weijie Li and Zhongquan Zhou and Hongming Xu and Yuekeng Li and Erenkai Zhu and Peng Yu and Wenhao Li and Jingyi Kang and Xixuan Hao and Xuzhou Zhu and Can Chen and Jiawei Liu and Chen Jiang and Tao Jiang and Fangwei Xiang and Zepeng Yao and Kai Chen and Guang Yang and Mingfei Lu and Chunhui Li and Zhengxuan Jiang and Bo Tang and Shiyu Zhan and Yihang Sun and Jie Chu and Wenjun Xiong and Zhenxiang Hua and Chen Lin and Detong Su and Dongyu Deng and Qiuping Ding and Guofeng Ding and Si Chen and Guohua Wen and Xu Chen and Zhi-Qin John Xu and Ying-Cong Chen and Ningyu Zhang and Haofen Wang and Muning Wen and Haotian Chen and Wentao Zhang and Xuanhe Zhou and Rita Zhang and Junchi Yan and Feiyu Xiong},
  year={2026},
  eprint={2610.26825},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.26825}
}