titleReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation
authorsYiran Wang, Zeyu Zhang, Ling Shao, Hao Tang
date2026-09-08
venuearXiv (preprint, not peer reviewed)
idarXiv:2609.08365
affiliationPeking University
upvotes1 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractText-to-motion (T2M) generation maps natural language to human joint movements, aiding gaming, VR, and robotics. Retrieval-Augmented Text-to-Motion (RAG-T2M) improves generation on complex descriptions by conditioning on retrieved motion-text pairs. However, existing RAG-T2M models face two challenges: coarse-grained retrieval and fusion mechanisms overlook the hierarchical, spatial-temporal topology of human motion, and a representation gap exists because retrieved evidence resides in a semantic space separate from the generator's latents. To address the first, we present ReMoMask, a structure-aware RAG framework coupling Hierarchical Bidirectional Momentum (HBM) contrastive learning to align global and part-level features with text; Semantic Spatial-Temporal Attention (SSTA) for topology-aware fusion; and Topology Structured Masking (TSM) to force robust part-level grounding via adaptive masking. To address the second, we introduce ReMoMask-2, which rebuilds the retrieval database directly within the generator's pre-quantization latent space and aligns text queries via a distilled lightweight projector, allowing the generator to directly consume the retrieved motion's semantic content. Extensive experiments on HumanML3D, KIT-ML, and SnapMoGen demonstrate our retriever achieves state-of-the-art accuracy, while ReMoMask-2 attains the lowest FID on KIT-ML and SnapMoGen; notably, its single mask-transformer stage surpasses ReMoMask's full two-stage pipeline and delivers the fastest inference.

cite

@misc{wang2026remomask,
  title={ReMoMask-2: Latent Retrieval-Augmented Masked Motion Generation},
  author={Yiran Wang and Zeyu Zhang and Ling Shao and Hao Tang},
  year={2026},
  eprint={2609.08365},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2609.08365}
}