titleMAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances
authorsMingyuan Lei, Yoonchang Sung, Tat-Jen Cham
date2026-10-08
venuearXiv (preprint, not peer reviewed)
idarXiv:2610.12416
affiliationNanyang Technological University Singapore
upvotes2 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractGenerating realistic human-object interactions (HOI) in complex 3D scenes requires two complementary capabilities: reasoning about interaction feasibility in the environment and synthesizing realistic human-object motion. However, supervision for these capabilities is rarely available jointly at scale. Human-scene datasets provide rich information about environment-aware motion, while human-object datasets capture detailed interaction dynamics, yet paired human-object-scene data remain scarce. We present MAMHOI, an affordance-mediated factorization for scene-aware human-object interaction generation. MAMHOI factorizes scene-aware HOI generation through an explicit motion-affordance interface between scene understanding and motion synthesis: a scene-conditioned model first predicts where and how an interaction can be feasibly executed, and an affordance-conditioned HOI model then generates the corresponding human-object motion. This factorization allows scene understanding and interaction dynamics to be learned from complementary sources of supervision without requiring paired human-object-scene data. Experiments in complex indoor environments show that MAMHOI reduces object--scene penetration while better preserving human--object interaction quality, yielding more realistic and physically feasible scene-aware interactions. Project page: https://leimingyuan.github.io/MAMHOI-project-page/

cite

@misc{lei2026mamhoi,
  title={MAMHOI: Factorizing Scene-Aware Human-Object Interaction through Affordances},
  author={Mingyuan Lei and Yoonchang Sung and Tat-Jen Cham},
  year={2026},
  eprint={2610.12416},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.12416}
}