| title | JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts |
|---|---|
| authors | Zheyuan Zhang, Suyu Ye, Nakul Agarwal, Hossein Nourkhiz Mahjoub, Ehsan Moradi Pari, Daniel Khashabi, Tianmin Shu, Vaishnav Tadiparthi |
| date | 2026-09-30 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.00722 |
| affiliation | Johns Hopkins |
| upvotes | 3 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| primer | World models |
| abstract | World models enable agents to plan by predicting future states of the environment, but their predictions can become unreliable when test-time dynamics differ from those seen during training. We present JEPA-TTT, which adapts the latent dynamics predictor of a pretrained action-conditioned Joint-Embedding Predictive Architecture world model throughout test time. Self-supervised updates accumulate across episodes, while the visual encoder and reward head remain fixed, preserving the pretrained representation and task objective. Planning requires neither a goal image nor online environment reward. JEPA-TTT uses dense replay, which forms prediction windows at every temporal offset, retains them in a growing buffer, and samples minibatches from that buffer for predictor updates. Across eight dynamics shifts in four continuous-control environments, JEPA-TTT improves planning on every shift. After 500 test-time episodes, it reduces autoregressive latent prediction error by 83% on average and improves planning performance by 153% over the frozen JEPA world model. These results show that persistent self-supervised test-time training can adapt a pretrained latent world model under changed dynamics. |
cite
@misc{zhang2026jepa,
title={JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts},
author={Zheyuan Zhang and Suyu Ye and Nakul Agarwal and Hossein Nourkhiz Mahjoub and Ehsan Moradi Pari and Daniel Khashabi and Tianmin Shu and Vaishnav Tadiparthi},
year={2026},
eprint={2610.00722},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.00722}
}