| title | Debiased Model-based Representations for Sample-efficient Continuous Control |
|---|---|
| authors | Jiafei Lyu, Zichuan Lin, Scott Fujimoto, Kai Yang, Yangkun Chen, Saiyong Yang, Zongqing Lu, Deheng Ye |
| date | 2026-05-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.11711 |
| affiliation | Tencent Hunyuan |
| upvotes | 7 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Model-based representations recently stand out as a promising framework that embeds latent dynamics information into the representations for downstream off-policy actor-critic learning. It implicitly combines the advantages of both model-free and model-based approaches while avoiding the training costs associated with model-based methods. Nevertheless, existing model-based representation methods can fail to capture sufficient information about relevant variables and can overfit to early experiences in the replay buffer. These incur biases in representation and actor-critic learning, leading to inferior performance. To address this, we propose Debiased model-based Representations for Q-learning, tagged DR.Q algorithm. DR.Q explicitly maximizes the mutual information between the representations of the current state-action pair and the next state besides minimizing their deviations, and samples transitions with faded prioritized experience replay. We evaluate DR.Q on numerous continuous control benchmarks with a single set of hyperparameters, and the results demonstrate that DR.Q can match or surpass recent strong baselines, sometimes outperforming them by a large margin. Our code is available at https://github.com/dmksjfl/DR.Q. |
cite
@misc{lyu2026debiased,
title={Debiased Model-based Representations for Sample-efficient Continuous Control},
author={Jiafei Lyu and Zichuan Lin and Scott Fujimoto and Kai Yang and Yangkun Chen and Saiyong Yang and Zongqing Lu and Deheng Ye},
year={2026},
eprint={2605.11711},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.11711}
}