| title | Turning Drift into Constraint: Robust Reasoning Alignment in Non-Stationary Environments |
|---|---|
| authors | Xiaoyu Yang, En Yu, Wei Duan, Jie Lu |
| date | 2026-05-02 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2510.04142 |
| upvotes | 1 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | This paper identifies a critical yet underexplored challenge in reasoning alignment from multiple multi-modal large language models (MLLMs): In non-stationary environments, the diverse reasoning distributions of source models often evolve unpredictably, transmitting systematic biases and drift to the target model. To address this, we formulate multi-source reasoning alignment as a constraint satisfaction problem under concept drift theory. We propose Autonomous Preference Optimization (APO), a novel framework that treats inter-model divergences not as noise, but as dynamic negative constraints. APO operates via a two-stage protocol: first, supervised bootstrapping projects the target model into the capability union of source models; second, constraint-aware optimization synthesizes a consistent consensus manifold by explicitly suppressing drifting trajectories via a multi-negative Plackett-Luce objective. Extensive experiments on chest X-ray interpretation demonstrate that our 7B model achieves superior robustness, outperforming even proprietary source models in average accuracy. Furthermore, we release CXR-MAX, a large-scale benchmark comprising 170,982 reasoning trajectories from seven large-scale MLLMs to facilitate research on reasoning alignment under drift. Code and data are available at: https://github.com/XiaoyuYoung/APO. |
cite
@misc{yang2026turning,
title={Turning Drift into Constraint: Robust Reasoning Alignment in Non-Stationary Environments},
author={Xiaoyu Yang and En Yu and Wei Duan and Jie Lu},
year={2026},
eprint={2510.04142},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2510.04142}
}