| title | Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior |
|---|---|
| authors | Renee Jia, Jinzhou Wu |
| date | 2026-11-03 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.04395 |
| affiliation | Cornell University |
| upvotes | 5 (Hugging Face Daily Papers, 2026-11-19) |
| links | abs pdf hf |
| cite | |
| related | Rethinking in Spikes: Mitigating Hallucinations in MDLMs with Step-Aware Decoding SceneAligner: 3D-Grounded Floorplan Localization in the Wild Inside the Trading Loop: Markets, Models, and Moves with LLM Agents Raster2Seq: Polygon Sequence Generation for Floorplan Reconstruction Quantum Boltzmann machine learning of ground-state energies |
| abstract | Fine-tuning data can include metadata that predicts the correct answer, such as a reward-model score or a quality label. We ask whether a model learns to rely on this metadata even when every training label is correct. We fine-tune 7-9B open-weight models from three families on multiple-choice questions in which (Reward: 5) always marks the correct option; when this annotation is moved to a wrong option at test time, models select that option on 97-100% of CommonsenseQA items. Removing it leaves most answers correct, so evaluation without it misses the dependence. Cue-following rises sharply with training-time reliability and develops even with neutral or nonsense annotation words, although test-time wording affects its strength. A Qwen3 experiment also finds cue dependence in option-letter scores after direct preference optimization. Audits can test for this dependence by moving or removing visible annotations while holding the task and correct answer fixed. |
bibtex
@misc{jia2026correct,
title={Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior},
author={Renee Jia and Jinzhou Wu},
year={2026},
eprint={2611.04395},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.04395}
}