titleCorrect Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior
authorsRenee Jia, Jinzhou Wu
date2026-11-03
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.04395
affiliationCornell University
upvotes5 (Hugging Face Daily Papers, 2026-11-19)
linksabs pdf hf
cite
relatedRethinking in Spikes: Mitigating Hallucinations in MDLMs with Step-Aware Decoding
SceneAligner: 3D-Grounded Floorplan Localization in the Wild
Inside the Trading Loop: Markets, Models, and Moves with LLM Agents
Raster2Seq: Polygon Sequence Generation for Floorplan Reconstruction
Quantum Boltzmann machine learning of ground-state energies
abstractFine-tuning data can include metadata that predicts the correct answer, such as a reward-model score or a quality label. We ask whether a model learns to rely on this metadata even when every training label is correct. We fine-tune 7-9B open-weight models from three families on multiple-choice questions in which (Reward: 5) always marks the correct option; when this annotation is moved to a wrong option at test time, models select that option on 97-100% of CommonsenseQA items. Removing it leaves most answers correct, so evaluation without it misses the dependence. Cue-following rises sharply with training-time reliability and develops even with neutral or nonsense annotation words, although test-time wording affects its strength. A Qwen3 experiment also finds cue dependence in option-letter scores after direct preference optimization. Audits can test for this dependence by moving or removing visible annotations while holding the task and correct answer fixed.

bibtex

@misc{jia2026correct,
  title={Correct Labels Are Not Enough: Reliable Metadata Can Silently Control Model Behavior},
  author={Renee Jia and Jinzhou Wu},
  year={2026},
  eprint={2611.04395},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.04395}
}