| title | Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models |
|---|---|
| authors | Jean de Dieu Nyandwi, Leena Mathur, Yonatan Bisk, Robert Hawkins, Graham Neubig |
| date | 2026-08-13 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.13760 |
| affiliation | Carnegie Mellon University |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Which reasoning behaviors are associated with correct answers in reasoning models, and does reasoning-oriented training amplify those behaviors? This distinction is important because reasoning-oriented training can make traces look more deliberative without amplifying the behaviors most tied to model correctness. We quantify this mismatch with Behavioral Lift, a metric that measures how much correctness changes when a behavior is present versus absent in a model's reasoning trace. Across 15 models and 6 benchmarks spanning text-only and vision-language reasoning, we annotate 15,282 traces with a taxonomy whose core behaviors are defined for both LLM and VLM traces. We find evidence for an Amplification-Lift Gap, in which thinking models strongly amplify self-correction, hypothesis testing, and uncertainty acknowledgment, while the highest-lift behaviors are confidence calibration, knowledge alignment, and self-awareness. Confidence calibration is among the strongest positive signals of correctness in both modalities, yet is barely amplified; uncertainty acknowledgment is amplified by 3--7times, yet is weakly or negatively associated with correctness. We find that reasoning-oriented training does not preferentially amplify the highest-Lift behaviors, motivating process-level objectives that reward calibrated and grounded reasoning rather than surface form alone. |
cite
@misc{nyandwi2026amplified,
title={Amplified Does Not Mean Predictive: Reasoning Behaviors in Thinking Models},
author={Jean de Dieu Nyandwi and Leena Mathur and Yonatan Bisk and Robert Hawkins and Graham Neubig},
year={2026},
eprint={2608.13760},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.13760}
}