| title | Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling |
|---|---|
| authors | Panagiotis Theodoropoulos, Nan Jiang, Xintong Duan, Ali Hasan, Yuriy Nevmyvaka, Evangelos A. Theodorou, Wei Deng |
| date | 2026-09-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.38104 |
| affiliation | Morgan Stanley |
| upvotes | 7 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Power-sharpened sampling is an inference-time alternative to reinforcement-learning (RL) post-training for enhancing reasoning in large language models (LLMs). High-probability sequences are amplified under the base model without parameter updates or external rewards, avoiding the costly optimization and jagged generalization of RL. However, this approach faces a fundamental exploration--exploitation trade-off, as % strong sharpening restricts exploration, trapping samplers in plausible but incorrect reasoning trajectories, whereas weak sharpening leaves the answer distribution diffuse. To resolve this trade-off, we introduce Parallel Power Tempering (PPT), instantiating power-sharpened LLM sampling via parallel tempering. Running multiple interacting replicas in parallel at different sharpening levels allows lower-power replicas to explore diverse reasoning trajectories and higher-power chains to further exploit higher-likelihood responses favored by the sharpened target. Specifically, we tailor to inference-time sampling by mitigating a truncation bias, identified in prior power samplers, and investigate effective swap strategies under finite memory and compute budgets. Extensive experimentation shows that substantially improves single-chain power-sharpened sampling and outperforms RL-post-trained models, producing higher-quality reasoning traces and even achieving performance comparable to frontier models. |
cite
@misc{theodoropoulos2026explore,
title={Explore Broadly, Reason Sharply: Push Small Models toward the Frontier via Sampling},
author={Panagiotis Theodoropoulos and Nan Jiang and Xintong Duan and Ali Hasan and Yuriy Nevmyvaka and Evangelos A. Theodorou and Wei Deng},
year={2026},
eprint={2609.38104},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.38104}
}