| title | ProVe: Proposer-Verifier Decoding |
|---|---|
| authors | Ziqi Jin, Weiwen Xu, Jiaxi Li, Yujie Zhong, Qi She, Aixin Sun |
| date | 2026-10-20 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2610.27815 |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-04) |
| links | abs pdf hf |
| cite | |
| abstract | Speculative decoding accelerates autoregressive (AR) language models by letting a small model draft tokens serially and a large model verify them in parallel. We reverse these roles. Starting from a standard AR checkpoint, we fine-tune the large model into a proposer that predicts a block of future tokens in a single forward pass, then use one additional forward pass to verify the proposed block. This gives ProVe, Proposer-Verifier decoding. The verifier can be a smaller AR model or the proposer itself; the latter gives ProVe-lossless, which reproduces the proposer's greedy AR decoding. Block proposals already provide substantial room for acceleration: with a block size of 16, the proposed block has an average AR-matching prefix of 6.6 tokens on GSM8K. Across six benchmarks with a 32B proposer, ProVe-lossless reaches 2.5x average AR throughput without training a separate draft head. With a small 3B verifier, ProVe reaches 2.7x average throughput at 2.6 accuracy points below AR on the six-task average, providing a controllable tradeoff between speed and accuracy. |
bibtex
@misc{jin2026prove,
title={ProVe: Proposer-Verifier Decoding},
author={Ziqi Jin and Weiwen Xu and Jiaxi Li and Yujie Zhong and Qi She and Aixin Sun},
year={2026},
eprint={2610.27815},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2610.27815}
}