| title | Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring |
|---|---|
| authors | Indraneil Paul, Glavaš Glavas, Iryna Gurevych |
| date | 2026-05-01 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.00754 |
| affiliation | Themis |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Reward models (RMs) have become an indispensable fixture of the language model (LM) post-training playbook, enabling policy alignment and test-time scaling. Research on the application of RMs in code generation, however, has been comparatively sparse, with existing work largely focusing on execution feedback. This choice constrains post-training to optimizing functional correctness over self-contained executable code. In this work, we examine the training and evaluation of multilingual, multi-criteria code RMs. To this end, we first compile Themis-CodeRewardBench, a benchmark to evaluate code RMs across five preference dimensions (i.e., criteria) and eight programming languages, on which we profile 50+ code, math, and general-purpose RMs. Observing the limited proficiency of current RMs beyond scoring for functional correctness, we develop Themis-CodePreference, the largest open-source collection of code preferences to date (more than 350k preference pairs), and use it to train Themis-RM, a suite of multilingual code reward models for flexible multi-criteria scoring, ranging in size from 600M to 32B parameters. Our experiments and ablations demonstrate positive scaling trends, strong cross-lingual transfer when training on diverse preferences, and the importance of multi-criteria training for reliable code reward modeling. |
cite
@misc{paul2026themis,
title={Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring},
author={Indraneil Paul and Glavaš Glavas and Iryna Gurevych},
year={2026},
eprint={2605.00754},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.00754}
}