| title | Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation |
|---|---|
| authors | Mario Sanz-Guerrero, Minh Duc Bui, Manuel Mager, Katharina von der Wense |
| date | 2026-09-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.36931 |
| upvotes | 3 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Reproducibility is essential for scientific research, yet prior work shows that LLM outputs vary with hardware and batching. We identify an overlooked factor: the hidden injection of the current date into system prompts, which users cannot control and which changes every day. Across 9 recent LLMs and 6 datasets spanning multiple-choice QA (MCQA), math reasoning, code generation, and machine translation, performance varies solely with the current date, with deltas of up to 6% on MCQA, 14% on math reasoning, 7% on code generation, and 2.84 BLEU on machine translation. Model rankings also shift, affecting leaderboards. This date effect exceeds other sources of non-determinism, such as batch size and numerical precision. Standard prompting techniques -- chain-of-thought and few-shot prompting -- do not reduce the sensitivity; chain-of-thought even amplifies it. Our findings underscore the need for careful evaluation protocols to ensure reproducibility and fair comparisons in LLM research. |
cite
@misc{sanzguerrero2026dating,
title={Dating the Model: Hidden Dates in System Prompts Affect LLM Evaluation},
author={Mario Sanz-Guerrero and Minh Duc Bui and Manuel Mager and Katharina von der Wense},
year={2026},
eprint={2609.36931},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.36931}
}