| title | Language-Switching Triggers Take a Latent Detour Through Language Models |
|---|---|
| authors | Francis Kulumba, Wissam Antoun, Théo Lasnier, Benoît Sagot, Djamé Seddah |
| date | 2026-05-18 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.18646 |
| affiliation | ALMAnaCH (Inria) |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Backdoor attacks on language models pose a growing security concern, yet the internal mechanisms by which a trigger sequence hijacks model computations remain poorly understood. We identify a circuit underlying a language-switching backdoor in an 8B-parameter autoregressive language model, where a three-word Latin trigger (nine tokens) redirects English output to French. We decompose the circuit into three phases: (1) distributed attention heads at early layers compose the trigger tokens into the last sequence position; (2) the resulting signal propagates through mid-layers in a subspace orthogonal to the model's natural language-identity direction; (3) the MLP at the final layer converts this latent signal into French logits. The entire circuit flows through a serial bottleneck at a single position: corrupting that position at any layer entirely mitigate the trigger but also hinder the model's capabilities. The orthogonal latent encoding suggests that defenses that search for language-like signals in intermediate representations would miss this trigger entirely. |
cite
@misc{kulumba2026language,
title={Language-Switching Triggers Take a Latent Detour Through Language Models},
author={Francis Kulumba and Wissam Antoun and Théo Lasnier and Benoît Sagot and Djamé Seddah},
year={2026},
eprint={2605.18646},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.18646}
}