titleSubliminal Learning in Vision-Language Models
authorsArman Hatami, Romina Aalishah, Ilya E. Monosov
date2026-11-04
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.08538
affiliationJohns Hopkins University
upvotes5 (Hugging Face Daily Papers, 2026-11-14)
linksabs pdf hf
cite
relatedContinual Learning Mechanisms Compose for Long-Horizon Memorization
PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perception
EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents
ThoughtTrace: Understanding User Thoughts in Real-World LLM Interactions
Steered LLM Activations are Non-Surjective
abstractSubliminal learning is a phenomenon in large language models (LLMs) where a teacher passes a hidden preference to a student through training data unrelated to that preference. We study whether subliminal learning also occurs in vision-language models (VLMs). We finetune a teacher VLM to prefer a specific animal or facial attribute and use a control teacher from the same base model. Both teachers generate numbers for an unrelated visual task involving geometric shapes. Two students initialized from the same base VLM are finetuned on the same images, using numbers generated by either the biased or control teacher. Across VLMs ranging from 2B to 7B parameters, we find that both animal and facial preferences can transfer through this unrelated task. Students trained on biased-teacher outputs select the target more often than control students, with a difference of up to sim10 percentage points in the animal preference task. Transfer can become stronger with more training data and varies with the target preference, model, and how the numbers are generated. Beyond preferences, we show that the biased student's explanations use more of the same words and phrases as the biased teacher's explanations than those of the control student, even when all three models select the same image. These results matter for training VLMs and show that training on unrelated data may change both a VLM's image preferences and how it explains its choices, and that transfer can become stronger with more training samples.

bibtex

@misc{hatami2026subliminal,
  title={Subliminal Learning in Vision-Language Models},
  author={Arman Hatami and Romina Aalishah and Ilya E. Monosov},
  year={2026},
  eprint={2611.08538},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.08538}
}