| title | Convergent Emergence of In-Context Learning Across Modalities |
|---|---|
| authors | Nathan Breslow, Seungwook Han, Daniel Hyunsoo Lee, Aayush Mishra, Anqi Liu, Daniel Khashabi |
| date | 2026-09-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.14011 |
| affiliation | Center for Language and Speech Processing @ JHU |
| upvotes | 3 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Few-shot in-context learning (ICL), the capacity of a model to infer abstract patterns from input-output examples provided in its prompt and apply them to new inputs, has been extensively studied in large language models trained for next-token prediction on human text. Recently, few-shot ICL has been demonstrated in autoregressive genomic models as well. This raises a question: does ICL emerge broadly across domains, and if so, what common structure is shared? To address both, we develop a controlled cross-modality framework that instantiates the same task suite in a variety of modalities to test what we call the Convergent Emergence Hypothesis: the idea that few-shot ICL, when it emerges, shares a common cross-modality difficulty profile - i.e., tasks that benefit from ICL in one modality tend to benefit in others. We show that paired-mapping ICL emerges across six modalities (language, genome, integer sequences, time series, images, and proteins), surpasses controlled baselines, and has correlated per-task effects across five of them. Together, these results provide support for the Convergent Emergence Hypothesis in some modalities, but not all. |
cite
@misc{breslow2026convergent,
title={Convergent Emergence of In-Context Learning Across Modalities},
author={Nathan Breslow and Seungwook Han and Daniel Hyunsoo Lee and Aayush Mishra and Anqi Liu and Daniel Khashabi},
year={2026},
eprint={2609.14011},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.14011}
}