titleEmergent Biological Capabilities in a Foundation Model for Molecular Interactions
authorsAdi Mashiach
date2026-11-10
venuearXiv (preprint, not peer reviewed)
idarXiv:2611.14504
affiliationOutput Labs
upvotes11 (Hugging Face Daily Papers, 2026-11-18)
linksabs pdf hf
cite
abstractFoundation models trained on large unlabeled corpora develop emergent capabilities, tasks that go beyond the training objective. We extend this paradigm to multimodal biology with Bonbon, a foundation model for molecular interactions trained on approximately 3.2 trillion tokens of paired protein-ligand sequences. From a single representation space, Bonbon exhibits three zero-shot emergent capabilities at hierarchical levels of resolution: functional (mechanism of action), residue (binding site localization, including orthosteric/allosteric discrimination), and bond (covalent versus non-covalent engagement). All three capabilities generalize to out-of-distribution complexes. Bonbon was trained on protein-ligand sequences alone, without any task-specific supervision and without structural coordinates. We conducted controlled scaling experiments across six model scales from 42M to 1.6B parameters; every capability evaluated shows a sharp jump at the largest scale on continuous metrics, even as pretraining loss decreases smoothly. This demonstrates that loss-based scaling laws alone are insufficient to predict when zero-shot emergent capabilities become available. The capabilities reported here emerge after training at a 1:2,000 parameter-to-token ratio to accommodate the sparsity of paired biological interaction data. The same frozen representations achieve state-of-the-art binding and affinity prediction at five orders of magnitude greater speed than co-folding methods. In prospective wet-lab validation across nine disease targets spanning seven protein families, Bonbon achieves a 61.5% dose-response hit rate on de novo compounds, yielding structurally novel molecules not previously reported to inhibit these proteins, including for out-of-distribution targets.

bibtex

@misc{mashiach2026emergent,
  title={Emergent Biological Capabilities in a Foundation Model for Molecular Interactions},
  author={Adi Mashiach},
  year={2026},
  eprint={2611.14504},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2611.14504}
}