| title | MuSViT: A Foundation Vision Model for Sheet Music Representation |
|---|---|
| authors | Carlos Penarrubia, Antonio Rios-Vila, Eliseo Fuentes-Martinez, Juan C. Martinez-Sevilla, Francisco J. Castellanos, María Alfaro-Contreras, Jorge Calvo-Zaragoza |
| date | 2026-06-30 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.31811 |
| affiliation | Pattern Recognition and Artificial Intelligence Group |
| upvotes | 9 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Foundation models have transformed vision and language processing by providing rich, reusable representations that transfer across diverse tasks. Sheet music, as a visual encoding of musical language, lacks such a strong domain-specific backbone. We introduce MuSViT (Music Score Vision Transformer): the first foundation vision model for sheet music representation -- a ViT encoder pre-trained via Masked Autoencoders on 9.7 million pages from the IMSLP. To handle the complexity of real-world scores, we adopt a two-stage curriculum: a synthetic warm-up on typeset scores followed by large-scale training on the full IMSLP corpus. We evaluate MuSViT on four downstream tasks -- full-page and staff-level music score recognition, music symbol detection, and score difficulty classification -- under two scenarios: linear probing (frozen encoder) and fine-tuning. Under linear probing, MuSViT consistently outperforms modern vision encoders, revealing that general-purpose representations, regardless of scale, fall systematically short on the structured symbolic properties of musical notation. Under fine-tuning, MuSViT generally improves upon task-specific state-of-the-art methods. An additional embedding-transcription consistency analysis reveals that MuSViT encodes symbolic musical structure directly in its representation space -- unlike other encoders, whose embeddings do not correlate with music notation content. These results establish MuSViT as a foundation backbone for sheet music understanding. |
cite
@misc{penarrubia2026musvit,
title={MuSViT: A Foundation Vision Model for Sheet Music Representation},
author={Carlos Penarrubia and Antonio Rios-Vila and Eliseo Fuentes-Martinez and Juan C. Martinez-Sevilla and Francisco J. Castellanos and María Alfaro-Contreras and Jorge Calvo-Zaragoza},
year={2026},
eprint={2606.31811},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.31811}
}