| title | GigaAM Multilingual: Foundation Model for Underrepresented Languages |
|---|---|
| authors | Andrei Kuzmenko, Alexandr Maximenko, Aleksandr Kutsakov, Georgii Gospodinov, Dmitrii Bolotov, Oleg Kutuzov, Pavel Bogomolov, Fyodor Minkin |
| date | 2026-07-11 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2607.10371 |
| affiliation | ai-sage |
| upvotes | 35 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Despite recent scaling successes, multilingual ASR performance remains highly uneven, with long-tail languages suffering from severe data scarcity. This work addresses the challenge of building robust foundation models for underrepresented Central Asian languages (Kazakh, Kyrgyz, Uzbek). We present GigaAM Multilingual, a Conformer encoder pre-trained on 2M hours of audio using a HuBERT-style objective. Crucially, we introduce a cluster-level data balancing strategy during pre-training and a domain-aware sampling method during fine-tuning to mitigate head-language dominance. In controlled comparisons, our approach outperforms strong open pretrained encoders (Whisper Large v3, Omnilingual-1B) on target languages, achieving significant gains on spontaneous speech while maintaining efficiency. We release the foundation encoder and ASR model, offering a proven recipe for effective multilingual adaptation under realistic data imbalance. |
cite
@misc{kuzmenko2026gigaam,
title={GigaAM Multilingual: Foundation Model for Underrepresented Languages},
author={Andrei Kuzmenko and Alexandr Maximenko and Aleksandr Kutsakov and Georgii Gospodinov and Dmitrii Bolotov and Oleg Kutuzov and Pavel Bogomolov and Fyodor Minkin},
year={2026},
eprint={2607.10371},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2607.10371}
}