titleOn the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters
authorsMind Lab, Song Cao, Vic Cao, Kaijie Chen, Bunny Fan, Hera Feng, Huan Feng, Arthur Fu, Jun Gao, Hongquan Gu, Aaron Guan, Mutian Hong, Hailee Hou, Peixuan Hua, Charles Huang, Miles Jiang, Nora Jiang, Yuyi Jiang, Autumn Jin, Fancy Kong, Kyrie Lei, Alexy Li, Dawn Li, Ray Li, Theo Li, Wenhao Li, Jiayi Lin, Domini Liu, Heshan Liu, Kairus Liu, Logan Liu, Maeve Luo, Runism Lv, Pony Ma, Verity Niu, Anson Qiu, Vincent Wang, Maxwell Yao, Regis Ye, Wenlin Ye, Yanying Ye, Josh Ying, Danney Zeng, Salmon Zhan, Anya Zhang, Ruijia Zhang, Shiyang Zhang, Sueky Zhang, Ya Zhang, Wei Zhao, Ada Zhou, Sizer Zhou, Xinyue Zhu, Murphy Zhuang
date2026-06-01
venuearXiv (preprint, not peer reviewed)
idarXiv:2606.02437
affiliationMind Lab
upvotes103 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf
abstractParameter-efficient fine-tuning (PEFT) is usually treated as a cheaper alternative to full fine-tuning. We study a broader role: small trainable adapters as persistent local state on top of strong shared foundation models. In this framing, the base model provides shared competence while adapters carry instance-specific behavior such as preferences, skills, tool habits, and memory-like updates. We organize the problem around three scaling axes: Scale Up, where stronger shared priors make small local updates more useful; Scale Down, where we study how small adapters can be while remaining reliable; and Scale Out, where many persistent adapted instances coexist. MinT provides one infrastructure example for managing adapter identity, revision, provenance, evaluation, and serving residency. Together, the results suggest that PEFT can be a compact substrate for persistent personal models rather than only a budget substitute for full fine-tuning.

cite

@misc{lab2026scaling,
  title={On the Scaling of PEFT: Towards Million Personal Models of Trillion Parameters},
  author={Mind Lab and Song Cao and Vic Cao and Kaijie Chen and Bunny Fan and Hera Feng and Huan Feng and Arthur Fu and Jun Gao and Hongquan Gu and Aaron Guan and Mutian Hong and Hailee Hou and Peixuan Hua and Charles Huang and Miles Jiang and Nora Jiang and Yuyi Jiang and Autumn Jin and Fancy Kong and Kyrie Lei and Alexy Li and Dawn Li and Ray Li and Theo Li and Wenhao Li and Jiayi Lin and Domini Liu and Heshan Liu and Kairus Liu and Logan Liu and Maeve Luo and Runism Lv and Pony Ma and Verity Niu and Anson Qiu and Vincent Wang and Maxwell Yao and Regis Ye and Wenlin Ye and Yanying Ye and Josh Ying and Danney Zeng and Salmon Zhan and Anya Zhang and Ruijia Zhang and Shiyang Zhang and Sueky Zhang and Ya Zhang and Wei Zhao and Ada Zhou and Sizer Zhou and Xinyue Zhu and Murphy Zhuang},
  year={2026},
  eprint={2606.02437},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2606.02437}
}