| title | Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training |
|---|---|
| authors | Michal Chudoba, Sergey Alyaev, Petra Galuscakova, Tomasz Wiktorski |
| date | 2026-06-10 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.11854 |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | There are two main Parameter-Efficient Fine-Tuning (PEFT) techniques for Large Language Models (LLMs). While Low-Rank Adaptation (LoRA) introduces additional weights between the LLM layers, Soft Prompting introduces additional fine-tuning-specific raw tokens to an LLM input. However, both require modification to the computational graphs of precompiled, preoptimized LLMs. As a result, neither is fully supported in high-throughput engines like vLLM. We propose fine-tuning with ART (Art-based Reinforcement Training). The method injects information into a frozen Multimodal Large Language Model (MLLM) by optimizing only its raw visual input, thus enabling the soft-token approach on pre-compiled computational graphs. It relies on backpropagation of gradients back into a plain pixel array and thus supports any fine-tuning objective. Moreover, the optimized visual input can be stylized as task-relevant computational artworks. The approach's effectiveness is confirmed for different sizes of a popular open Qwen architecture and for several textual benchmarks. Specifically, ART reaches accuracy competitive with LoRA across mathematics and structured-tool-use benchmarks. |
cite
@misc{chudoba2026fine,
title={Fine-tuning Multi-modal LLMs with ART: Art-based Reinforcement Training},
author={Michal Chudoba and Sergey Alyaev and Petra Galuscakova and Tomasz Wiktorski},
year={2026},
eprint={2606.11854},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.11854}
}