| title | Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination |
|---|---|
| authors | Wei Feng, Meiling Chen, Boxiao Chen, Lu Liu, Bin Zi, Libin Ye, Xingyu Lu, Lei Xu, Jun Zhou |
| date | 2026-11-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.16661 |
| affiliation | inclusionAI |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-18) |
| links | abs pdf hf |
| cite | |
| related | Realtime-Venus: A full-duplex interaction system with asynchronous delegation RULER: Instance-aware Rubric Rewards for SVG Generation Ling and Ring 2.6 Technical Report: Efficient and Instant Agentic Intelligence at Trillion-Parameter Scale Open-AoE: An Open Egocentric Manipulation Dataset and Toolchain for Embodied Learning LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes |
| abstract | Efficient batch execution of agent tasks is critical to LLM training and development workflows, including benchmark evaluation, synthetic-data generation, and RL rollouts. Since these tasks alternate between LLM inference and environment interaction, we present coordinated optimizations to reduce batch makespan, the time required to complete all tasks in a batch. For hybrid LLMs, a workload-based resource model guides GPU-memory allocation between key-value (KV) cache and recurrent state and provides a reference sample concurrency to exploit parallelism while limiting recomputation. Trajectory-aware routing and scheduling promote cache reuse while balancing worker load. Under constrained environment capacity, we decouple sample admission from environment execution and introduce Chain-Advancing Action Scheduling (CAAS), which prioritizes long chains using observed progress to shorten the batch tail. Controlled replay combines actual inference with recorded environment responses and durations for ablations of individual optimizations. Live evaluations with Ling-3.0-Flash and Ling-3.1-Flash on the BFCL, SWE-bench Multilingual, WideSearch, and MCP-Atlas benchmarks demonstrate makespan reductions of 10.4% to 65.0%, while evaluation scores remain aligned. Furthermore, these methods have been successfully applied to training and evaluation workflows for the Ling-3.x model series, accelerating the model-development cycle. |
bibtex
@misc{feng2026reducing,
title={Reducing Batch Makespan for LLM Agent Workloads through Inference-Environment Coordination},
author={Wei Feng and Meiling Chen and Boxiao Chen and Lu Liu and Bin Zi and Libin Ye and Xingyu Lu and Lei Xu and Jun Zhou},
year={2026},
eprint={2611.16661},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.16661}
}