| title | Accelerating Mixture of Block Attention via Dynamic Routing, KV-Centric Execution and Runtime Pipelining |
|---|---|
| authors | Yingqi Shi, Junru Lu, Yun Liu, Di Yin, Xing Sun |
| date | 2026-11-19 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.26268 |
| affiliation | Tencent |
| upvotes | 12 (Hugging Face Daily Papers, 2026-11-21) |
| links | abs pdf hf code |
| cite | |
| related | EvolvingWorld: An Open-Schema Framework for Co-Evolving Role-Play Agents and World Model in Interactive Literary World A New Role for Relevance: Guiding Corpus Interaction in Agentic Search WeMM-Embedding: WeChat Multi-Modal Embedding Technical Report FlashMemory-DeepSeek-V4: Lightning Index Ultra-Long Context via Lookahead Sparse Attention VibeWorlding: Can Multimodal Agents Construct 3D Open Worlds End-to-End? |
| abstract | Mixture of Block Attention (MoBA) reduces long-context computation by selecting key-value (KV) blocks for each query. FlashMoBA implements MoBA efficiently but retains a fixed Top-k budget and query-centric schedule. Fixed Top-k applies one budget across different score distributions and couples routing to task granularity; query-range ownership fragments K/V reuse when several ranges select the same column. We present LightningMoBA, which co-designs selection and execution through dynamic routing, KV-centric ownership, and runtime pipelining. Dynamic routing thresholds MoBA's per-query block scores and directly emits variable-cardinality sparse metadata. KV-centric ownership groups selected rows by (query head, KV block) column, splits high-fanout columns into bounded tasks, and reuses each task's loaded KV tile. On Hopper, pipelining overlaps regular K/V transfers and irregular query gathers with Tensor Core computation. Together, these mechanisms decouple retention from GPU scheduling granularity. At matched Top-16 retention across five checkpoints, LightningMoBA accelerates end-to-end generation by 1.20 to 1.60x while raising mean RULER score from 74.42 to 75.28. |
bibtex
@misc{shi2026accelerating,
title={Accelerating Mixture of Block Attention via Dynamic Routing, KV-Centric Execution and Runtime Pipelining},
author={Yingqi Shi and Junru Lu and Yun Liu and Di Yin and Xing Sun},
year={2026},
eprint={2611.26268},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.26268}
}