titleMatAnyone-E: Sparse Memory Propagation for Efficient High-Resolution Video Matting
authorsShuyun Wang, Shangchen Zhou, Peiqing Yang, Yuekun Dai, Xin Yu, Chen Change Loy
date2026-10-26
venuearXiv (preprint, not peer reviewed)
idarXiv:2610.35724
affiliationMMLab@NTU
upvotes19 (Hugging Face Daily Papers, 2026-11-01)
linksabs pdf hf code
abstractHigh-resolution video matting requires preserving fine details, such as hair strands and soft transparency, while maintaining consistency across frames. Memory-based methods achieve strong matting quality by matching current-frame features with information stored from earlier frames, but this process becomes increasingly costly as resolution grows. We present MatAnyone-E, an efficient extension of MatAnyone 2 that focuses computation where it is most needed. Our approach first performs low-resolution propagation to identify uncertain regions that require closer inspection and the memory candidates most relevant to them. It then performs high-resolution matching only for these regions using the selected memory, while reusing low-resolution matching results elsewhere. This strategy reduces both the number of locations requiring detailed matching and the amount of memory searched, while preserving the information needed to produce a complete matte. We further reduce the cost of generating high-resolution mattes with a lightweight decoder designed around measured runtime bottlenecks and trained to reproduce the original decoder's representations. Experiments show that MatAnyone-E maintains matting quality comparable to MatAnyone 2 while achieving an estimated 7.4x inference speedup and reducing peak GPU memory usage to 7.6% of the usage of MatAnyone 2 at 4K. With compilation-enabled inference on a single NVIDIA H200 GPU, it reaches 73.07 FPS at 1080p and 28.62 FPS at 4K.

cite

@misc{wang2026matanyone,
  title={MatAnyone-E: Sparse Memory Propagation for Efficient High-Resolution Video Matting},
  author={Shuyun Wang and Shangchen Zhou and Peiqing Yang and Yuekun Dai and Xin Yu and Chen Change Loy},
  year={2026},
  eprint={2610.35724},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2610.35724}
}