| title | FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion |
|---|---|
| authors | Yu Lu, Junjie Yang, Piotr Koniusz, YuXin Song, Yi Yang |
| date | 2026-06-09 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.10671 |
| links | abs pdf hf |
| abstract | Autoregressive video generators synthesize long videos by generating successive temporal segments, but their historical KV cache grows with video length. Existing bounded-cache methods reduce this cost with local windows, sink tokens, or compressed memory states, yet they usually assign fixed roles to different parts of the history. We propose FadeMem, a distance-aware KV memory consolidation mechanism that organizes historical KV blocks into a temporal hierarchy under a fixed cache budget. This design is motivated by frequency-dependent temporal decay: fine details decorrelate quickly, while coarse scene structure and identity remain useful over longer horizons. During generation, new history is inserted as fine-grained entries, while older adjacent entries are progressively merged under a power-law temporal allocation schedule, yielding a dense-near, sparse-far memory within one cache. Without architectural changes, FadeMem preserves recent context for short-term dynamics and compact long-range anchors for identity and scene coherence. Experiments show improved subject consistency, background stability, and temporal coherence over existing bounded-cache strategies. |
cite
@misc{lu2026fademem,
title={FadeMem: Distance-Aware Memory Consolidation for Autoregressive Video Diffusion},
author={Yu Lu and Junjie Yang and Piotr Koniusz and YuXin Song and Yi Yang},
year={2026},
eprint={2606.10671},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.10671}
}