| title | AVA-Encoder: Towards Agent-Native Video Representation Learning |
|---|---|
| authors | Chuyue Li, Jinpeng Yu, Haozhe Wang, Tian Xueyun, Zhijing Zhang, Bingnan Li, Shuqi Gu, Kan Ren, Jiaming Liu, Ruihua Hua |
| date | 2026-08-12 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2608.12313 |
| affiliation | Qwen Business Unit |
| upvotes | 30 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Creative agents still lack an effective way to learn from high-quality human films, limiting their ability to produce cinematic-grade videos. A key challenge is the absence of a structured video representation that is both faithful to film content and directly usable for agentic reasoning and manipulation. To address the challenge, we propose the Agentic Video Auto-Encoder (AVA-Encoder), a framework for learning agent-native video representations via agentic auto-encoding. AVA-Encoder transforms a video into a knowledge graph (KG) representation and then reconstructs it back into video. Its hierarchy and state nodes store structured text, while a linked asset layer holds generated images, audio, and video. Typed edges preserve the relations between these text descriptions and assets in a form that agents can easily understand, query, and edit. The video reconstruction differences drive a textual-gradient optimization framework, which expresses evaluation feedback as natural-language update directions for Data-Independent Encoding Policy Pseudo-Training in the outer loop and optional Data-Dependent KG Representation Refinement in the test-time inner loop. Extensive experiments show that AVA-Encoder improves by 20.7 percentage points over the strongest external baseline. In the controlled policy-only setting, its pseudo-trained shot-level Agentic Video Encoder policy also outperforms a carefully human-tuned policy while using 74.3% fewer system-prompt tokens. We release the complete AVA-Encoder framework, a reliable agentic video reconstruction benchmark, and the first dataset of high-quality film KG representations. |
cite
@misc{li2026ava,
title={AVA-Encoder: Towards Agent-Native Video Representation Learning},
author={Chuyue Li and Jinpeng Yu and Haozhe Wang and Tian Xueyun and Zhijing Zhang and Bingnan Li and Shuqi Gu and Kan Ren and Jiaming Liu and Ruihua Hua},
year={2026},
eprint={2608.12313},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2608.12313}
}