| title | SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control |
|---|---|
| authors | Zhida Zhang, Jie Ma, Zhan Peng, Haoxue Wu, Yang Han, Jun Liang, Jie Cao, Jing Li |
| date | 2026-05-27 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.27891 |
| affiliation | Orange Team |
| upvotes | 4 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | The narrative quality of a video fundamentally determines its perceptual value. Although existing video generation methods can produce visually appealing content, they predominantly rely on sparse conditioning signals such as text prompts or first/last frames, which limits precise control over narrative structure and temporal pacing. In this paper, we propose SmartDirector, a framework that enhances the narrative capacity of video generation models through multiple keyframes. SmartDirector supports flexible generation scenarios including single-shot generation, multi-shot narrative synthesis, and video extension. The framework operates in two stages: Director-Gen generates a low-resolution video conditioned on the provided keyframes, and Director-SR refines the output by exploiting high-resolution keyframes as semantic anchors to recover fine-grained details. To enable robust multi-keyframe training, we construct a data pipeline that curates single-shot and multi-shot sequences from movies. Extensive experiments demonstrate that SmartDirector substantially outperforms existing state-of-the-art approaches. We will release the code to facilitate further research. |
cite
@misc{zhang2026smartdirector,
title={SmartDirector: Keyframe-Conditioned Cinematic Video Generation with Narrative Pacing Control},
author={Zhida Zhang and Jie Ma and Zhan Peng and Haoxue Wu and Yang Han and Jun Liang and Jie Cao and Jing Li},
year={2026},
eprint={2605.27891},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.27891}
}