| title | I Have a Stream: Making Self-Supervised Learning Work on Continuous Video |
|---|---|
| authors | Ivan Martinović, Lukas Knobel, Yuki M. Asano |
| date | 2026-09-30 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2609.40333 |
| affiliation | Fundamental AI Lab at UTN |
| upvotes | 9 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Self-supervised learning draws inspiration from infant visual development, yet standard training pipelines bear little resemblance to it: images are independently sampled and globally shuffled across epochs. We study self-supervised learning from continuous video streams, where frames are consumed in temporal order using strict sliding-window batches, without global reshuffling or multi-epoch replay. To this end, we construct WT++, a 95-hour urban walking-tour video dataset for streaming pretraining. Combined with a comprehensive evaluation suite we find that contrastive and distillation-based methods struggle in this setting, while MAE is more robust but still falls short of standard i.i.d. pretraining. We find that high inter-batch similarity, caused by sliding-window consumption across consecutive batches, does not explain this gap. The main challenge is high intra-batch similarity, where frames within each batch are near-duplicates. To mitigate this, we propose StreamMAE, which preserves the core MAE reconstruction objective while adapting the input pipeline with stream-aware regularization and motion-biased crop selection. StreamMAE outperforms streaming baselines, matches i.i.d. MAE trained on the same video data, remains competitive with ImageNet-pretrained MAE, and scales positively as the pretraining stream grows from 12 to 95 hours. |
cite
@misc{martinovi2026i,
title={I Have a Stream: Making Self-Supervised Learning Work on Continuous Video},
author={Ivan Martinović and Lukas Knobel and Yuki M. Asano},
year={2026},
eprint={2609.40333},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2609.40333}
}