| title | BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generation |
|---|---|
| authors | Max Van Puyvelde, Ibrahim Gulluk, Wim Van Criekinge, Olivier Gevaert |
| date | 2026-06-17 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.19651 |
| affiliation | Gevaert Lab |
| upvotes | 10 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf |
| abstract | Three-dimensional (3D) brain MRI is central to clinical neurology and neuro-oncology, where generative models could augment under-represented cohorts, simulate disease trajectories, and support privacy-preserving data sharing. Latent diffusion has been the go-to solution for modeling imaging data, but it places two competing demands on the tokenizer: encoder embeddings must retain the clinical information that downstream tasks act on, and the decoder must reconstruct anatomically faithful volumes. Existing reconstruction-driven tokenizers achieve the second at the expense of the first. To address this, we introduce a fully volumetric masked-autoencoder (MAE) based tokenizer for 3D brain MRI latent diffusion, decoupling encoder and decoder: a frozen 3D MAE encoder produces clinically informative embeddings, while a dedicated CNN decoder reconstructs voxels from a linear projection of those embeddings. We pretrain the encoder on 35,309 volumes from 18 public cohorts spanning four modalities, ten disease categories, and 200+ acquisition sites, and demonstrate its dual utility in two settings. First, on a 23-task linear-probing benchmark, the encoder outperforms or matches SOTA models (i.e., BrainIAC, BrainSegFounder, and MedicalNet) on 21 of 23 tasks. Second, a conditional diffusion transformer (DiT) trained on these clinically informative embeddings supports both conditional generation across six variables and patient-specific longitudinal forecasting. Together these results establish a single 3D brain-MRI embedding space capable of both downstream clinical tasks and controllable generation. |
cite
@misc{puyvelde2026braing,
title={BrainG3N: A Dual-Purpose Tokenizer for Controllable 3D Brain MRI Generation},
author={Max Van Puyvelde and Ibrahim Gulluk and Wim Van Criekinge and Olivier Gevaert},
year={2026},
eprint={2606.19651},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.19651}
}