| title | Functional Attention: From Pairwise Affinities to Functional Correspondences |
|---|---|
| authors | Jiefang Xiao, Maolin Gao, Simon Weber, Guandao Yang, Daniel Cremers |
| date | 2026-05-29 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2605.31559 |
| affiliation | Technical University of Munich |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Learning mappings between infinite-dimensional function spaces, or operator learning, is essential for many machine learning applications. Although transformer-based operators are popular, they often rely on token-wise attention. These methods treat continuous fields as discrete tokens and usually ignore the global functional structure. We introduce Functional Attention, which reinterprets attention as a functional correspondence between adaptive bases. Inspired by geometric functional maps, our method replaces softmax affinities with structured linear operators. This yields a compact, generalizable, resolution-invariant representation that explicitly captures global dependencies. Experiments demonstrate that Functional Attention can match state-of-the-art performance in many operator learning tasks, including solving PDEs, 3D segmentation, and regression, while remaining robust to varying discretizations. Project page is available at https://github.com/xjffff/FUNCATTN. |
cite
@misc{xiao2026functional,
title={Functional Attention: From Pairwise Affinities to Functional Correspondences},
author={Jiefang Xiao and Maolin Gao and Simon Weber and Guandao Yang and Daniel Cremers},
year={2026},
eprint={2605.31559},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2605.31559}
}