VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech

Yi-Cheng Lin; Yusuke Hirota; Sung-Feng Huang; Hung-yi Lee

title	VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech
authors	Yi-Cheng Lin, Yusuke Hirota, Sung-Feng Huang, Hung-yi Lee
date	2026-07-03
venue	arXiv (preprint, not peer reviewed)
id	arXiv:2604.17248
links	abs pdf hf
abstract	Large Audio-Language Models (LALMs) are increasingly integrated into daily applications, yet their generative biases remain underexplored. Existing speech fairness benchmarks rely on synthetic speech and Multiple-Choice Questions (MCQs), both offering a fragmented view of fairness. We propose VIBE, a framework that evaluates generative bias through open-ended tasks such as personalized recommendations, using human-recorded speech. Unlike MCQs, our method allows stereotypical associations to manifest organically without predefined options, making it easily extensible to new tasks. Evaluating 12 state-of-the-art LALMs reveals systematic biases in realistic scenarios. Both gender and accent cues trigger statistically significant distributional shifts, and bias magnitude is strongly task-dependent.

cite

@misc{lin2026vibe,
  title={VIBE: Voice-Induced open-ended Bias Evaluation for Large Audio-Language Models via Real-World Speech},
  author={Yi-Cheng Lin and Yusuke Hirota and Sung-Feng Huang and Hung-yi Lee},
  year={2026},
  eprint={2604.17248},
  archivePrefix={arXiv},
  url={https://arxiv.org/abs/2604.17248}
}