| title | Agents' Last Exam |
|---|---|
| authors | Yiyou Sun, Xinyang Han, Weichen Zhang, Yuanbo Pang, Tianyu Wang, Yuhan Cao, Yixiao Huang, Chris Duroiu, Haoyun Zhang, Jeffrey Lin, Weishu Zhang, Tyler Zeng, Ying Yan, Bo Liu, Hanson Wen, Mingyang Xu, Xiaoyuan Liu, Zimeng Chen, Weiyan Shi, Amanda Dsouza, Vincent Sunn Chen, Patrick Bryant, Carl Boettiger, Yamini Rangan, Bradley Rothenberg, Kyle Steinfeld, Arvind Rao, Tapio Schneider, Georgios Yannakakis, Laure Zanna, Kaan Ozbay, Ida Sim, Tarek Zohdi, George Em Karniadakis, Jack Gallant, Teresa Head-gordon, Yushan Li, Wenxi Deng, Tao Sun, Huiqi Wang, Zhun Wang, Justin Xu, Chris Yuhao Liu, Yafei Cheng, Rongwang Hu, Aras Bacho, Shengcao Cao, Zengyi Qin, Yixiong Chen, Hengduan Fan, Hao Liu, Lin Zeng, Shashank Muralidhar Bharadwaj, Litian Gong, Yingxuan Yang, Maojia Song, Ruheng Wang, Zongzheng Zhang, Honglin Bao, Shuo Lu, Jianhong Tu, Zhonghua Wang, Zheng Zhang, Zijiao Chen, yanqiong Jiang, Zhendong Li, Bohan Lyu, Chang Ma, Peiran Xu, Benran Zhang, Shangding Gu, Haoyue Hua, Haoyang Li, Wanzhe Liao, Chengzhi Liu, Junbo Peng, Haoran Sun, Zechen Xu, Bo Chen, Jiayi Cheng, Yi Jiang, Keying Kuang, Yuan Li, Youbang Pan, Ziyan Rao, Alexander Schubert, Yifan Shen, Vincent Siu, Xiatao Sun, Kangqi Zhang, Xiaopan Zhang, Yuchen Zhu, Ishaan Singh Chandok, Lei Ding, Jingxuan Fan, Andrew Glover, Jiaming Hu, Yiran Hu, Wenbo Huang, Zixin Jiang, Haoran Jin, Lukas Kim, Ming Liu, Yang Liu, Alireza Rafiei, Xuhuan Shen, Kunyang Sun, Sophia Sun, Ting Sun, Eric Wang, Yixin Wang, Hanwen Xing, Sihan Xu, Yuzheng Xu, Zhongxing Xu, Zhiling Yan, Boqin Yuan, Ruiqi Zhang, Yifan Zhang, Zibo Zhao, Liana, Santanu Bosu Antu, Haoyue Bai, Carlo Bosio, Joseph Cavanagh, Patricia Cavazos-Rehg, Tianxing Chen, Xuewen Chen, Yipu Chen, Zhu Chenyu, Chen Dai, Stefano De Castro, Yunfu Deng, Kaustubh Dhole, Jiayuan Ding, Chenchen Du, Zhehang Du, Hao Fan, Run-ze Fan, Hengyu Fu, Shi Gu, Yifan Gu, Charlie Guo, Baihe Huang, Baixiang Huang, Rimika Jaiswal, Zhihan Jiang, Ran Jin, Erin Kasson, Xin Lan, Joseph Lee, Deren Lei, Chenyu Li, Daofeng Li, Haitao Li, Hongwei Li, Jingyan Li, Xiao Li, Yi Li, Yinsheng Li, Yuangang Li, Zhixu Li, Wenyu Liang, Longtai Liao, Kevin Qinghong Lin, AndyZeyi Liu, Che Liu, Jiaming Liu, Kaiyuan Liu, Xuan Liu, Pan Lu, Wenbo Lv, Yicheng Lv, Qiuyang Mang, Kyle Montgomery, Yuzhou Nie, Ruoxi Ning, Jorin Overwiening, Xu Pan, Layna Paraboschi, Core Francisco Park, Justin Purnomo, Swati Rajwal, Scott Rankin, Bixuan Ren, Yiren Rong, HaoYang Shang, Ventus Shaw, Fiona Shen, Jiawei Shen, Minqi Shi, Qiu Shi, Huaxiu Yao, Tianneng Shi, Jonah So, Vladislav Susoy, Hannah Szlyk, Haocheng Wang, Jialu Wang, Wei Wang, Xinyu Wang, Zehao Wang, Dowling Wong, Angela Wu, Dehao Wu, Fangyu Wu, Mengyuan "Millie" Wu, Yu Wu, Yuchen Wu, Yuhao Wu, Qingpo Wuwu, Weihang Xiao, Yongyi Xiong, Fan Xu, Ruiling Xu, Mingxuan Yan, Benjamin Yang, Jirong Yang, Sen Yang, Xiaoli Yang, Yushi Yang, Haoran Ye, Xiaohu Yu, Zhengming Yu, Chenlong Zhang, Chi Zhang, Hanning Zhang, Hanwen Zhang, Junge Zhang, Kunpeng Zhang, Song Zhang, Wenjin Zhang, Wenshuo Zhang, Ying Zhang, Yizhi Zhang, Brian Zhao, Qijian Zhao, Yimin Zhao, Yuhaohua Zheng, Liwei Zhou, Tianyue Zhou, Sichen Zhu, Siqi Zhu, Yan Zhu, Yishu Zhu, Jierui Zuo, Chonghao Cai, Helena Casademunt, Wenjia Chen, Benjamin Cheng, Nawen Deng, Rao Fu, Tianfu Fu, Yifan Han, Ren He, Zhenyu He, Qiao Jin, Lang Lang, Yuetai Li, Sylvia Liu, Lu Lu, Qing Lu, Subhabrata Mukherjee, Yunqi Ouyang, Yin Ren, Dawei Shi, Haoran Wu, Zhiyue Wu, Hannah Yao, Zhuoran Yi, Jenny Yu, Rhea Zhan, Hang Zhou, Blake Zhu, Junfan Zhu, Alan Yuille, Yang Liu, Russell Alan Poldrack, Jiachen Li, Zhenglu Li, Molei Tao, Jing Huang, Wenqi Shi, Costas Spanos, Lichao Sun, Chenguang Wang, Orson Xu, Zhen Dong, Hector Gomez, Aylin Caliskan, Ali Emami, Haimin Hu, Zhi Li, Lihui Liu, Murphy Niu, Yi Shao, Jianxin Sun, Mikko Tolonen, Ting Wang, Sanjiv Das, Yanjun Gao, Wenbo Guo, Erika J Schneider, Zhiyong Lu, Mark Mueller, Radha Poovendran, Somayeh Sojoudi, Dawn Song |
| date | 2026-06-03 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2606.05405 |
| affiliation | UC Berkeley |
| upvotes | 273 (Hugging Face Daily Papers, 2026-11-01) |
| links | abs pdf hf code |
| abstract | Recent AI systems have achieved strong results on a wide range of benchmarks, yet these gains have not translated into economically meaningful deployment across many professional domains. We argue that this gap is largely an evaluation problem: widely used benchmarks lack sustained performance measurement on real and economically valuable workflows. This paper introduces Agents' Last Exam (ALE), a benchmark designed to evaluate AI agents on long-horizon, economically valuable, real-world tasks with verifiable outcomes. Developed in collaboration with 250+ industry experts, ALE covers non-physical industries defined with reference to O*NET / SOC 2018 (the U.S. federal occupational taxonomy). It is organized around a task taxonomy with 55 subfields grouped into 13 industry clusters covering 1K+ tasks. Current results show that the hardest tier remains far from saturated: across mainstream harness and backbone configurations, the average full pass rate is 2.6%. ALE is designed as a living benchmark: its task pool grows continuously as new workflows and industries are onboarded. More broadly, ALE is intended not merely as another leaderboard, but as an instrument for closing the gap between benchmark success and GDP-relevant impact. |
cite
@misc{sun2026agents,
title={Agents' Last Exam},
author={Yiyou Sun and Xinyang Han and Weichen Zhang and Yuanbo Pang and Tianyu Wang and Yuhan Cao and Yixiao Huang and Chris Duroiu and Haoyun Zhang and Jeffrey Lin and Weishu Zhang and Tyler Zeng and Ying Yan and Bo Liu and Hanson Wen and Mingyang Xu and Xiaoyuan Liu and Zimeng Chen and Weiyan Shi and Amanda Dsouza and Vincent Sunn Chen and Patrick Bryant and Carl Boettiger and Yamini Rangan and Bradley Rothenberg and Kyle Steinfeld and Arvind Rao and Tapio Schneider and Georgios Yannakakis and Laure Zanna and Kaan Ozbay and Ida Sim and Tarek Zohdi and George Em Karniadakis and Jack Gallant and Teresa Head-gordon and Yushan Li and Wenxi Deng and Tao Sun and Huiqi Wang and Zhun Wang and Justin Xu and Chris Yuhao Liu and Yafei Cheng and Rongwang Hu and Aras Bacho and Shengcao Cao and Zengyi Qin and Yixiong Chen and Hengduan Fan and Hao Liu and Lin Zeng and Shashank Muralidhar Bharadwaj and Litian Gong and Yingxuan Yang and Maojia Song and Ruheng Wang and Zongzheng Zhang and Honglin Bao and Shuo Lu and Jianhong Tu and Zhonghua Wang and Zheng Zhang and Zijiao Chen and yanqiong Jiang and Zhendong Li and Bohan Lyu and Chang Ma and Peiran Xu and Benran Zhang and Shangding Gu and Haoyue Hua and Haoyang Li and Wanzhe Liao and Chengzhi Liu and Junbo Peng and Haoran Sun and Zechen Xu and Bo Chen and Jiayi Cheng and Yi Jiang and Keying Kuang and Yuan Li and Youbang Pan and Ziyan Rao and Alexander Schubert and Yifan Shen and Vincent Siu and Xiatao Sun and Kangqi Zhang and Xiaopan Zhang and Yuchen Zhu and Ishaan Singh Chandok and Lei Ding and Jingxuan Fan and Andrew Glover and Jiaming Hu and Yiran Hu and Wenbo Huang and Zixin Jiang and Haoran Jin and Lukas Kim and Ming Liu and Yang Liu and Alireza Rafiei and Xuhuan Shen and Kunyang Sun and Sophia Sun and Ting Sun and Eric Wang and Yixin Wang and Hanwen Xing and Sihan Xu and Yuzheng Xu and Zhongxing Xu and Zhiling Yan and Boqin Yuan and Ruiqi Zhang and Yifan Zhang and Zibo Zhao and Liana and Santanu Bosu Antu and Haoyue Bai and Carlo Bosio and Joseph Cavanagh and Patricia Cavazos-Rehg and Tianxing Chen and Xuewen Chen and Yipu Chen and Zhu Chenyu and Chen Dai and Stefano De Castro and Yunfu Deng and Kaustubh Dhole and Jiayuan Ding and Chenchen Du and Zhehang Du and Hao Fan and Run-ze Fan and Hengyu Fu and Shi Gu and Yifan Gu and Charlie Guo and Baihe Huang and Baixiang Huang and Rimika Jaiswal and Zhihan Jiang and Ran Jin and Erin Kasson and Xin Lan and Joseph Lee and Deren Lei and Chenyu Li and Daofeng Li and Haitao Li and Hongwei Li and Jingyan Li and Xiao Li and Yi Li and Yinsheng Li and Yuangang Li and Zhixu Li and Wenyu Liang and Longtai Liao and Kevin Qinghong Lin and AndyZeyi Liu and Che Liu and Jiaming Liu and Kaiyuan Liu and Xuan Liu and Pan Lu and Wenbo Lv and Yicheng Lv and Qiuyang Mang and Kyle Montgomery and Yuzhou Nie and Ruoxi Ning and Jorin Overwiening and Xu Pan and Layna Paraboschi and Core Francisco Park and Justin Purnomo and Swati Rajwal and Scott Rankin and Bixuan Ren and Yiren Rong and HaoYang Shang and Ventus Shaw and Fiona Shen and Jiawei Shen and Minqi Shi and Qiu Shi and Huaxiu Yao and Tianneng Shi and Jonah So and Vladislav Susoy and Hannah Szlyk and Haocheng Wang and Jialu Wang and Wei Wang and Xinyu Wang and Zehao Wang and Dowling Wong and Angela Wu and Dehao Wu and Fangyu Wu and Mengyuan "Millie" Wu and Yu Wu and Yuchen Wu and Yuhao Wu and Qingpo Wuwu and Weihang Xiao and Yongyi Xiong and Fan Xu and Ruiling Xu and Mingxuan Yan and Benjamin Yang and Jirong Yang and Sen Yang and Xiaoli Yang and Yushi Yang and Haoran Ye and Xiaohu Yu and Zhengming Yu and Chenlong Zhang and Chi Zhang and Hanning Zhang and Hanwen Zhang and Junge Zhang and Kunpeng Zhang and Song Zhang and Wenjin Zhang and Wenshuo Zhang and Ying Zhang and Yizhi Zhang and Brian Zhao and Qijian Zhao and Yimin Zhao and Yuhaohua Zheng and Liwei Zhou and Tianyue Zhou and Sichen Zhu and Siqi Zhu and Yan Zhu and Yishu Zhu and Jierui Zuo and Chonghao Cai and Helena Casademunt and Wenjia Chen and Benjamin Cheng and Nawen Deng and Rao Fu and Tianfu Fu and Yifan Han and Ren He and Zhenyu He and Qiao Jin and Lang Lang and Yuetai Li and Sylvia Liu and Lu Lu and Qing Lu and Subhabrata Mukherjee and Yunqi Ouyang and Yin Ren and Dawei Shi and Haoran Wu and Zhiyue Wu and Hannah Yao and Zhuoran Yi and Jenny Yu and Rhea Zhan and Hang Zhou and Blake Zhu and Junfan Zhu and Alan Yuille and Yang Liu and Russell Alan Poldrack and Jiachen Li and Zhenglu Li and Molei Tao and Jing Huang and Wenqi Shi and Costas Spanos and Lichao Sun and Chenguang Wang and Orson Xu and Zhen Dong and Hector Gomez and Aylin Caliskan and Ali Emami and Haimin Hu and Zhi Li and Lihui Liu and Murphy Niu and Yi Shao and Jianxin Sun and Mikko Tolonen and Ting Wang and Sanjiv Das and Yanjun Gao and Wenbo Guo and Erika J Schneider and Zhiyong Lu and Mark Mueller and Radha Poovendran and Somayeh Sojoudi and Dawn Song},
year={2026},
eprint={2606.05405},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2606.05405}
}