| title | Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization |
|---|---|
| authors | Gabriele Oliaro, Jaeseong Lee, Zhaoyuan Su, Sreeram Vennam, Zhihao Jia, Samyam Rajbhandari |
| date | 2026-11-02 |
| venue | arXiv (preprint, not peer reviewed) |
| id | arXiv:2611.03517 |
| affiliation | Snowflake |
| upvotes | 2 (Hugging Face Daily Papers, 2026-11-06) |
| links | abs pdf hf |
| cite | |
| abstract | LLM agents for GPU kernel optimization sit at two ends of a structure-freedom tradeoff. Scripted agents fix what the model does at each step, via a learned policy, an optimization menu, or fixed roles, and are sample-efficient where the recipe applies and stall where it does not. Open-ended agents let a self-directed model mutate a population of kernels and pay for the freedom in trials: thousands of evaluations, repeated edits, and wrong candidates. Both respond to hard targets by equipping the agent with more: parsed profiler output, disassembly analysis, retrieval over documentation and libraries, and stores of lessons. We argue that both lack direction, a hypothesis about where the remaining gap is, and that a short question asked at the right moment supplies it without narrowing what the agent may do next. We present KernelSeek, a question-driven agent that makes an existing CUDA kernel faster with no operator-specific knowledge and no faster implementation to imitate: a general-purpose coding agent, a minimal harness that enforces a verifiable goal, and questions at decision points, first from a human, then from a bank the agent applies to itself. On the same task, evaluator, hardware, and week, a tool-rich autonomous agent built in the prevailing style plateaued at 0.50x FlashAttention-3 after 133 model-driven steps; KernelSeek reached 1.13x. It then produced dependency-free FlashAttention forward kernels for Hopper and Blackwell, single CUDA files with no CUTLASS, CuTe, cuBLAS, or cuDNN, correct on all 182 scenarios per architecture and 1.16x FlashAttention-3 and 1.46x FlashAttention-4 (geometric mean). Across 162 human messages and 11.5K agent tool calls, every large gain followed a question that moved the agent to a different layer of the system or rejected a claimed limit, never one that supplied information. We release the kernels, harness, question bank, and all traces. |
bibtex
@misc{oliaro2026ask,
title={Ask, Don't Equip: Question-Driven Coding Agents for Deep Kernel Optimization},
author={Gabriele Oliaro and Jaeseong Lee and Zhaoyuan Su and Sreeram Vennam and Zhihao Jia and Samyam Rajbhandari},
year={2026},
eprint={2611.03517},
archivePrefix={arXiv},
url={https://arxiv.org/abs/2611.03517}
}