SWE-Duel: Towards a Self-Scaling Arena for Coding Agents
Static coding benchmarks are fixed at release. Over time their tasks leak into training data, and stronger agents eventually saturate them. We argue that a benchmark should instead aim to be self-scaling: its tasks and oracles are created at evaluation time, so no content exists in advance to leak,…
arXiv - cs.CL (NLP)2 min read