The JetBrains Blog13 min read
Coding-Agent Benchmarks Should Match Their Users’ Task Flows
Left: SWE-Bench Pro, where each task is a single user message, almost always a new feature or a bug fix. Right: the first five developer messages in sessions of software engineers working in JetBrains IDEs, counting only sessions with at least three messages.
By Igor SlinkoJetBrains AI Blog

Read the full story
Originally published by JetBrains AI Blog on Oct 20, 2026.
Comments
Sign in to join the discussion.