Automatically Evaluating AI Coding Assistants with Each Git Commit

AI coding assistants are performing an ever-increasing share of software engineering work. Generic LLM coding benchmarks fail to capture the effectiveness of a coding assistant for specific use cases and workflows.