AI Evaluation is Becoming an Exciting Standalone Discipline

Having worked on robustness problems during my PhD, I see many of the characteristics appearing in the evaluation of LLMs and AI systems. Adversarial attacks such as jailbreaks are becoming more relevant, edge cases finally become relevant, test sets and benchmarks are not static anymore but…