LLM-Assisted2 min read
When F1 Is Not Enough: Auditing Codebooks for LLM-Assisted Annotation
Large language models (LLMs) are increasingly used for text annotation in social science, but standard performance metrics do not explain why errors occur. Low F1 may reflect limitations of the annotation system, defects in the written codebook, or inconsistency in reference labels.
By Yuehong Cassandra Tai, Qunhua LiarXiv - cs.AI
Read the full story
Originally published by arXiv - cs.AI on Nov 9, 2026.
Comments
Sign in to join the discussion.