CARAT: Do Materials LLMs Reason or Recite?
When a materials LLM answers a question about crystal structure, does it reason from the structure or copy an answer already printed in its input? Accuracy cannot tell: a structural description often prints the very field it is scored against.
arXiv - cs.AI2 min read