System 1 vs System 2 model routing: which decisions a decision model should take, and which still need a reasoning model

Route a decision to a System 1 model only when it is bounded, labelled and cheap to get wrong. A 0.90 confidence gate kept 84.0% accuracy at $0.69 per 1,000.