When to use LLMs vs traditional ML
Most teams reach for LLMs because they're trendy, when traditional ML would be cheaper, faster, more accurate. We frame the decision tree for picking between rule-based, classical ML, fine-tuned BERT, and frontier LLMs.
- · Decision tree: rules → regex → classical ML → small LM → frontier LLM
- · When LLMs are the wrong tool — classification, structured extraction at scale
- · Cost math: GPT-4 vs Claude vs Llama 70B vs your own fine-tune
- · Latency budgets: when 2-second LLM responses are unacceptable
- · The 'eval gap' — what makes LLM systems hard to test