Why is evaluating LLM applications harder than testing traditional software?

Traditional software is deterministic — the same input gives the same output, so a test asserts an exact expected value. LLMs are probabilistic and open-ended: the same prompt can yield different wordings that are all correct, or subtly wrong ones that look fine. There's often no single 'right' answer to assert against, quality is partly subjective (is this summary good?), and the model can regress on things you never tested when you change a prompt or model version. So you can't rely on exact-match unit tests alone. You need evaluation on representative datasets using a mix of automated metrics, model-graded scoring, and human review — and you re-run it continuously, because behaviour drifts.