How do you measure bias or fairness in a machine learning model?

There's no single fairness number — you pick metrics that fit the harm you care about, usually comparing outcomes across protected groups. Common definitions: demographic parity (equal positive rates across groups), equalized odds (equal true-positive and false-positive rates), and calibration (predicted scores mean the same thing per group). A key result is that these often can't all hold at once when base rates differ — the impossibility theorems (Kleinberg, Chouldechova) mean you must choose. In practice you disaggregate error metrics by subgroup, use tools like Fairlearn or Aequitas, and involve domain and legal stakeholders to decide which fairness notion is appropriate. Fairness is a sociotechnical choice, not a metric you maximize blindly.