Fairness metrics

Fairness metrics turn the abstract goal of non-discrimination into numbers that can be tested and monitored. Common families include demographic parity (similar rates of positive outcomes across groups), equalized odds or equal opportunity (similar true positive and false positive rates), predictive parity (similar precision) and individual fairness (similar people treated similarly). ISO/IEC TR 24027 and NIST SP 1270 describe these measures and their limits.

A key point for practitioners and auditors is that several of these metrics cannot all be satisfied at once when base rates differ between groups. Choosing a metric is therefore a documented governance decision tied to the context, the harm being prevented and applicable law, not a purely technical default. Results should appear in model cards and be monitored after deployment, because drift can reintroduce disparities.

Example: a fraud model shows equal overall accuracy but flags legitimate transactions from one region twice as often; the team adopts false positive rate parity as its release criterion.