Model drift

Models are trained on a snapshot of the world, and the world moves. Data drift occurs when the distribution of inputs changes (new customer segments, a new sensor, seasonal behaviour); concept drift occurs when the relationship between inputs and the correct answer changes (fraudsters adapt, a pandemic reshapes demand). Both can reduce accuracy and can quietly reintroduce bias for some groups even when overall metrics look stable.

The NIST AI RMF expects AI systems to be monitored for performance and trustworthiness in deployment (Measure and Manage functions), and ISO/IEC 42001 controls cover operation and monitoring across the lifecycle. Under the EU AI Act, providers of high-risk systems run post-market monitoring. In practice, teams define drift metrics and thresholds, compare live data to reference data, monitor fairness metrics by group, and trigger retraining, rollback or incident handling when limits are crossed.

Example: a demand forecasting model trained before a change in shopping habits starts over-ordering stock until monitoring alerts trigger retraining.