Model drift: the system never crashes, it just quietly starts being wrong

When a web service breaks it returns a 500 and someone gets paged. When a machine learning model breaks, nothing happens. The service stays up, latency is normal, the error rate is zero. Only the predictions gradually become wrong.

That silence is what makes model drift one of the most expensive problems in production.

Three kinds of drift, three responses

"Drift" is not one phenomenon. If you do not separate them, you put the alarm in the wrong place.

Covariate shift. The distribution of the input data changes. A new customer segment arrives, a campaign shifts traffic, a sensor calibration slips. The model applies the same rule, but to a different world.

Prior shift. The distribution of the target changes. Fraud rates rise seasonally; the model stays optimistic relative to the base rate it was trained on.

Concept drift. The relationship between input and output changes. This is the worst kind, because it can happen with no change in the input distribution at all. Fraudsters change technique; the same features now mean something else.

What we measure

Labels usually arrive late — you learn whether a loan was repaid months afterwards. So we split monitoring into two layers.

Immediately measurable (no labels needed):

• Distribution distance per feature (PSI or Jensen-Shannon)
• Distribution of prediction scores
• Missing-value rate and out-of-range counts
• Shift in the feature correlation matrix

Delayed (labels required):

• Accuracy, precision, recall — per segment
• Calibration curve
• Performance gap between segments

The second group tells the truth but tells it late. The first warns early but can raise false alarms. You need both.

How we pick thresholds

General rules such as "PSI > 0.2 means drift" are a starting point, not a decision rule. Our approach:

1. After the model goes live, collect measurements for four weeks with no alerting at all.
2. From that period's distribution, take the 95th and 99th percentiles for each metric.
3. Make the 95th the "investigate" threshold and the 99th the "intervene" threshold.
4. Recompute the baseline every quarter.

In domains with seasonality this produces far fewer false alarms than generic thresholds.

def drift_thresholds(baseline: pd.Series) -> tuple[float, float]:
    """Derive investigate/intervene thresholds from the observed baseline."""
    return baseline.quantile(0.95), baseline.quantile(0.99)

What happens when an alarm fires

An alarm without a runbook gets muted eventually. Our flow:

1. Verify. Is the drift real, or a glitch in the data pipeline? Most first alarms are the latter.
2. Scope it. All traffic, or one segment?
3. Measure impact. Has the drift reached a business metric? Some drift is harmless.
4. Decide. Retrain, add a feature, adjust a threshold — or accept it and write it down.

The fourth option is missing on most teams. "There is drift but no impact, we accept it" is a valid decision, as long as it is documented.

Retraining: automatic or manual?

Automatic retraining looks appealing but carries a risk: a model trained on corrupted data reinforces the drift instead of correcting it.

The middle path we use:

• Retraining is triggered automatically, but produces a candidate model; it does not go live.
• The candidate is compared with the incumbent on a fixed validation set and on recent data.
• If it wins on both, it goes to 5% of traffic in shadow mode.
• If nothing breaks in two days, it takes over progressively.

For one client this loop cut average model refresh time from six weeks to four days — without a single rollback along the way.

Summary

Model monitoring is not about building a dashboard. It is about writing down three things:

• Which metric, at which threshold, alerts whom?
• What are the first three steps when the alert fires?
• On what evidence does a retrained model go live?

Without those three, the dashboard becomes a pile of charts nobody looks at.