Model performance monitoring & drift detection¶


ID	`MOD-174`
System	SD06
Repo	`bank-risk-platform`
Build status	Not started
Deployed	No

Snowflake-native ongoing-monitoring platform that runs automatically in every customer deployment. Generates the monitoring evidence the customer's validation function needs — drift detection, population-stability indices, calibration tracking, back-test refresh, and performance dashboards — without any manual engagement. Turns a recurring customer cost (typically a quarterly or annual model monitoring engagement) into an automatic platform feature.

Why ongoing monitoring matters¶

SR 11-7 and PRA SS1/23 both treat ongoing monitoring as a core validation pillar. APRA's APS 113 requires evidence that models continue to perform as intended, and IFRS 9's ECL models attract scrutiny over whether staging logic and PD/LGD estimates remain appropriate as the economic cycle evolves. Without automated monitoring, smaller institutions typically rely on point-in-time annual validation engagements — which means model drift can go undetected for months.

The platform is in a unique position: it runs the same models across many customers and can aggregate performance evidence that no single institution could generate independently.

What it monitors per model¶

Drift detection. Population stability index (PSI) on input features; characteristic stability index (CSI) per predictor. Alerts generated when PSI exceeds configurable thresholds (default warn >0.1, critical >0.2).

Calibration tracking. For PD models: Hosmer-Lemeshow test, Brier score, binomial test on actual default rates vs predicted. For ECL models: monthly comparison of provision outcomes against ECL estimates. For scoring models: Gini coefficient, KS statistic, AUC trend over rolling 12-month window.

Back-test refresh. Automated quarterly back-test using outcomes data from MOD-048 (system decision log) and MOD-047 (agent action logger) where deployed. Results appended to the evidence pack in MOD-173.

Population monitoring. Input distribution monitoring; missing value rates; out-of-range flags. Especially important for AI/ML models (MOD-017, MOD-023, MOD-039, MOD-055) where data distribution shift is the primary failure mode.

Dashboards¶

Snowflake-native Streamlit dashboards per model tier. Tier 1 models (MOD-028, 030, 031, 033) have a full performance dashboard accessible to the customer's validation function and internal audit. All models have a model-health summary card displayed on the SD06 risk-platform overview.

Evidence pack integration¶

All monitoring outputs are automatically written to the model's evidence pack record in MOD-173, timestamped and immutable. The customer's validator can pull the full monitoring history at any time. This is the artefact that closes the "ongoing monitoring" criterion in a model validation — supplied automatically, not built manually.

Module dependencies¶

Depends on¶

Module	Title	Required?	Contract	Reason
MOD-173	Model risk register & inventory	Required	—	Model register provides the authoritative inventory of models to monitor; MOD-174 is not deployable without MOD-173 defining the monitored model set.
MOD-102	Snowflake account configuration & governance	Required	—	Snowflake compute (MOD-102) runs drift-detection and calibration jobs natively; required for all monitoring workloads.
MOD-047	Agent action logger	Optional	—	Agent action log provides model decision outputs for outcomes analysis and back-testing; enriches monitoring evidence but monitoring runs without it.
MOD-048	System decision log	Optional	—	System decision log provides model input/output records for calibration tracking; enriches monitoring evidence but is not required.

Required by¶

Module	Title	As	Contract
MOD-177	SD06 risk dashboard renderer	Optional enhancement	—

Policies satisfied¶

Policy	Title	Mode	How
DT-013	Model Validation & Audit Policy	`AUTO`	Drift detection, population-stability indices, calibration tracking and back-test refresh run automatically on schedule for every deployed model — no manual trigger required; satisfies the ongoing-monitoring pillar of DT-013.
DT-005	Model Risk Management Policy	`AUTO`	Continuous model performance monitoring satisfies the ongoing-monitoring pillar of the Model Risk Management Policy without requiring manual customer engagement.

Capabilities satisfied¶

(No capabilities mapped)

Part of SD06 — Snowflake Analytics & Risk Platform Compiled 2026-05-22 from source/entities/modules/MOD-174.yaml