Model Validation & Audit Policy¶


Code	`DT-013`
Domain	Data & Technology
Owner	Chief Risk Officer
Status	Draft
Applicability	Platform
Jurisdiction	NZ + AU
Business domain	BD08
Review date	2027-05-21

Regulations: APS 113 IRB · CPS 220 Risk Management · CPS 230 Operational Risk Management · RBNZ BS2A/BS2B Capital · DTA Capital Standard · DTA Risk Management Standard · RBNZ Model Risk Guidance · IFRS 9 · SR 11-7 Model Risk · PRA SS1/23 Model Risk ¶

The platform builds and ships quantitative models that customers use for regulatory capital calculation, expected-credit-loss provisioning, AML detection, fraud scoring, and credit decisions. Every one of those models is subject to independent validation and audit at every customer that deploys it. This policy governs how the platform discharges its obligations in that chain — and how it makes the customer's own obligations dramatically easier to discharge.

Internal build standard¶

The platform adopts SR 11-7 (Federal Reserve / OCC, 2011) and PRA SS1/23 (Bank of England / PRA, 2024) simultaneously as its internal model-documentation and validation-evidence build standard. Building to both simultaneously:

Exceeds RBNZ and APRA requirements today
Pre-positions for UK and Irish market entry without re-work
Provides the strongest possible starting point for any customer's own validation function

The three-line model¶

Three distinct accountabilities apply to every platform model:

Model owner — the platform module team accountable for the model's design, documentation, and ongoing performance
Validation function — an independent function that validates the model; the platform may commission independent third-party pre-validation, but the customer's validation function must also independently review and confirm
Audit — internal audit (or equivalent), at minimum annually, assessing whether the validation framework and governance are effective

These three accountabilities must not be conflated. The platform is the developer; it cannot also be the independent validator for the same model.

The non-delegable boundary¶

This boundary must be stated clearly and repeated in customer contracts and onboarding:

The platform cannot discharge the customer's own model validation obligation. SR 11-7, APS 113, and the RBNZ framework are explicit that third-party and vendor models must be validated by the customer's own validation function with the same rigour as in-house models.
The platform cannot be the customer's independent reviewer. The annual independent review required by APS 113 must be performed by a function independent of the model's development. The platform is the developer.
The platform cannot obtain the customer's regulatory approvals. IRB accreditation (APRA), capital-model approval, and the RBNZ internal models compendium agreement are between the customer and its regulator.

What the platform can do — and what makes this commercially powerful — is make the customer's validation faster, cheaper, and more likely to succeed by doing all the centralisable work once.

Evidence packs — definition of done¶

A model-bearing module is not considered Deployed until its evidence pack exists and meets the reconstruction standard. The evidence pack for each material model comprises:

Model documentation to reconstruction standard — purpose, methodology, theory, assumptions, limitations — documented so a knowledgeable third party could recreate the model without access to source code
Data requirements and data-quality / representativeness guidance
Development and testing evidence, including challenger-model benchmarking
Implementation specification with code-to-spec traceability and reproducible-build evidence
Back-testing and outcomes-analysis methodology, plus reference results on representative data
Ongoing-monitoring tooling description and the live dashboards shipped in the deployment (MOD-174)
An independent third-party pre-validation report against SR 11-7 / PRA SS1/23 (for material models)
A model register entry pre-populated to APS 113 / RBNZ compendium field requirements (MOD-173)
A change-control and re-validation playbook, including the RBNZ re-approval pathway for capital models (MOD-175)
A clear statement of the validation boundary — what the customer must still do themselves

This definition-of-done applies from the first deployment of any new model module, and to any material model update.

Model change control and the RBNZ re-approval pathway¶

The Westpac NZ case (2016) demonstrated that operating a capital model without RBNZ approval — or making changes to an approved model without approval — is a breach of the conditions of registration, not merely a risk-management deficiency. The platform treats this as existential.

Rule: The platform will never push a material change to a capital model in a NZ customer's production environment without: - Generating the change-control record and re-validation evidence (MOD-175) - Supplying the artefacts the customer needs to seek RBNZ re-approval - The customer's deployment gate being open only after their RBNZ approval is in hand

This requirement is enforced in the deployment pipeline via the maker-checker enforcement gate (MOD-168) wired to MOD-175.

CPS 230 material service provider packaging¶

Australian customers using the platform for model-driven critical operations must manage the platform as a material service provider under CPS 230 (effective 1 July 2025). The evidence packs, model documentation, and change-control playbook are specifically packaged to satisfy the MSP documentation obligations under CPS 230, so that customers can include the platform on their MSP register with adequate supporting evidence.

Model risk tiers¶

Models are risk-tiered by materiality and regulatory scrutiny. Tier 1 (highest) models receive the full evidence pack and mandatory independent third-party pre-validation:

Module	Model	Tier
MOD-031	ECL calculation & GL posting	1 — annual external audit, IFRS 9
MOD-030	Stage allocation model	1 — highest IFRS 9 scrutiny area
MOD-033	RWA & capital ratio engine	1 — RBNZ conditions of registration
MOD-028	Credit score & risk rating	1 — APS 113 IRB gate
MOD-034	Stress testing scenario engine	2
MOD-035	IRRBB / EVE / NII model	2
MOD-017	AML ML behavioural scoring	2
MOD-023	Transaction fraud scorer	2
MOD-039	Customer risk score model	2
MOD-055	Onboarding fraud scoring engine	2
MOD-041	Categorisation & merchant enrichment	3

Tier 1 models require an independent third-party pre-validation report before first deployment. All tiers require a complete evidence pack.

Ongoing monitoring¶

Live ongoing-monitoring tooling (MOD-174) runs automatically in every customer deployment. This generates the monitoring evidence the customer's validation function needs, including drift detection, population-stability indices, calibration tracking, and performance dashboards. Monitoring evidence is generated without manual trigger, turning a recurring customer engagement cost into an automatic platform feature.

Satisfying modules¶

Module	Name	Mode	Description
MOD-173	Model risk register & inventory	`GATE`	A model-bearing module cannot be marked Deployed without a corresponding register entry; the deployment pipeline gate enforces this check before promotion to production.
MOD-174	Model performance monitoring & drift detection	`AUTO`	Drift detection, population-stability indices, calibration tracking and back-test refresh run automatically on schedule for every deployed model — no manual trigger required; satisfies the ongoing-monitoring pillar of DT-013.
MOD-175	Model change control & re-approval workflow	`GATE`	No model change reaches production without a completed change-control record; for NZ capital models the gate requires RBNZ re-approval artefacts to be present before the deployment lock is released — directly preventing a conditions-of-registration breach of the Westpac NZ type.

Part of Data & Technology · Governance overview Compiled 2026-05-22 from source/entities/policies/DT-013.yaml