Skip to content

ADR-045: Test data and environment fidelity strategy
Status Accepted
Date 2026-04-27
Deciders CTO, Head of Platform, Head of Engineering
Affects repos bank-platform, bank-core, bank-kyc, bank-aml, bank-payments, bank-credit, bank-risk-platform, bank-app

Status: Accepted — 2026-04-27

Context

Integration tests require realistic data to already exist in the database before the test runs. An empty schema is not a valid test baseline — tests for the balance engine require accounts with balances, tests for the payment validator require customers with verified KYC status, tests for the accrual calculator require accounts with interest rate assignments.

Beyond individual module integration tests, there is a higher-order confidence requirement: the independently-deployed modules must work together as a bank. An account opened via MOD-007 must be readable by MOD-003; a payment submitted through SD04 must post to the ledger via MOD-001 and trigger a notification via MOD-063. Validating this cross-module behaviour requires a continuously operating environment — not a one-time test run.

The Neon database (ADR-001, MOD-103) is an external managed service. It persists across SST deployments. Deploying a new version of a module does not tear down the database. UAT therefore accumulates state over time, which is desirable — it should look like a bank that has been running for months, not a freshly-provisioned schema.

Decision

1. Seed data loader as an auto-triggered Lambda (MOD-158).

A seed data loader Lambda is deployed by MOD-158. On first deploy to a fresh stage, an SST Custom Resource triggers the Lambda automatically. The loader checks the SSM key /{repo}/{stage}/seed-version; if the current seed version is already recorded, it exits immediately. If not, it loads the seed profile for that stage, then writes the version key.

This means standing up a new dev or UAT environment requires no manual data-loading step. The correct seed profile is loaded automatically on first deploy.

2. Per-stage seed profiles.

Two seed profiles are defined:

  • dev — 10 customers (5 NZ, 3 AU passing eIDV, 1 guaranteed eIDV fail, 1 guaranteed sanctions hit), ~20 accounts (everyday and savings), realistic NZD/AUD balances, chart of accounts and GL configuration, service account credentials for integration test runners.

  • uat — 100 customers across NZ and AU with realistic names and demographics, ~250 accounts across all product types (everyday, savings, notice, term deposit), 90 days of pre-loaded transaction history, accounts in edge states (1 dormant, 1 pending KYC review, 1 sanctions-restricted), full chart of accounts and GL configuration, interest rate schedule.

Both profiles use deterministic test identities that trigger known outcomes in the provider stubs (ADR-044). The dev profile is a strict subset of uat.

3. Synthetic transaction engine (MOD-159).

A scheduled Lambda generates synthetic banking activity, making both environments look like an operating bank:

  • Dev: EventBridge schedule, twice weekly (Tuesday and Thursday, 09:00 NZST). Generates ~50 transactions per run across the 10 seed customers. Volume is low to avoid test interference.
  • UAT: EventBridge schedule, twice daily (08:30 and 16:30 NZST). Generates ~200 transactions per run across the 100 seed customers — approximately 400 transactions per day.
  • Prod: schedule disabled. MOD-159 is not deployed to prod.

The engine reads the current balance from Neon before generating any debit, respects account state from MOD-007, and submits all transactions as EventBridge payment instruction events — the same path a real payment would take. It does not write directly to the database.

4. Cross-module acceptance suite (MOD-160).

A nightly Lambda invokes a set of end-to-end acceptance scenarios that span multiple system domains. Examples:

  • Onboard a new customer (MOD-009 → MOD-010 → MOD-007) and verify account activates
  • Submit a payment (SD04 → MOD-001 → MOD-003) and verify balance reflects and notification dispatched
  • Trigger a daily accrual (MOD-005) and verify interest posts to the correct ledger accounts

The suite reports results to a CloudWatch dashboard and emits a custom metric E2EAcceptanceSuitePass (1 = pass, 0 = fail). An alarm fires to the on-call channel on failure.

5. UAT Neon database is never torn down by SST.

SST's removal: remove setting (used in non-prod stages) applies only to SST-managed resources. The Neon database is managed outside SST (via MOD-103 and the Neon API) and persists across all deployments. UAT state accumulates naturally. The only way to reset UAT state is to explicitly invoke the seed loader with --reset, which truncates and reloads.

Alternatives considered

Option A — Manual seed data scripts run by the operator. Rejected. Manual steps create inconsistency — different operators run different versions, fresh environments are sometimes left unseeded, and automation gaps compound as the number of modules grows. Auto-trigger on first deploy is more reliable.

Option B — Factory fixtures inside each module's integration tests. Rejected for cross-cutting test data (customers, accounts, balances). Per-module tests already create their own fixture data for module-specific entities. But customer and account records are shared across modules — each module cannot own them independently without creating conflicts. Shared seed data loaded by a dedicated module is the correct boundary.

Option C — Shared UAT and dev databases. Rejected. Dev is unstable by design — frequent schema migrations and redeployments would corrupt UAT state. Separate Neon branches per stage (provisioned by MOD-103) provide complete isolation.

Consequences

  • MOD-158 must be deployed after MOD-103 (Neon bootstrap) but before any module whose integration tests depend on pre-existing data.
  • MOD-159 must not be deployed until MOD-001, MOD-003, MOD-007, and MOD-020 are all deployed to the target stage. The module YAML dependency list enforces this via the CI dependency graph.
  • The seed version key (/{repo}/{stage}/seed-version) is the authoritative record of which seed profile is loaded. Never manually insert or modify records in the seed tables without bumping this key.
  • Resetting UAT seed data requires explicit operator action (pnpm run seed:reset --stage uat) and will delete all synthetic transaction history accumulated since the last seed load.

All ADRs Compiled 2026-05-22 from source/entities/adrs/ADR-045.yaml