Skip to content

Test strategy

This is the authoritative test strategy for all module development across the Totara Bank platform. Every module implementation — whether produced by an AI coding agent or a human engineer — must comply with this strategy. The execution plan for each module specifies which test types apply, what coverage targets must be met, and which CI gates must pass before a PR may merge.

Principles

  1. Tests as acceptance criteria. Every FR has a corresponding automated test. A module is not done until its FRs pass as tests. Tests are not written after — they are the acceptance criteria.
  2. Test at the right level. Unit tests prove logic. Contract tests prove interfaces. Integration tests prove interactions. Do not test integration concerns in unit tests or logic concerns in integration tests.
  3. Compliance is not optional. Regulatory obligations (GATE and AUTO modes) must have dedicated compliance tests. A module cannot be deployed if its GATE policies are not covered.
  4. NFR thresholds are tested, not assumed. Latency, availability, and throughput targets from the NFR register must be verified in staging before promotion to production.
  5. AI-generated tests require human review. AI-generated test code must be reviewed by the tech lead before merge, especially for compliance and security test suites.

Test types

Unit tests

What they test: Business logic, data transformations, validation rules, error handling within a single Lambda handler or service class. No network calls, no database, no real dependencies.

Scope: 100% of FR acceptance criteria must be covered by at least one unit test. One test per acceptance criterion clause is the minimum.

Tools: Jest (TypeScript), pytest (Python), with in-process mocking for all external dependencies.

Naming convention: src/handlers/[name].test.ts or src/services/[name].test.ts

CI gate: Unit tests run on every PR push. Failure blocks merge.

Coverage target: - 100% of FR acceptance criteria (tracked against FR ID, not line coverage) - Line coverage ≥ 80% (enforced by CI; reported but not blocking unless below 60%)

Contract tests

What they test: The exact shape of every interface the module participates in — Lambda invocation request/response schemas, EventBridge event schemas, and REST API request/response schemas.

Scope: All event schemas this module emits or consumes. All Lambda invocation contracts this module calls or exposes. All API endpoints.

Why they exist: Modules cannot break each other's interfaces silently. A contract test fails fast when a schema change would break a downstream consumer. This is especially important in agentic development where multiple modules are being built in parallel.

Tools: Schema validation libraries (Zod, AJV) against the canonical schemas in the system-level Interface contracts and Event catalogue.

Naming convention: src/__tests__/contracts/[name].test.ts

CI gate: Contract tests run on every PR push. Failure blocks merge.

Requirements: - Every event emitted must validate against its registered schema in the event catalogue - Every Lambda invocation input/output must validate against the contracts.md entry for that pair - Adding a required field to an outbound schema is a breaking change — requires a new contract version or a coordinated deployment

Integration tests

What they test: The module's behaviour when connected to real downstream services — a live (staging) database, real Lambda invocations of dependent modules, real EventBridge buses.

Scope: Happy path for each FR, plus critical failure paths (dependency unavailable, invalid input, timeout, partial failure).

Tools: Jest or pytest with real AWS credentials pointing to the staging environment. Fixtures from fixtures.md pre-seeded by CI.

Naming convention: src/__tests__/integration/[name].test.ts

CI gate: Integration tests run against the staging environment after unit and contract tests pass. They gate promotion from staging to production.

Requirements: - Happy path must pass for every FR in scope - At least one failure path test per external dependency - Tests must clean up their own data (use fixture-managed test customer records)

End-to-end tests (E2E)

What they test: Complete user journeys that cross multiple modules and system domains — e.g. account opening (KYC → Core Banking → App), domestic payment (App → Payments → Core Banking → Notifications).

Scope: Defined per product and per capability, not per module. E2E tests are owned by the platform team and maintained in the bank-e2e repository.

CI gate: E2E tests run nightly against staging. Critical journeys (account opening, payment, login) also run on every deployment to production (smoke test subset).

Requirements: - Each launched product capability must have at least one E2E test covering the full happy path - Failure in E2E smoke tests blocks production deployment

NFR tests

What they test: Non-functional thresholds from the NFR register — primarily latency, throughput, and availability.

Scope: All modules with real-time customer-facing handlers (any module that handles a synchronous request from the App or from another module within the p99 path).

Tools: k6 or Artillery for load testing; custom Lambda invocation scripts for p99 measurement.

Naming convention: src/__tests__/nfr/[name]-load.test.ts

CI gate: NFR tests run in staging as a pre-production gate. They do not run on every PR (too slow), but must pass before a module is promoted to production for the first time, and after any change that touches the hot path.

Thresholds (from NFR register — always check current values): - Real-time Lambda handlers: p99 ≤ 200ms (excluding cold starts) - Batch processors: not subject to p99 gate; throughput target applies - Database reads: p99 ≤ 50ms at expected peak QPS

Security tests

What they test: Input validation, authentication and authorisation enforcement, secrets handling, injection vulnerabilities, OWASP Top 10 as applicable to serverless.

Scope: All modules with external-facing API endpoints, all modules that handle PII, all modules that emit events containing customer data.

Tools: - Static: ESLint security plugins, Semgrep rules run as part of CI lint gate - Dynamic: OWASP ZAP or Burp Suite run against the staging API gateway in scheduled security scans

CI gate: Static security lint runs on every PR. Failure blocks merge. Dynamic scans run weekly in staging; findings above "Medium" severity must be resolved before next production deployment.

Requirements: - No secrets hardcoded or logged (enforced by Semgrep rule no-hardcoded-secrets) - PII fields must not appear in CloudWatch log output (enforced by log sanitisation middleware) - All API endpoints must enforce authentication — no unauthenticated paths except documented public endpoints - IAM permissions for Lambda execution roles must follow least privilege — reviewed at PR time

Regulatory compliance tests

What they test: That GATE and AUTO policy satisfaction modes work correctly. These tests prove that the module cannot be bypassed in ways that would violate a regulatory obligation.

Scope: Every module that has a GATE or AUTO policy satisfaction mode in its YAML.

Structure: For each GATE policy: - Test that the protected flow is blocked when the policy condition is not met - Test that the protected flow is permitted when the condition is met - Test that there is no bypass path (no alternative code path that skips the gate)

Naming convention: src/__tests__/compliance/[policy-code]-[mode].test.ts

CI gate: Compliance tests run on every PR and are a hard gate — a PR that touches a GATE module cannot merge if any compliance test fails, regardless of other test results.

Example (AML-003, GATE mode in KYC module): 

describe('AML-003 GATE — account activation', () => {
  it('blocks activation when KYC status is not VERIFIED', async () => { ... })
  it('permits activation when KYC status is VERIFIED', async () => { ... })
  it('rejects bypass attempt via direct DB write path', async () => { ... })
})

Test data and fixtures

Each module's fixtures.md defines the seed data required for test environments. CI pre-seeds fixtures before running integration and compliance tests.

Rules for test data: - Test customer records must use clearly fictional data: names like "Test Customer One", IRD numbers like 000-000-000, account numbers with UUID prefix 00000000-0000-0000-... - Do not use real names, real IRD/TFN numbers, or real account numbers — even obfuscated - Fixtures are idempotent — running them twice produces the same result (use INSERT ... ON CONFLICT DO NOTHING) - Fixtures are environment-specific — never run fixture scripts against production

CI pipeline gates summary

Stage Tests Gate
PR open / push Lint (ESLint, Semgrep), type check, unit tests, contract tests Hard gate — all must pass
PR merge to main Above + integration tests (staging) Hard gate
Staging deploy Integration tests, compliance tests Hard gate
Pre-production NFR load tests (first deploy or hot-path change) Hard gate
Production deploy E2E smoke tests Hard gate
Nightly Full E2E suite, dynamic security scan Advisory — creates ticket on failure

AI-assisted test generation

When an AI coding agent generates test code, the following rules apply:

  1. Test names must reference FR IDs. The test name or describe block must include the FR ID it covers (FR-xxx), so the CI report can map test results to requirements.

  2. Compliance tests require human review. The tech lead must sign off on any test that covers a GATE or AUTO policy satisfaction. AI-generated compliance tests are marked with // [AI-generated: requires review] until reviewed.

  3. Contract tests are generated from the canonical schemas. Do not write contract tests by hand — generate them from the interface contracts and event catalogue documents. This ensures tests break when the canonical schema changes, not when the implementation changes.

  4. Fixtures must be reviewed for PII. AI agents sometimes generate realistic-looking test data. All AI-generated fixture files must be reviewed to confirm no real-looking PII is present.

  5. Coverage reports are not sufficient. Line coverage is a floor, not a ceiling. The tech lead reviews test quality (meaningful assertions, correct boundary conditions) not just line coverage percentages.

Artefact checklist per module PR

The following must be present before a module PR is approved:

  • [ ] execution-plan.md — updated with actual FR coverage achieved
  • [ ] drift-report.md — written, with any deviations from the execution plan documented
  • [ ] Unit tests — one per FR acceptance criterion clause
  • [ ] Contract tests — for all events emitted/consumed and all Lambda invocations
  • [ ] Integration tests — happy path per FR + key failure paths
  • [ ] Compliance tests — for every GATE or AUTO policy in the module YAML
  • [ ] Fixtures — idempotent, fictional data only
  • [ ] config.md — all env vars and secrets documented (no values)
  • [ ] Security lint — clean (no blocking Semgrep findings)
  • [ ] NFR test results — if module is on the real-time p99 path

Test strategy by module category

Module category Unit Contract Integration Compliance NFR load
Real-time Lambda (customer-facing) ✓ ✓ ✓ If GATE/AUTO ✓
Real-time Lambda (internal-only) ✓ ✓ ✓ If GATE/AUTO Optional
Batch processor ✓ ✓ ✓ If GATE/AUTO Throughput test
EventBridge consumer ✓ ✓ (schema) ✓ If GATE/AUTO —
Snowflake Dynamic Table — ✓ (schema) ✓ — —
CDC pipeline — ✓ ✓ — Throughput test
Reporting / read-only ✓ ✓ ✓ — Optional