Post-deployment checklist¶

Resolves: GAP-D08 — No post-deployment verification checklist.

This checklist is the operational confirmation that a deployment is working correctly before declaring "deployment complete." Work through it after every deployment — initial provisioning or module update. It is not a test suite; it is the human or AI agent confirmation that the platform is healthy and the compliance gates are enforced.

All commands assume the correct AWS profile is active and environment variables are set for the target environment (ENV=dev|uat|prod).

Phase 1 — Infrastructure health¶

Run these checks before anything else. If any fail, do not proceed to Phase 2.

1. EventBridge buses: test event delivery

aws events put-events --entries '[{
  "Source": "deployment.healthcheck",
  "DetailType": "HealthCheck",
  "Detail": "{\"check\":\"post-deploy\"}",
  "EventBusName": "bank-core"
}]'

Repeat for buses: bank-core, bank-kyc, bank-aml, bank-payments, bank-credit, bank-platform.

Then confirm each DLQ is empty:

aws sqs get-queue-attributes \
  --queue-url https://sqs.{region}.amazonaws.com/{account}/bank-{domain}-dlq \
  --attribute-names ApproximateNumberOfMessages

Pass: All put-events calls return FailedEntryCount: 0. All DLQ depths are 0.

2. S3 buckets: accessible and versioning enabled

aws s3api get-bucket-versioning --bucket bank-{env}-{bucket-name}

Check all buckets in the Terraform state for the environment.

Pass: Each bucket returns {"Status": "Enabled"}. Any bucket returning Suspended or no status is a fail.

3. Secrets Manager: all secrets in manifest exist and are accessible

aws secretsmanager list-secrets --filter Key=tag-key,Values=env Key=tag-value,Values={ENV} \
  | jq '[.SecretList[].Name] | sort'

Compare the output against the secrets manifest for this environment.

Then confirm the Lambda execution role can read a representative secret:

aws secretsmanager get-secret-value --secret-id bank/{ENV}/core/db-connection-string

Pass: All secrets in the manifest are present. The Lambda execution role access test returns the secret value without an AccessDeniedException.

4. Cognito: both user pools active, test auth flow

aws cognito-idp describe-user-pool --user-pool-id {CUSTOMER_POOL_ID} | jq '.UserPool.Status'
aws cognito-idp describe-user-pool --user-pool-id {STAFF_POOL_ID}    | jq '.UserPool.Status'

For each pool, initiate an auth flow with a test user provisioned for deployment verification:

aws cognito-idp initiate-auth \
  --auth-flow USER_PASSWORD_AUTH \
  --auth-parameters USERNAME=deploy-verify@test.internal,PASSWORD={TEST_PASSWORD} \
  --client-id {APP_CLIENT_ID}

Pass: Both pools return "Active". Auth flow returns an AuthenticationResult with AccessToken, IdToken, and RefreshToken. JWT claims include custom:jurisdiction.

5. KMS keys: all CMKs enabled and accessible

for KEY_ALIAS in alias/bank-{ENV}-pii alias/bank-{ENV}-financial alias/bank-{ENV}-operational; do
  aws kms describe-key --key-id "$KEY_ALIAS" | jq '.KeyMetadata | {KeyId, KeyState, Enabled}'
done

Pass: All three CMKs show "KeyState": "Enabled" and "Enabled": true. Any key in PendingDeletion or Disabled state is a P1 blocker.

Phase 2 — Database health¶

6. All 6 Neon databases reachable via PgBouncer

for DB in core kyc aml payments credit platform; do
  psql "$PGBOUNCER_BASE_URL/$DB" -c "SELECT 1 AS alive;" 2>&1
done

Connection strings are sourced from Secrets Manager (bank/{ENV}/{domain}/pgbouncer-connection-string).

Pass: All six databases return alive: 1. Any connection refused or authentication failure is a fail.

7. Flyway migration history: expected count for current schema version

psql "$PGBOUNCER_BASE_URL/core" \
  -c "SELECT COUNT(*) FROM flyway_schema_history WHERE success = true;"

Compare the count against the expected value in the release notes for the current version. Repeat for all 6 databases.

Pass: Each database migration count matches the expected value. Any shortfall indicates a migration did not apply.

8. No pending migrations

Run from each code repository:

flyway -url="$FLYWAY_URL" info | grep -E "Pending|Failed"

Pass: No rows returned — all migrations are in Applied state. Any Pending or Failed migration is a blocker that must be resolved before proceeding.

Phase 3 — Core banking¶

9. Double-entry posting via MOD-001

POST a test transaction using an idempotency key:

curl -s -X POST https://api-internal.{ENV}.bank/v1/ledger/postings \
  -H "Idempotency-Key: deploy-verify-$(date +%s)" \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  -d '{
    "entries": [
      {"account_id": "TEST-ASSET-001", "side": "DR", "amount": 100, "currency": "NZD"},
      {"account_id": "TEST-LIABILITY-001", "side": "CR", "amount": 100, "currency": "NZD"}
    ],
    "description": "Deployment verification posting"
  }'

Pass: Response is HTTP 201 with status: committed and the same idempotency_key echoed back. Re-submit the same request (same idempotency key) and confirm HTTP 200 with the original posting returned (idempotent replay).

10. Real-time balance query via MOD-003

curl -s https://api-internal.{ENV}.bank/v1/accounts/TEST-ASSET-001/balance \
  -H "Authorization: Bearer {SERVICE_TOKEN}"

Pass: Response reflects the posting from checkpoint 9 — the balance includes the test debit. Response time is under 200ms (check x-response-time header or CloudWatch).

11. Posting event on EventBridge bus within 500ms

After the posting in checkpoint 9, query CloudWatch Logs for the EventBridge target that receives bank-core events:

aws logs filter-log-events \
  --log-group-name /aws/events/bank-core-audit \
  --start-time $(date -d '5 minutes ago' +%s000) \
  --filter-pattern "deploy-verify"

Pass: The test posting event appears in the log within 500ms of the posting timestamp. If the event is missing after 1 minute, this indicates the EventBridge rule or target is misconfigured.

Phase 4 — KYC and identity¶

12. Test customer record and KYC session initialisation

curl -s -X POST https://api-internal.{ENV}.bank/v1/kyc/customers \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  -d '{
    "given_name": "Deploy",
    "family_name": "Verification",
    "date_of_birth": "1990-01-01",
    "email": "deploy-verify@test.internal"
  }'

Pass: HTTP 201 with a customer_id and kyc_session_id in the response body. The KYC session status is initialised.

13. eIDV provider API key health check

curl -s https://api-internal.{ENV}.bank/v1/kyc/eidv/health \
  -H "Authorization: Bearer {SERVICE_TOKEN}"

This endpoint proxies a health check call to the configured eIDV provider using the API key from Secrets Manager.

Pass: HTTP 200 with {"provider_status": "ok"}. An api_key_invalid or provider_unreachable response means the Secrets Manager value for the eIDV API key needs updating.

14. JWT claims include jurisdiction

Using the token obtained in checkpoint 4:

echo {JWT_ID_TOKEN} | cut -d. -f2 | base64 -d 2>/dev/null | jq '{sub, custom_jurisdiction: .["custom:jurisdiction"]}'

Pass: The decoded JWT payload includes custom:jurisdiction set to the correct value for this deployment (NZ, AU, or NZ+AU).

Phase 5 — Payments¶

15. Test domestic payment in pending state

curl -s -X POST https://api-internal.{ENV}.bank/v1/payments/domestic \
  -H "Idempotency-Key: deploy-pmt-$(date +%s)" \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  -d '{
    "from_account": "TEST-ASSET-001",
    "to_bsb": "123456",
    "to_account": "00000001",
    "amount": 1.00,
    "currency": "NZD",
    "reference": "Deployment verification"
  }'

Pass: HTTP 201 with status: pending and a payment_id. The payment must not proceed to submitted in a test environment — confirm the test environment payment gateway is in sandbox mode.

16. Payment event on EventBridge bus

aws logs filter-log-events \
  --log-group-name /aws/events/bank-payments-audit \
  --start-time $(date -d '5 minutes ago' +%s000) \
  --filter-pattern "deploy-pmt"

Pass: Payment created event appears on the bank-payments EventBridge bus.

17. BPAY health check (AU deployments only)

Skip if deployment.jurisdiction does not include AU.

curl -s https://api-internal.{ENV}.bank/v1/payments/bpay/health \
  -H "Authorization: Bearer {SERVICE_TOKEN}"

Pass: HTTP 200 with {"bpay_gateway": "ok"}.

18. NPP service connectivity (AU deployments only)

Skip if deployment.jurisdiction does not include AU.

curl -s https://api-internal.{ENV}.bank/v1/payments/npp/connectivity \
  -H "Authorization: Bearer {SERVICE_TOKEN}"

Pass: HTTP 200 confirming NPP participant connectivity. Any gateway_timeout indicates NPP connectivity credentials need verification in Secrets Manager.

Phase 6 — AML monitoring¶

19. AML engine receiving posting events

Confirm the AML rule engine (MOD-022) has processed the posting from checkpoint 9:

aws logs filter-log-events \
  --log-group-name /aws/lambda/bank-aml-rule-engine \
  --start-time $(date -d '5 minutes ago' +%s000) \
  --filter-pattern "deploy-verify"

Pass: Log entry shows the test posting was received and evaluated by the AML rule engine. If absent after 2 minutes, check the EventBridge subscription from bank-core to bank-aml.

20. Sanctions list last-refresh timestamp

curl -s https://api-internal.{ENV}.bank/v1/aml/sanctions/status \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  | jq '{last_refreshed, list_count, provider}'

Pass: last_refreshed is within the last 24 hours. If the list has never been loaded (new deployment), trigger a manual refresh:

curl -s -X POST https://api-internal.{ENV}.bank/v1/aml/sanctions/refresh \
  -H "Authorization: Bearer {SERVICE_TOKEN}"

Confirm the refresh completes and last_refreshed updates before proceeding.

21. STR submission endpoint reachable

curl -s https://api-internal.{ENV}.bank/v1/aml/str/health \
  -H "Authorization: Bearer {SERVICE_TOKEN}"

Pass: HTTP 200 with {"str_gateway": "ok", "regulator": "AUSTRAC|FIU"} as appropriate for the jurisdiction.

Phase 7 — Regulatory reporting pipelines¶

22. CDC pipeline running (MOD-042)

aws cloudwatch get-metric-statistics \
  --namespace AWS/KinesisFirehose \
  --metric-name IncomingRecords \
  --dimensions Name=DeliveryStreamName,Value=bank-cdc-{ENV} \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 3600 \
  --statistics Sum

Pass: Sum is greater than 0 for a running system. On a new deployment with no data flowing yet, accept 0 with a note in the sign-off record that data flow begins with first customer activity.

23. Snowflake receiving CDC records

Connect to Snowflake and check a recently-loaded table:

SELECT MAX(loaded_at) AS last_load, COUNT(*) AS records_today
FROM bank_core.public.postings_cdc
WHERE DATE(loaded_at) = CURRENT_DATE();

Pass: last_load is within the last hour for active environments. On initial deployment, this may be empty — acceptable if checkpoint 22 confirms the pipeline is running.

24. Regulatory report last-run timestamp

curl -s https://api-internal.{ENV}.bank/v1/reporting/schedule \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  | jq '.reports[] | {name, last_run, next_due, status}'

Pass: No report shows status: overdue. All reports show a next_due timestamp in the future. On initial deployment, last_run may be null — acceptable if the schedule is correctly configured.

Phase 8 — Observability¶

25. CloudWatch dashboard loads with metrics

Navigate to the CloudWatch console and open the bank-{ENV}-platform dashboard.

Pass: All widgets load without Insufficient data errors. At least one metric datapoint exists from within the last hour (or from the test postings in earlier checkpoints).

26. X-Ray traces visible for each system domain

aws xray get-trace-summaries \
  --time-range-type EventTime \
  --start-time $(date -u -d '30 minutes ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  | jq '.TraceSummaries | length'

Pass: At least one trace per system domain that received traffic during the verification run. Confirm traces are visible in the X-Ray service map console.

27. Test alert fires and routes correctly

Manually trigger the CloudWatch alarm used for deployment verification:

aws cloudwatch set-alarm-state \
  --alarm-name bank-{ENV}-deploy-verify-alarm \
  --state-value ALARM \
  --state-reason "Post-deployment routing test"

Pass: SNS notification arrives at the on-call channel (Slack or PagerDuty) within 2 minutes. Reset the alarm afterwards:

aws cloudwatch set-alarm-state \
  --alarm-name bank-{ENV}-deploy-verify-alarm \
  --state-value OK \
  --state-reason "Post-deployment routing test complete"

Phase 9 — Go-live gate (production only)¶

Run this phase for production deployments only. All Phase 1–8 checks must be recorded as passed before proceeding.

28. All Phase 1–8 checks passed and recorded

Review the sign-off table below. Every checkpoint must have a Pass entry before proceeding.

29. Balance reconciliation — zero discrepancies

curl -s https://api-internal.prod.bank/v1/reconciliation/current \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  | jq '{run_at, total_accounts, discrepancies}'

Pass: discrepancies: 0. Any non-zero value is a P1 incident — do not proceed and page the on-call engineer immediately.

30. AML monitoring confirmed active and processing events

Confirm that the AML rule engine has processed at least one event since the deployment completed (checkpoint 19 may be used as evidence if timing is appropriate).

Pass: Log evidence of event processing post-deployment. AML alert queue depth is 0 or within expected norms.

31. KYC gate enforced

Attempt to open an account for the test customer from checkpoint 12 without completing KYC verification:

curl -s -X POST https://api-internal.prod.bank/v1/accounts \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  -d '{"customer_id": "{TEST_CUSTOMER_ID}", "product_id": "PRD-001"}'

Pass: HTTP 403 with {"error": "kyc_not_verified"}. Any HTTP 201 or 200 response means the KYC gate is not enforced — this is a P1 blocker.

32. Disclosure gate enforced

Attempt to accept a product offer for the test customer without completing the disclosure flow:

curl -s -X POST https://api-internal.prod.bank/v1/products/accept \
  -H "Authorization: Bearer {SERVICE_TOKEN}" \
  -d '{"customer_id": "{TEST_CUSTOMER_ID}", "product_id": "PRD-001", "bypass_disclosure": true}'

Pass: HTTP 403 with {"error": "disclosure_not_completed"}. Any 2xx response is a P1 blocker.

Sign-off record¶

Complete this table and attach it to the deployment record (Jira release ticket or equivalent).

Checkpoint	Description	Pass / Fail	Verified by	Timestamp
1	EventBridge bus test events and DLQ depth
2	S3 bucket versioning
3	Secrets Manager completeness and access
4	Cognito user pools active, auth flow
5	KMS CMKs enabled
6	Neon databases reachable via PgBouncer
7	Flyway migration count
8	No pending migrations
9	Double-entry posting — atomic commit, idempotency
10	Real-time balance query
11	Posting event on EventBridge within 500ms
12	KYC session initialised
13	eIDV provider API key health
14	JWT includes jurisdiction claim
15	Domestic payment in pending state
16	Payment event on EventBridge
17	BPAY health check (AU only)
18	NPP connectivity (AU only)
19	AML engine receiving posting events
20	Sanctions list last-refresh within 24h
21	STR submission endpoint reachable
22	CDC pipeline running
23	Snowflake receiving CDC records
24	Regulatory report schedule valid
25	CloudWatch dashboard loads with metrics
26	X-Ray traces visible
27	Test alert routes correctly
28	All Phase 1–8 checks recorded (prod only)
29	Balance reconciliation — zero discrepancies (prod only)
30	AML monitoring active (prod only)
31	KYC gate enforced (prod only)
32	Disclosure gate enforced (prod only)

Deployment declared complete by: ___ Date/time: _____