Skip to content

Technical design — MOD-045 Secrets & key management

Module: MOD-045 — Secrets & key management System: SD07 — Data Platform & Governance Infrastructure Repo: bank-platform Module type: Hybrid — IaC + 2 Lambdas (rotation, anomaly detector / access-log writer) FR scope: FR-277, FR-278, FR-279, FR-280 Policies satisfied: DT-001 (AUTO), DT-002 (AUTO), AML-007 (AUTO) Author: AI agent (Claude Opus 4.7) Date: 2026-04-19 Stage covered: dev (account 647751526084, region ap-southeast-2) SST permalink: https://sst.dev/u/e25776b6


Objective

MOD-045 owns the secrets- and key-management substrate for the whole bank platform. It layers on top of MOD-104 (which already provisioned the three classification CMKs and the bank-secrets-read policy). The module:

  • Reserves the bank-* Secrets Manager namespace and publishes an IAM policy that scopes every mutation to that namespace (FR-277 / DT-001).
  • Ships a generic 4-step rotation Lambda and attaches a 90-day automated rotation schedule to every MOD-045-managed secret (FR-278 / DT-002).
  • Ships an anomaly-detector Lambda fed by EventBridge-routed CloudTrail Secrets Manager events; it publishes to the MOD-104 bank-platform-alerts-{env} SNS topic on anomalous access (FR-279).
  • Writes an FR-280 access record to a dedicated 7-year-retention CloudWatch log group on every access event, containing identity, secret ARN, event name, and timestamp — never a secret value.

Every resource is tagged tenant_id=totara-bank, module_id=MOD-045, environment=<env> via Pulumi defaultTags. All downstream consumers integrate via SSM outputs.


Execution model

Aspect Decision
IaC tool SST v3 Ion + raw @pulumi/aws (ADR-025)
Lambda runtime Node 20, arm64 default, CJS bundle via esbuild
Rotation model AWS Secrets Manager 4-step contract (createSecret, setSecret, testSecret, finishSecret) — one Lambda handles all bank-* secrets
Anomaly detection EventBridge source=aws.secretsmanager / detail-type=AWS API Call via CloudTrail → Lambda with simple allow-list heuristic
Access log Dedicated CloudWatch log group /aws/secretsmanager/bank-access-{env} with 3653-day retention (closest supported ≈10y; 7y exact not offered)
Deploy AWS_PROFILE=bank-dev pnpm sst deploy --stage dev

Stack layout

secrets-key-management/
├── sst.config.ts                          — SST app + run() orchestration
├── src/
│   ├── stacks/
│   │   ├── secret-hierarchy.ts            — namespace IAM policy; sanctions-list secret envelope
│   │   ├── rotation-lambdas.ts            — 4-step rotation Lambda + demo secret + schedules
│   │   ├── anomaly-detector.ts            — detector Lambda + EventBridge rule + alerts wiring
│   │   └── access-log.ts                  — 7y-retention CloudWatch log group
│   ├── lambdas/
│   │   ├── rotation/index.ts              — AWS 4-step rotation handler (per-secret-type dispatch)
│   │   └── anomaly-detector/index.ts      — FR-279/280 handler (allow-list heuristic + access log)
│   └── outputs.ts                         — 4 SSM parameters under /bank/{env}/secrets/*
├── scripts/build-lambdas.mjs              — esbuild → dist/{rotation,anomaly-detector}/index.js
├── __tests__/integration/                 — 7 test files, Jest + AWS SDK v3
└── docs/                                  — (global) docs/design/MOD-045.md + handoff

AWS resources provisioned (dev stage)

IAM — reserved-namespace policy (FR-277)

Resource Value
Managed policy arn:aws:iam::647751526084:policy/bank-secrets-namespace-dev
Allow PutSecretValue, UpdateSecret, UpdateSecretVersionStage, RotateSecret, DescribeSecret, GetSecretValue on arn:aws:secretsmanager:*:647751526084:secret:bank-*
Deny (NotResource) CreateSecret, PutSecretValue, UpdateSecret, RotateSecret on any secret outside bank-*
Deny ssm:PutParameter (SecureString) on any /bank-secrets/* path — shadow-store guard

The policy is designed for attachment to the MOD-045 rotation role (already attached). Any future service that legitimately needs to mutate bank-* secrets receives a targeted attachment.

Rotation Lambda + schedules (FR-278 / DT-002)

Resource Value
Lambda bank-secrets-rotation-dev (Node 20, 256 MB, 60 s)
IAM role bank-secrets-rotation-devsecretsmanager:{Describe,Get,Put,UpdateSecretVersionStage} on bank-* + kms:Decrypt via secretsmanager.*.amazonaws.com
Rotation schedule (probe) bank-platform/rotation/probe-dev — 90 days, 4-step, automated
Rotation schedule (sanctions) bank-aml/sanctions/decryption-key — 90 days, 4-step, automated

The rotation Lambda dispatches per secret secret_type tag:

  • API_KEY / ENCRYPTION_KEY — opaque random values via GetRandomPassword
  • DATABASE_CREDENTIALTODO (MOD-103 Neon API integration)
  • OAUTH_SECRETTODO (IDP coordination)
  • JWT_KEY / WEBHOOK_SECRET — treated as API_KEY for now

Each of the 4 steps (createSecret, setSecret, testSecret, finishSecret) is idempotent per the AWS contract. createSecret short-circuits if the pending version already exists; finishSecret uses UpdateSecretVersionStage with MoveToVersionId/RemoveFromVersionId to atomically promote/demote.

Anomaly detector Lambda (FR-279) + access log writer (FR-280)

Resource Value
Lambda bank-secrets-anomaly-dev (Node 20, 256 MB, 30 s)
IAM role bank-secrets-anomaly-dev — logs RW on access-log group, SNS Publish on MOD-104 alerts topic, IAM ListRoles/GetRole (future ML-based detection)
EventBridge rule bank-secrets-anomaly-devsource=aws.secretsmanager + detail-type=AWS API Call via CloudTrail
Access log group /aws/secretsmanager/bank-access-dev — retention 3653 days

Heuristic (dev): substring-match detail.userIdentity.arn against ALLOWED_ROLE_PATTERNS env (comma-separated). Production upgrade path is documented as a TODO: plug learned access patterns via MOD-076 once it ships.

Sanctions-list decryption key envelope (AML-007)

Resource Value
Secret bank-aml/sanctions/decryption-key
KMS alias/bank/financial
Rotation 90-day automated via MOD-045 Lambda
Tags sensitivity=aml, secret_type=ENCRYPTION_KEY, module_id=MOD-045 (default-tag)

The envelope is provisioned here because MOD-045 owns sanctions-list key provisioning. Real sanctions-list material is populated by MOD-051 at runtime.


SSM outputs table (consumer contract)

All under arn:aws:ssm:ap-southeast-2:647751526084:parameter.

SSM path Value Consumed by
/bank/{env}/secrets/namespace-policy/arn MOD-045 namespace IAM policy ARN Any service role that is granted mutation rights on bank-* secrets (attach alongside bank-secrets-read)
/bank/{env}/secrets/rotation/fn-arn Rotation Lambda ARN Downstream modules wiring aws.secretsmanager.SecretRotation on their own bank-* secrets
/bank/{env}/secrets/anomaly/fn-arn Anomaly detector Lambda ARN Diagnostics / runbook references
/bank/{env}/secrets/access-log/group-arn CloudWatch log group ARN MOD-076 log consumer (future), regulatory export Glue jobs

Resolution example

// Downstream module (Pulumi) — attach rotation to a bank-* secret
const rotationFn = aws.ssm.getParameterOutput({
  name: `/bank/${stage}/secrets/rotation/fn-arn`,
}).value;

new aws.secretsmanager.SecretRotation("my-secret-rotation", {
  secretId: mySecret.id,
  rotationLambdaArn: rotationFn,
  rotationRules: { automaticallyAfterDays: 90 },
});

Acceptance criteria status (dev stage)

Run: AWS_PROFILE=bank-dev STAGE=dev pnpm test from secrets-key-management/.

FR / Policy Mode Tests Pass Fail Status
FR-277 — secrets only in Secrets Manager; namespace scoping 5 5 0 PASS
FR-278 — 90-day automatic rotation, zero-downtime 3 3 0 PASS
FR-279 — <5 min alert on anomalous access 2 2 0 PASS
FR-280 — immutable 7y access log (identity/ref/timestamp, no value) 3 3 0 PASS
DT-001 — secrets uncopyable by devs, no values in logs AUTO 3 3 0 PASS
DT-002 — rotation automated, no manual schedule AUTO 2 2 0 PASS
AML-007 — sanctions-list key centrally managed, no offline copy AUTO 4 4 0 PASS
Total 22 22 0 100%

All IaC-module quality gates met: one infrastructure integration test per FR, one policy test per policies_satisfied row, AUTO policies verified end-to-end against deployed AWS.


Test approach

Jest + AWS SDK v3, no mocks. Each test queries live AWS in the dev stage.

File Coverage
fr-277-secrets-manager-only.test.ts Namespace policy is published, scoped to bank-*, denies mutations outside; bank-secrets-read grants read-only; shadow-store guard
fr-278-rotation.test.ts Rotation Lambda published, 90-day schedule on probe, force-rotation flips AWSCURRENT with AWSPREVIOUS still retrievable
fr-279-anomaly-alert.test.ts Temporary SQS subscribed to alerts topic; synthetic unallowed principal event → SNS publish observed within 60 s
fr-280-access-log.test.ts 3653-day retention on dedicated log group; access record has identity/ref/timestamp with no SecretString / secret_value / password strings
pol-dt-001.test.ts IAM policy simulator for default role; read policy grants no mutation actions; log sweep finds no secret-value leakage
pol-dt-002.test.ts Every MOD-045-managed secret (tagged rotation_enabled=true or module_id=MOD-045) has rotation enabled + Lambda bound + 90-day schedule
pol-aml-007.test.ts Sanctions-list key exists, encrypted with alias/bank/financial, rotation enabled, tagged sensitivity=aml + secret_type=ENCRYPTION_KEY

Run: AWS_PROFILE=bank-dev STAGE=dev pnpm test from secrets-key-management/.


Operational notes

  • Deploy: AWS_PROFILE=bank-dev pnpm deploy --stage dev (runs pnpm build:lambda then sst deploy)
  • Remove: AWS_PROFILE=bank-dev pnpm sst remove --stage dev — AWS Secrets Manager deletion is a 30-day pending-delete by default; force-delete with --force-delete-without-recovery on the individual secrets if a full teardown is needed.
  • Lambda code: src/lambdas/{rotation,anomaly-detector}/index.ts compiled via esbuild to dist/{name}/index.js. The deploy script invokes the build automatically; in CI / in editor, run pnpm build:lambda before sst diff/sst deploy.
  • Adding a new secret with rotation: create the aws.secretsmanager.Secret in your module, tag rotation_enabled=true, then new aws.secretsmanager.SecretRotation({ secretId, rotationLambdaArn: <MOD-045 rotation fn ARN from SSM>, rotationRules: { automaticallyAfterDays: 90 } }). The MOD-045 POL-DT-002 test will automatically pick it up on next run.
  • DescribeLogStreams IAM gotcha: the action evaluates against the log-group:NAME:log-stream:* resource pattern, not the bare log-group ARN. The inline policy lists all three forms (:*, :log-stream:*, bare ARN) to be safe across SDK versions.

Deviations / TODOs

  1. CloudWatch log retention = 3653 days (≈10y) — exact 7y is not a valid CloudWatch retention value; 3653 is the smallest supported option satisfying the FR-280 "≥7 years" contract. If the wiki later clarifies that exactly 7y is required via S3 archive, this module would add a Kinesis Firehose delivery to S3 with Object Lock GOVERNANCE (7y retention) alongside or instead of CloudWatch.
  2. Per-secret-type rotation handlers. DATABASE_CREDENTIAL and OAUTH_SECRET are not implemented — noted as TODO in src/lambdas/rotation/index.ts. Needed when MOD-103 (Neon) and MOD-044 (JWT/OAuth) land and start requesting managed rotation.
  3. Anomaly heuristic is substring-matched. Production-grade ML-based detection is explicitly out of scope for MOD-045 and should be layered as a consumer of /aws/secretsmanager/bank-access-{env} once MOD-076 deploys.
  4. Namespace policy shadow-store guard scoped to /bank-secrets/*. If a wider shadow-store prefix is identified (e.g. /bank/* for the SSM outputs themselves), the guard can be tightened — but the current scope avoids false-positives against the legitimate /bank/{env}/... SSM output tree used by MOD-104 and siblings.

  • Wiki spec: bank-wiki/source/entities/modules/MOD-045.{yaml,md}
  • FR register: bank-wiki/source/pages/goals/fr-register.md — FR-277..280
  • Data model: bank-wiki/source/pages/design/system/data-models/SD07-data-platform.mdplatform.secrets_refs
  • Handoff: docs/handoffs/MOD-045-complete.handoff.md
  • MOD-104 design (upstream dependency): docs/design/MOD-104.md
  • Methodology (IaC-module quality gates): https://bank-wiki.pages.dev/delivery/methodology/
  • ADRs in effect: ADR-023, ADR-025, ADR-030, ADR-031