Secret storage and KMS envelope encryption

Keep Preloop tenant secret values in the control database as KMS envelope-encrypted rows, so a database or state-directory copy yields no value.

Preloop’s hosted control plane stores tenant workflow-secret values in the control database as KMS envelope-encrypted rows. A database dump, a state-directory copy, or a backup contains only ciphertext, a wrapped data key, a nonce, and the CMK’s key id; it cannot produce a value without kms:Decrypt in the owning organization’s assumed session. control.secrets keeps only metadata and a workflow-secret/... reference.

The value store is selected by PRELOOP_SECRETS_STORE and defaults by control store:

Control store	Default	Why
PostgreSQL (hosted, shared)	envelope	A copy of the database must not yield a value.
SQLite (local single node)	file	Sealed blobs under the engine’s private state directory; no AWS dependency for local development.

envelope is compiled only with the aws-kms feature:

cargo build -p preloop-cli --features local-engine,aws-kms

A hosted engine selects it explicitly:

export PRELOOP_SECRETS_STORE=envelope
export PRELOOP_KMS_SECRETS_KEY=alias/preloop-secrets
export PRELOOP_ORG_ROLE_ARN=arn:aws:iam::111122223333:role/preloop-engine-org-secrets
preloop serve

Setting	Default	Purpose
PRELOOP_SECRETS_STORE	envelope on PostgreSQL, file on SQLite	Where tenant workflow-secret values live. file is the local sealed-blob store; selecting envelope in a build without aws-kms is a startup error, never a silent fallback.
PRELOOP_KMS_SECRETS_KEY	alias/preloop-secrets	CMK (key id, ARN, or alias) that wraps each row’s data key.
PRELOOP_ORG_ROLE_ARN	unset (required for envelope)	ABAC role assumed per organization with session tag org_id=<org-id>.
PRELOOP_ORG_SESSION_SECONDS	900	Assumed-session lifetime, 900–43200 seconds. Sessions are cached per organization and renewed about two minutes before expiry.
PRELOOP_KMS_ENDPOINT	unset (AWS)	Endpoint override for a KMS stub or LocalStack. Region and base credentials still come from the standard AWS chain.

​

What one row is

Each value is one row in control.secret_envelopes, keyed by (org_id, secret_ref):

Column	Contents
org_id, secret_ref, version	The row’s identity and write version. All three are bound into the AEAD’s associated data.
key_id	The CMK that wrapped this row’s data key.
nonce	12-byte AES-256-GCM nonce, random per write.
ciphertext	The value, AES-256-GCM encrypted with the tag appended.
wrapped_key	A fresh AES-256 data key, encrypted by KMS under encryption context {org_id, secret_ref}.

A write calls kms:GenerateDataKey (AES-256) with that context, seals the value under the returned data key, and drops the plaintext key when the call returns. A read calls kms:Decrypt on the wrapped key with the same context and opens the ciphertext with associated data org_id NUL secret_ref NUL version; a row copied onto another organization, reference, or version fails to authenticate instead of decrypting. Nothing caches a data key beyond one fetch, and no value, key, or ciphertext reaches a log.

control.secrets.value_backend records where each value lives (envelope or file), so a mixed deployment — a hosted PostgreSQL engine and a local SQLite engine — is auditable. Every write stamps it.

​

IAM: the encryption context must equal the session tag

The engine authenticates as a base role (IAM Roles Anywhere) and assumes preloop-engine-org-secrets for each organization, passing org_id as a session tag. The role’s only tenant-secret grant is KMS, conditioned on the request’s encryption context matching the session’s own tag:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "OrgSecretsEnvelope",
      "Effect": "Allow",
      "Action": [
        "kms:Encrypt",
        "kms:Decrypt",
        "kms:GenerateDataKey",
        "kms:DescribeKey"
      ],
      "Resource": "arn:aws:kms:us-east-1:111122223333:key/<secrets-cmk-id>",
      "Condition": {
        "StringEquals": {
          "kms:EncryptionContext:org_id": "${aws:PrincipalTag/org_id}"
        }
      }
    }
  ]
}

Because the caller supplies the encryption context on Decrypt, the condition is what stops organization B from naming organization A’s context: the request context must equal the session’s own tag. The trust policy on the ABAC role requires the org_id session tag (and sts:TagSession on the base role), so the store cannot obtain an untagged session. The base role needs kms:Sign for its own OIDC keys; that is a separate policy, not this one.

One CMK (alias/preloop-secrets) covers every organization for now; per-organization CMKs are a later option and need no store change.

​

Local development: the file store

A SQLite engine defaults to file, which seals each value with AES-256-GCM + HMAC-SHA256 under a key derived from the engine’s private state directory. It is a development and single-node store: the value and its key are both on the host, so a state-directory copy yields the value. Use it only where that is acceptable:

export PRELOOP_SECRETS_STORE=file   # the SQLite default; never use on a hosted PostgreSQL engine

A write does not fall back from file or envelope to any other backend. A failed write fails the command.

​

What is never written

• The control database stores ciphertext and references, never a plaintext value or a plaintext data key.
• Errors name the organization and secret reference (identifiers) and the AWS error code; they never contain a value. SecretString redacts itself in Debug.
• Job resolution fetches values at job start and masks them in logs, exactly as with the file store.

​

Operational notes

• Rotating the CMK is a KMS operation. The engine reads key_id from each row, so old rows stay readable while their key is enabled; re-wrap by re-writing a value if you require one key.
• A KMS access denial fails the write or the job loudly, naming the organization and reference. A missing row is the only condition reported as “no value”.
• There is no in-place migration command from a previous store: a value that is absent from secret_envelopes resolves as absent. Re-run preloop ci secrets import (or preloop ci migrate) to re-import from GitHub, or preloop ci secrets set for a value the human holds.
• Back up the control database and the KMS key policy together. A restored database without the CMK is unreadable by design.