Key Rotation

Rotating the master key creates a new key on the Securosys HSM and re-encrypts the local keystore. The database keys and the data itself are left untouched, so no data is re-encrypted.

Rolling of the internal database encryption keys is not covered by this guide. You can read up here on how to MongoDB does backup and restore.

How it works

The master key is the only externally-managed key per node. When you rotate it, MongoDB requests a new key from the KMIP Server and re-encrypts the keystore under it.

tip

This guide assumes that you are running in a cluster of N nodes. After a successful rotation of a master key, the target node will exit and needs to be started again without the --kmipRotateMasterKey flag.

In a single node setup, rotation happens on the only node, but it introduces technical downtime.

Rotate the replicas

Rotate one member at a time to preserve the write quorum. Never rotate two members or more simultaneously.

1. Rotate each secondary, one at a time

Rotating the master key on a node is straightforward. Restart the SECONDARY nodes with the rotate flag. Identify the secondaries by running

mongosh --port 27017 --eval 'rs.status().members.map(m => m.name + " " + m.stateStr)'
[
  'replica-set-node1:27017 PRIMARY',
  'replica-set-node2:27018 SECONDARY',
  'replica-set-node3:27019 SECONDARY'
]

As a command-line flag:

mongod --config /path/to/mongod.conf --kmipRotateMasterKey

Or as a config file:

security:
  enableEncryption: true
  kmip:
    serverName: <kmip-host>
    port: 5696
    clientCertificateFile: /path/to/kmip-client.pem
    serverCAFile: /path/to/kmip-server.pem
    rotateMasterKey: true

MongoDB requests a new key, re-encrypts the keystore, and the process then exits.

info

If the member pins a specific key with kmip.keyIdentifier, either point it at the new key ID or remove it to have the server issue a fresh key.

2. Restart the secondary

Drop the --kmipRotateMasterKey flag (or remove rotateMasterKey from the config) and start the node again.

mongod --config /path/to/mongod.conf

Confirm it rejoins as a healthy SECONDARY before moving to the next member.

mongosh --port <primary-port> --eval 'rs.status().members.map(m => m.name + " " + m.stateStr)'

3. Rotate the primary

Step down the primary and let the cluster elect a new one:

mongosh --port <primary-port> --eval 'rs.stepDown()'

Once rs.status() shows a new PRIMARY, rotate the stepped-down member using steps 1 and 2.

Verification

After each rotation, confirm on each node that the new key is in use - the master key id logged at startup should match the new key id in the Securosys KMS:

grep -i 'Rotated master encryption key' /path/to/mongod.log | tail -n 1

The log should produce a message similar to:

"Rotated master encryption key","attr": "oldMasterKeyId":"bfb88f41-ca2c-44e5-9011-a7d9f4ef2490","newMasterKeyId":"acd72107-0576-4cd9-b937-5c991847dde1"

In the end, you should see the cluster in a healthy state again, having 1 Primary and N-1 Secondary nodes.

mongosh --port 27017 --eval 'rs.status().members.map(m => m.name + " " + m.stateStr)'
[
  'replica-set-node1:27017 PRIMARY',
  'replica-set-node2:27018 SECONDARY',
  'replica-set-node3:27019 SECONDARY'
]

warning

Do not destroy the old master key on the HSM until every member has rotated and rejoined successfully. A member still wrapped under the old key cannot start once that key is gone.