Skip to content

Commit 8ffe358

Browse files
committed
final edits for kms etcd
1 parent f65266d commit 8ffe358

1 file changed

Lines changed: 32 additions & 31 deletions

File tree

articles/aks/use-kms-etcd-encryption.md

Lines changed: 32 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -32,29 +32,29 @@ For more information on using KMS, see [Encrypting Secret Data at Rest](https://
3232
The following limitations apply when you integrate KMS etcd encryption with AKS:
3333

3434
* Deleting the key, the key vault, or the associated identity isn't supported.
35-
* KMS etcd encryption doesn't work with system-assigned managed identity. The key vault access policy must be set before the feature is turned on. System-assigned managed identity isn't available until after the cluster is created. So, there's a cycle dependency.
36-
* Azure Key Vault with a firewall turned on to allow public access isn't supported because it blocks traffic from the KMS plugin to the key vault.
35+
* KMS etcd encryption doesn't work with system-assigned managed identity. The key vault access policy must be set before the feature is turned on. System-assigned managed identity isn't available until after the cluster is created. Consider the cycle dependency.
36+
* Azure Key Vault with a firewall to allow public access isn't supported because it blocks traffic from the KMS plugin to the key vault.
3737
* The maximum number of secrets that are supported by a cluster that has KMS turned on is 2,000. However, it's important to note that [KMS v2][kms-v2-support] isn't limited by this restriction and can handle a higher number of secrets.
3838
* Bring your own (BYO) Azure key vault from another tenant isn't supported.
3939
* With KMS turned on, you can't change the associated key vault mode (public versus private). To [update a key vault mode][update-a-key-vault-mode], you must first turn off KMS, and then turn it on again.
40-
* If a cluster has KMS turned on, a private key vault, and isn't using the `API Server VNet integration` tunnel, you can't stop and then start the cluster.
41-
* Using the Virtual Machine Scale Sets API to scale the nodes in the cluster down to zero deallocates the nodes, causing the cluster to go down and become unrecoverable.
40+
* If a cluster has KMS turned on, has a private key vault, and isn't using the API Server VNet integration tunnel, you can't stop and then start the cluster.
41+
* Using the Virtual Machine Scale Sets API to scale the nodes in the cluster down to zero deallocates the nodes. The cluster then goes down and becomes unrecoverable.
4242
* After you turn off KMS, you can't destroy the keys. Destroying the keys causes the API server to stop working.
4343

44-
KMS supports a [public key vault][turn-on-kms-for-a-public-key-vault] and a [private key vault][turn-on-kms-for-a-private-key-vault].
44+
KMS supports a [public key vault][turn-on-kms-for-a-public-key-vault] or a [private key vault][turn-on-kms-for-a-private-key-vault].
4545

4646
## Turn on KMS for a public key vault
4747

4848
The following sections describe how to turn on KMS for a public key vault.
4949

50-
### Create a key vault and key
50+
### Create a public key vault and key
5151

5252
> [!WARNING]
5353
> Deleting the key or the key vault is not supported and causes the secrets in the cluster to be unrecoverable.
5454
>
5555
> If you need to recover your key vault or your key, see [Azure Key Vault recovery management with soft delete and purge protection](../key-vault/general/key-vault-recovery.md?tabs=azure-cli).
5656
57-
#### Create a key vault and key for a non-RBAC key vault
57+
#### Create a key vault and key for a non-RBAC public key vault
5858

5959
Use `az keyvault create` to create a key vault without using Azure role-based access control (Azure RBAC):
6060

@@ -77,7 +77,7 @@ echo $KEY_ID
7777

7878
This example stores the key ID in `KEY_ID`.
7979

80-
#### Create a key vault and key for an RBAC key vault
80+
#### Create a key vault and key for an RBAC public key vault
8181

8282
Use `az keyvault create` to create a key vault by using Azure RBAC:
8383

@@ -106,7 +106,7 @@ echo $KEY_ID
106106

107107
This example stores the key ID in `KEY_ID`.
108108

109-
### Create a user-assigned managed identity
109+
### Create a user-assigned managed identity for a public key vault
110110

111111
Use `az identity create` to create a user-assigned managed identity:
112112

@@ -132,47 +132,49 @@ echo $IDENTITY_RESOURCE_ID
132132

133133
This example stores the value of the identity resource ID in `IDENTITY_RESOURCE_ID`.
134134

135-
### Assign permissions to decrypt and encrypt a key vault
135+
### Assign permissions to decrypt and encrypt a public key vault
136136

137-
#### Permissions for a non-RBAC key vault
137+
The following sections describe how to assign decrypt and encrypt permissions for a private key vault.
138+
139+
#### Assign permissions for a non-RBAC public key vault
138140

139141
If your key vault is not set with `--enable-rbac-authorization`, you can use `az keyvault set-policy` to create an Azure key vault policy.
140142

141143
```azurecli-interactive
142144
az keyvault set-policy -n MyKeyVault --key-permissions decrypt encrypt --object-id $IDENTITY_OBJECT_ID
143145
```
144146

145-
#### Permissions for an RBAC key vault
147+
#### Assign permissions for an RBAC public key vault
146148

147149
If your key vault is set with `--enable-rbac-authorization`, assign the Key Vault Crypto User role to give decrypt and encrypt permissions.
148150

149151
```azurecli-interactive
150152
az role assignment create --role "Key Vault Crypto User" --assignee-object-id $IDENTITY_OBJECT_ID --assignee-principal-type "ServicePrincipal" --scope $KEYVAULT_RESOURCE_ID
151153
```
152154

153-
### Create an AKS cluster with KMS etcd encryption turned on
155+
### Create an AKS cluster that has a public key vault and turn on KMS etcd encryption
154156

155157
To turn on KMS etcd encryption, create an AKS cluster by using the [az aks create][az-aks-create] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az aks create`.
156158

157159
```azurecli-interactive
158160
az aks create --name myAKSCluster --resource-group MyResourceGroup --assign-identity $IDENTITY_RESOURCE_ID --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $KEY_ID
159161
```
160162

161-
### Update an existing AKS cluster to turn on KMS etcd encryption
163+
### Update an existing AKS cluster to turn on KMS etcd encryption for a public key vault
162164

163-
To turn on KMS etcd encryption on an existing cluster, use the [az aks update][az-aks-update] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az-aks-update`.
165+
To turn on KMS etcd encryption for an existing cluster, use the [az aks update][az-aks-update] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az-aks-update`.
164166

165167
```azurecli-interactive
166168
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $KEY_ID
167169
```
168170

169-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are no longer encrypted. For larger clusters, you might want to subdivide the secrets by namespace or create a script update.
171+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are no longer encrypted. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
170172

171173
```azurecli-interactive
172174
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
173175
```
174176

175-
### Rotate existing keys
177+
### Rotate existing keys in a public key vault
176178

177179
After you change the key ID (including changing either the key name or the key version), you can use the [az aks update][az-aks-update] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az-aks-update` to rotate existing keys in KMS.
178180

@@ -185,7 +187,7 @@ After you change the key ID (including changing either the key name or the key v
185187
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $NEW_KEY_ID
186188
```
187189

188-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
190+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
189191

190192
```azurecli-interactive
191193
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
@@ -238,25 +240,25 @@ This example stores the value of the identity resource ID in `IDENTITY_RESOURCE_
238240

239241
### Assign permissions to decrypt and encrypt a private key vault
240242

241-
The following sections describe how to assign decrypt and encrypt permissions for a key vault.
243+
The following sections describe how to assign decrypt and encrypt permissions for a private key vault.
242244

243-
#### Permissions for a non-RBAC private key vault
245+
#### Assign permissions for a non-RBAC private key vault
244246

245247
If your key vault is not set with `--enable-rbac-authorization`, you can use `az keyvault set-policy` to create a key vault policy in Azure:
246248

247249
```azurecli-interactive
248250
az keyvault set-policy -n MyKeyVault --key-permissions decrypt encrypt --object-id $IDENTITY_OBJECT_ID
249251
```
250252

251-
#### Permissions for an RBAC private key vault
253+
#### Assign permissions for an RBAC private key vault
252254

253-
If your key vault is set with `--enable-rbac-authorization`, assign an Azure RBAC role that contains decrypt and encrypt permissions:
255+
If your key vault is set with `--enable-rbac-authorization`, assign an Azure RBAC role that includes decrypt and encrypt permissions:
254256

255257
```azurecli-interactive
256258
az role assignment create --role "Key Vault Crypto User" --assignee-object-id $IDENTITY_OBJECT_ID --assignee-principal-type "ServicePrincipal" --scope $KEYVAULT_RESOURCE_ID
257259
```
258260

259-
### Assign permission to create a private link
261+
### Assign permissions to create a private link
260262

261263
For private key vaults, the Key Vault Contributor role is required to create a private link between the private key vault and the cluster.
262264

@@ -280,7 +282,7 @@ To turn on KMS etcd encryption on an existing cluster that has a private key vau
280282
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $KEY_ID --azure-keyvault-kms-key-vault-network-access "Private" --azure-keyvault-kms-key-vault-resource-id $KEYVAULT_RESOURCE_ID
281283
```
282284

283-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier aren't encrypted. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
285+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier aren't encrypted. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
284286

285287
```azurecli-interactive
286288
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
@@ -299,7 +301,7 @@ After you change the key ID (including the key name and the key version), you ca
299301
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $NewKEY_ID --azure-keyvault-kms-key-vault-network-access "Private" --azure-keyvault-kms-key-vault-resource-id $KEYVAULT_RESOURCE_ID
300302
```
301303

302-
Use the following command to update all secrets. If you don't update all secrets, secrets that were created earlier are encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
304+
Use the following command to update all secrets. If you don't update all secrets, secrets that were created earlier are encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
303305

304306
```azurecli-interactive
305307
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
@@ -328,9 +330,9 @@ Update the key vault from public to private:
328330
az keyvault update --name MyKeyVault --resource-group MyResourceGroup --public-network-access Disabled
329331
```
330332

331-
### Turn on KMS on the cluster by using the updated key vault
333+
### Turn on KMS for the cluster by using the updated key vault
332334

333-
Turn on KMS again by using the updated private key vault:
335+
Turn on KMS by using the updated private key vault:
334336

335337
```azurecli-interactive
336338
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $NewKEY_ID --azure-keyvault-kms-key-vault-network-access "Private" --azure-keyvault-kms-key-vault-resource-id $KEYVAULT_RESOURCE_ID
@@ -352,7 +354,7 @@ If the results confirm KMS that is on, run the following command to turn off KMS
352354
az aks update --name myAKSCluster --resource-group MyResourceGroup --disable-azure-keyvault-kms
353355
```
354356

355-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key, and the encrypt and decrypt permissions on the key vault are still required. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
357+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key, and the encrypt and decrypt permissions on the key vault are still required. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
356358

357359
```azurecli-interactive
358360
kubectl get secrets --all-namespaces -o json | kubectl replace -f -
@@ -396,16 +398,15 @@ To upgrade an AKS cluster, use the `az aks upgrade` command. Set the version to
396398
az aks upgrade --resource-group myResourceGroup --name myAKSCluster --kubernetes-version <AKS version>
397399
```
398400

399-
For example:
401+
Here's an example:
400402

401403
```azurecli-interactive
402404
az aks upgrade --resource-group myResourceGroup --name myAKSCluster --kubernetes-version 1.27.1
403405
```
404406

405407
#### Turn on KMS after storage migration
406408

407-
You can turn on the KMS feature on the cluster again to encrypt the secrets. Afterward, the AKS cluster uses KMS v2.
408-
If you don't want to migrate to KMS v2, you can create a new cluster that is version 1.27 or later with KMS turned on.
409+
You can turn on the KMS feature on the cluster again to encrypt the secrets. Afterward, the AKS cluster uses KMS v2. If you don't want to migrate to KMS v2, you can create a new cluster that is version 1.27 or later with KMS turned on.
409410

410411
#### Migrate storage for KMS v2
411412

0 commit comments

Comments
 (0)