You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/aks/use-kms-etcd-encryption.md
+32-31Lines changed: 32 additions & 31 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,29 +32,29 @@ For more information on using KMS, see [Encrypting Secret Data at Rest](https://
32
32
The following limitations apply when you integrate KMS etcd encryption with AKS:
33
33
34
34
* Deleting the key, the key vault, or the associated identity isn't supported.
35
-
* KMS etcd encryption doesn't work with system-assigned managed identity. The key vault access policy must be set before the feature is turned on. System-assigned managed identity isn't available until after the cluster is created. So, there's a cycle dependency.
36
-
* Azure Key Vault with a firewall turned on to allow public access isn't supported because it blocks traffic from the KMS plugin to the key vault.
35
+
* KMS etcd encryption doesn't work with system-assigned managed identity. The key vault access policy must be set before the feature is turned on. System-assigned managed identity isn't available until after the cluster is created. Consider the cycle dependency.
36
+
* Azure Key Vault with a firewall to allow public access isn't supported because it blocks traffic from the KMS plugin to the key vault.
37
37
* The maximum number of secrets that are supported by a cluster that has KMS turned on is 2,000. However, it's important to note that [KMS v2][kms-v2-support] isn't limited by this restriction and can handle a higher number of secrets.
38
38
* Bring your own (BYO) Azure key vault from another tenant isn't supported.
39
39
* With KMS turned on, you can't change the associated key vault mode (public versus private). To [update a key vault mode][update-a-key-vault-mode], you must first turn off KMS, and then turn it on again.
40
-
* If a cluster has KMS turned on, a private key vault, and isn't using the `API Server VNet integration` tunnel, you can't stop and then start the cluster.
41
-
* Using the Virtual Machine Scale Sets API to scale the nodes in the cluster down to zero deallocates the nodes, causing the cluster to go down and become unrecoverable.
40
+
* If a cluster has KMS turned on, has a private key vault, and isn't using the API Server VNet integration tunnel, you can't stop and then start the cluster.
41
+
* Using the Virtual Machine Scale Sets API to scale the nodes in the cluster down to zero deallocates the nodes. The cluster then goes down and becomes unrecoverable.
42
42
* After you turn off KMS, you can't destroy the keys. Destroying the keys causes the API server to stop working.
43
43
44
-
KMS supports a [public key vault][turn-on-kms-for-a-public-key-vault]and a [private key vault][turn-on-kms-for-a-private-key-vault].
44
+
KMS supports a [public key vault][turn-on-kms-for-a-public-key-vault]or a [private key vault][turn-on-kms-for-a-private-key-vault].
45
45
46
46
## Turn on KMS for a public key vault
47
47
48
48
The following sections describe how to turn on KMS for a public key vault.
49
49
50
-
### Create a key vault and key
50
+
### Create a public key vault and key
51
51
52
52
> [!WARNING]
53
53
> Deleting the key or the key vault is not supported and causes the secrets in the cluster to be unrecoverable.
54
54
>
55
55
> If you need to recover your key vault or your key, see [Azure Key Vault recovery management with soft delete and purge protection](../key-vault/general/key-vault-recovery.md?tabs=azure-cli).
56
56
57
-
#### Create a key vault and key for a non-RBAC key vault
57
+
#### Create a key vault and key for a non-RBAC public key vault
58
58
59
59
Use `az keyvault create` to create a key vault without using Azure role-based access control (Azure RBAC):
60
60
@@ -77,7 +77,7 @@ echo $KEY_ID
77
77
78
78
This example stores the key ID in `KEY_ID`.
79
79
80
-
#### Create a key vault and key for an RBAC key vault
80
+
#### Create a key vault and key for an RBAC public key vault
81
81
82
82
Use `az keyvault create` to create a key vault by using Azure RBAC:
83
83
@@ -106,7 +106,7 @@ echo $KEY_ID
106
106
107
107
This example stores the key ID in `KEY_ID`.
108
108
109
-
### Create a user-assigned managed identity
109
+
### Create a user-assigned managed identity for a public key vault
110
110
111
111
Use `az identity create` to create a user-assigned managed identity:
112
112
@@ -132,47 +132,49 @@ echo $IDENTITY_RESOURCE_ID
132
132
133
133
This example stores the value of the identity resource ID in `IDENTITY_RESOURCE_ID`.
134
134
135
-
### Assign permissions to decrypt and encrypt a key vault
135
+
### Assign permissions to decrypt and encrypt a public key vault
136
136
137
-
#### Permissions for a non-RBAC key vault
137
+
The following sections describe how to assign decrypt and encrypt permissions for a private key vault.
138
+
139
+
#### Assign permissions for a non-RBAC public key vault
138
140
139
141
If your key vault is not set with `--enable-rbac-authorization`, you can use `az keyvault set-policy` to create an Azure key vault policy.
140
142
141
143
```azurecli-interactive
142
144
az keyvault set-policy -n MyKeyVault --key-permissions decrypt encrypt --object-id $IDENTITY_OBJECT_ID
143
145
```
144
146
145
-
#### Permissions for an RBAC key vault
147
+
#### Assign permissions for an RBAC public key vault
146
148
147
149
If your key vault is set with `--enable-rbac-authorization`, assign the Key Vault Crypto User role to give decrypt and encrypt permissions.
148
150
149
151
```azurecli-interactive
150
152
az role assignment create --role "Key Vault Crypto User" --assignee-object-id $IDENTITY_OBJECT_ID --assignee-principal-type "ServicePrincipal" --scope $KEYVAULT_RESOURCE_ID
151
153
```
152
154
153
-
### Create an AKS cluster with KMS etcd encryption turned on
155
+
### Create an AKS cluster that has a public key vault and turn on KMS etcd encryption
154
156
155
157
To turn on KMS etcd encryption, create an AKS cluster by using the [az aks create][az-aks-create] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az aks create`.
156
158
157
159
```azurecli-interactive
158
160
az aks create --name myAKSCluster --resource-group MyResourceGroup --assign-identity $IDENTITY_RESOURCE_ID --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $KEY_ID
159
161
```
160
162
161
-
### Update an existing AKS cluster to turn on KMS etcd encryption
163
+
### Update an existing AKS cluster to turn on KMS etcd encryption for a public key vault
162
164
163
-
To turn on KMS etcd encryption on an existing cluster, use the [az aks update][az-aks-update] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az-aks-update`.
165
+
To turn on KMS etcd encryption for an existing cluster, use the [az aks update][az-aks-update] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az-aks-update`.
164
166
165
167
```azurecli-interactive
166
168
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $KEY_ID
167
169
```
168
170
169
-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are no longer encrypted. For larger clusters, you might want to subdivide the secrets by namespace or create a script update.
171
+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are no longer encrypted. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
After you change the key ID (including changing either the key name or the key version), you can use the [az aks update][az-aks-update] command. You can use the `--enable-azure-keyvault-kms`, `--azure-keyvault-kms-key-vault-network-access`, and `--azure-keyvault-kms-key-id` parameters with `az-aks-update` to rotate existing keys in KMS.
178
180
@@ -185,7 +187,7 @@ After you change the key ID (including changing either the key name or the key v
185
187
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-vault-network-access "Public" --azure-keyvault-kms-key-id $NEW_KEY_ID
186
188
```
187
189
188
-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
190
+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
@@ -238,25 +240,25 @@ This example stores the value of the identity resource ID in `IDENTITY_RESOURCE_
238
240
239
241
### Assign permissions to decrypt and encrypt a private key vault
240
242
241
-
The following sections describe how to assign decrypt and encrypt permissions for a key vault.
243
+
The following sections describe how to assign decrypt and encrypt permissions for a private key vault.
242
244
243
-
#### Permissions for a non-RBAC private key vault
245
+
#### Assign permissions for a non-RBAC private key vault
244
246
245
247
If your key vault is not set with `--enable-rbac-authorization`, you can use `az keyvault set-policy` to create a key vault policy in Azure:
246
248
247
249
```azurecli-interactive
248
250
az keyvault set-policy -n MyKeyVault --key-permissions decrypt encrypt --object-id $IDENTITY_OBJECT_ID
249
251
```
250
252
251
-
#### Permissions for an RBAC private key vault
253
+
#### Assign permissions for an RBAC private key vault
252
254
253
-
If your key vault is set with `--enable-rbac-authorization`, assign an Azure RBAC role that contains decrypt and encrypt permissions:
255
+
If your key vault is set with `--enable-rbac-authorization`, assign an Azure RBAC role that includes decrypt and encrypt permissions:
254
256
255
257
```azurecli-interactive
256
258
az role assignment create --role "Key Vault Crypto User" --assignee-object-id $IDENTITY_OBJECT_ID --assignee-principal-type "ServicePrincipal" --scope $KEYVAULT_RESOURCE_ID
257
259
```
258
260
259
-
### Assign permission to create a private link
261
+
### Assign permissions to create a private link
260
262
261
263
For private key vaults, the Key Vault Contributor role is required to create a private link between the private key vault and the cluster.
262
264
@@ -280,7 +282,7 @@ To turn on KMS etcd encryption on an existing cluster that has a private key vau
280
282
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $KEY_ID --azure-keyvault-kms-key-vault-network-access "Private" --azure-keyvault-kms-key-vault-resource-id $KEYVAULT_RESOURCE_ID
281
283
```
282
284
283
-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier aren't encrypted. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
285
+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier aren't encrypted. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
@@ -299,7 +301,7 @@ After you change the key ID (including the key name and the key version), you ca
299
301
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $NewKEY_ID --azure-keyvault-kms-key-vault-network-access "Private" --azure-keyvault-kms-key-vault-resource-id $KEYVAULT_RESOURCE_ID
300
302
```
301
303
302
-
Use the following command to update all secrets. If you don't update all secrets, secrets that were created earlier are encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
304
+
Use the following command to update all secrets. If you don't update all secrets, secrets that were created earlier are encrypted with the previous key. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
@@ -328,9 +330,9 @@ Update the key vault from public to private:
328
330
az keyvault update --name MyKeyVault --resource-group MyResourceGroup --public-network-access Disabled
329
331
```
330
332
331
-
### Turn on KMS on the cluster by using the updated key vault
333
+
### Turn on KMS for the cluster by using the updated key vault
332
334
333
-
Turn on KMS again by using the updated private key vault:
335
+
Turn on KMS by using the updated private key vault:
334
336
335
337
```azurecli-interactive
336
338
az aks update --name myAKSCluster --resource-group MyResourceGroup --enable-azure-keyvault-kms --azure-keyvault-kms-key-id $NewKEY_ID --azure-keyvault-kms-key-vault-network-access "Private" --azure-keyvault-kms-key-vault-resource-id $KEYVAULT_RESOURCE_ID
@@ -352,7 +354,7 @@ If the results confirm KMS that is on, run the following command to turn off KMS
352
354
az aks update --name myAKSCluster --resource-group MyResourceGroup --disable-azure-keyvault-kms
353
355
```
354
356
355
-
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key, and the encrypt and decrypt permissions on the key vault are still required. For larger clusters, you might want to subdivide the secrets by namespace or script an update.
357
+
Use the following command to update all secrets. If you don't run this command, secrets that were created earlier are still encrypted with the previous key, and the encrypt and decrypt permissions on the key vault are still required. For larger clusters, you might want to subdivide the secrets by namespace or create an update script.
@@ -396,16 +398,15 @@ To upgrade an AKS cluster, use the `az aks upgrade` command. Set the version to
396
398
az aks upgrade --resource-group myResourceGroup --name myAKSCluster --kubernetes-version <AKS version>
397
399
```
398
400
399
-
For example:
401
+
Here's an example:
400
402
401
403
```azurecli-interactive
402
404
az aks upgrade --resource-group myResourceGroup --name myAKSCluster --kubernetes-version 1.27.1
403
405
```
404
406
405
407
#### Turn on KMS after storage migration
406
408
407
-
You can turn on the KMS feature on the cluster again to encrypt the secrets. Afterward, the AKS cluster uses KMS v2.
408
-
If you don't want to migrate to KMS v2, you can create a new cluster that is version 1.27 or later with KMS turned on.
409
+
You can turn on the KMS feature on the cluster again to encrypt the secrets. Afterward, the AKS cluster uses KMS v2. If you don't want to migrate to KMS v2, you can create a new cluster that is version 1.27 or later with KMS turned on.
0 commit comments