You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: support/azure/azure-kubernetes/extensions/istio-add-on-egress-gateway.md
+14-14Lines changed: 14 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -15,37 +15,37 @@ ms.custom: sap:Extensions, Policies and Add-Ons
15
15
16
16
## Overview
17
17
18
-
The Istio add-on egress gateway is an Envoy-based proxy that can be used to route outbound traffic from applications in the mesh. The Istio egress gateway is a `ClusterIP` type service and thus is not exposed externally.
18
+
The Istio add-on egress gateway is an Envoy-based proxy that can be used to route outbound traffic from applications in the mesh. The Istio egress gateway is a `ClusterIP` type service and thus isn't exposed externally.
19
19
20
-
The Istio add-on egress gateway also takes a hard dependency on the [Static Egress Gateway feature](https://learn.microsoft.com/en-us/azure/aks/configure-static-egress-gateway). You must enable Static Egress Gateway on your cluster, create an agentpool of mode: `gateway`, and configure a `StaticGatewayConfiguration` custom resource prior to enabling an Istio add-on egress gateway.
20
+
The Istio add-on egress gateway also takes a hard dependency on the [Static Egress Gateway feature](https://learn.microsoft.com/en-us/azure/aks/configure-static-egress-gateway). You must enable Static Egress Gateway on your cluster, create an agent pool of mode: `gateway`, and configure a `StaticGatewayConfiguration` custom resource before enabling an Istio add-on egress gateway.
21
21
22
-
You can create multiple Istio add-on egress gateways across different namespaces with a Deployment/Service `name` of you choice, with a max of `2000` egress gateways per cluster. Names must be unique per namespace. Names should be a valid DNS name, and must be less than or equal to 63 characters in length, can only consist of lowercase alphanumerical characters, '.' and '-', and must start and end with a lowercase alphanumerical character. The regex used for Istio egress name validations is: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$`.
22
+
You can create multiple Istio add-on egress gateways across different namespaces with a Deployment/Service `name` of your choice, with a max of `2000` egress gateways per cluster. Names must be unique per namespace. Names should be a valid DNS name, and must be less than or equal to 63 characters in length, can only consist of lowercase alphanumerical characters, '.' and '-,' and must start and end with a lowercase alphanumerical character. The regex used for Istio egress name validations is: `^[a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*$`.
23
23
24
24
## Troubleshooting Checklist
25
25
26
26
### Networking and Firewall Errors
27
-
- If you are using Azure Firewall, Network Security Group (NSG) rules, or other outbound traffic restrictions, ensure that the IP ranges from the `egressIpPrefix` for the Istio add-on egress gateway `StaticGatewayConfigurations` are whitelisted for egress communication.
28
-
- Because Static Egress Gateway is currently not supported on Azure CNI Pod Subnet clusters, the Istio add-on egress gateway cannot be used on Azure CNI Pod Subnet clusters either.
27
+
- If you're using Azure Firewall, Network Security Group (NSG) rules, or other outbound traffic restrictions, ensure that the IP ranges from the `egressIpPrefix` for the Istio add-on egress gateway `StaticGatewayConfigurations` are allowlisted for egress communication.
28
+
- Because Static Egress Gateway is currently not supported on Azure CNI Pod Subnet clusters, the Istio add-on egress gateway can't be used on Azure CNI Pod Subnet clusters either.
29
29
30
30
### Egress Gateway Provisioning Issues
31
31
32
-
- If the egress gateway pods are stuck in `ContainerCreating`, this is likely because the `kube-egress-gateway-cni-manager`is preventing the `istio-proxy` container from being created. This can happen if the `StaticGatewayConfiguration` for that Istio add-on egress gateway does not have an `egressIpPrefix` assigned to it yet. You can check the `status` of the `StaticGatewayConfiguration` for that Istio egress gateway to verify whether it has been assigned an `egressIpPrefix`, and also by running `kubectl describe` against the `StaticGatewayConfiguration` to view if there are any errors with the `egressIpPrefix` provisioning. You can also check the logs of the `kube-egress-gateway-cni-manager` pod running on the node of the failing Istio egress pod. Note that it can take up to ~5 minutes for a Static Egress Gateway `StaticGatewayConfiguration` to be assigned an `egressIpPrefix`.
33
-
- Ensure that self-managed mutating and validating webhooks are not blocking provisioning of the Istio egress gateway resources. Because the Istio egress gateway can be deployed in user-managed namespaces (BYO-namespace), [AKS admissions enforcer](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces-)cannot prevent custom admission controllers from affecting the Istio egress gateway deployment/namespace.
34
-
- Ensure that the egress gateway name adheres to the regex pattern mentioned above and is a valid DNS name.
32
+
- If the egress gateway pods are stuck in `ContainerCreating`, the `kube-egress-gateway-cni-manager`could be preventing the `istio-proxy` container from being created because `StaticGatewayConfiguration` for that Istio add-on egress gateway doesn't have an `egressIpPrefix` assigned to it yet. You can check the `status` of the `StaticGatewayConfiguration` for that Istio egress gateway to verify whether it has been assigned an `egressIpPrefix`, and also by running `kubectl describe` against the `StaticGatewayConfiguration` to view if there are any errors with the `egressIpPrefix` provisioning. You can also check the logs of the `kube-egress-gateway-cni-manager` pod running on the node of the failing Istio egress pod. Note that it can take up to ~5 minutes for a Static Egress Gateway `StaticGatewayConfiguration` to be assigned an `egressIpPrefix`.
33
+
- Ensure that self-managed mutating and validating webhooks aren't blocking provisioning of the Istio egress gateway resources. Because the Istio egress gateway can be deployed in user-managed namespaces (BYO-namespace), [AKS admissions enforcer](https://learn.microsoft.com/en-us/azure/aks/faq#can-admission-controller-webhooks-impact-kube-system-and-internal-aks-namespaces-)can't prevent custom admission controllers from affecting the Istio egress gateway deployment/namespace.
34
+
- Ensure that the egress gateway name adheres to the aforementioned regex pattern and is a valid DNS name.
35
35
36
36
### Static Egress Gateway Errors
37
37
38
-
- Because the Istio add-on egress gateway routes traffic through the Static Egress Gateway, it is possible that underlying networking and provisioning issues could be due to an error with Static Egress Gateway - for instance, Istio egress `ContainerCreating` issues due to a missing `egressIpPrefix` as mentioned above. See the [Static Egress Gateway docs](https://learn.microsoft.com/en-us/azure/aks/configure-static-egress-gateway) for more information on creating and configuring the Static Egress Gateway.
39
-
- Verify that the `spec.gatewayNodepoolName` for the `StaticGatewayConfiguration` for each Istio egress gateway references a valid agentpool of mode: `Gateway` on the cluster. You should not delete a gateway agentpool if it is being referenced by any Istio add-on egress gateway `StaticGatewayConfiguration`.
40
-
- Ensure that the `StaticGatewayConfiguration` for the Istio add-on egress gateway has a valid configuration and has not been deleted.
38
+
- Because the Istio add-on egress gateway routes traffic through the Static Egress Gateway, it's possible that underlying networking and provisioning issues could be due to an error with Static Egress Gateway - for instance, Istio egress `ContainerCreating` issues due to a missing `egressIpPrefix` as mentioned above. See the [Static Egress Gateway docs](https://learn.microsoft.com/en-us/azure/aks/configure-static-egress-gateway) for more information on how to create and configure the Static Egress Gateway.
39
+
- Verify that the `spec.gatewayNodepoolName` for the `StaticGatewayConfiguration` for each Istio egress gateway references a valid agent pool of mode: `Gateway` on the cluster. You shouldn't delete a gateway agent pool if any Istio add-on egress gateway `StaticGatewayConfiguration` is referencing it in `spec.gatewayNodepoolName`.
40
+
- Ensure that the `StaticGatewayConfiguration` for the Istio add-on egress gateway has a valid configuration and hasn't been deleted.
41
41
- Verify that the Istio add-on egress gateway pod spec has the `kubernetes.azure.com/static-gateway-configuration` annotation set to the name of the `StaticGatewayConfiguration` for that Istio add-on egress gateway.
42
-
- To validate that requests from the egress gateway are being routed correctly via the Static Egress Gateway nodepool, you can use a Kubernetes ephemeral container with `kubectl debug` to send an external request from the egress pod and verify the source IP of the request. If you're doing this, make sure that you temporarily set `outboundTrafficPolicy.mode` to `ALLOW_ANY` so that the egress gateway can access `ifconfig.me`.
42
+
- To validate that requests from the egress gateway are being routed correctly via the Static Egress Gateway nodepool, you can use a Kubernetes ephemeral container with `kubectl debug` to send an external request from the egress pod and verify the source IP of the request. Make sure that you temporarily set `outboundTrafficPolicy.mode` to `ALLOW_ANY` so that the egress gateway can access `ifconfig.me`.
The source IP address returned in the above commandshould match the `egressIpPrefix` of the `StaticGatewayConfiguration` associated with that Istio egress gateway.
48
+
The source IP address returned should match the `egressIpPrefix` of the `StaticGatewayConfiguration` associated with that Istio egress gateway.
49
49
50
50
- Updates to certain `StaticGatewayConfiguration` fields, such as `defaultRoute` and `excludeCidrs` require the Istio add-on egress gateway pods to be restarted for the changes to the `StaticGatewayConfiguration` take effect. You can bounce the pod by triggering a restart of the egress gateway deployment:
51
51
@@ -91,7 +91,7 @@ More information about Istio egress configuration can be found on the open sourc
91
91
92
92
- You can enable Envoy access logging via the [Istio MeshConfig](https://learn.microsoft.com/en-us/azure/aks/istio-meshconfig) or [Telemetry API](https://learn.microsoft.com/en-us/azure/aks/istio-telemetry) to inspect traffic flowing through the egress gateway.
93
93
94
-
- Ensure that you have created a `ServiceEntry` custom resource for the specific external service that that the egress gateway is routing traffic to. This may be necessary even if the `outboundTrafficPolicy.mode` is set to `ALLOW_ANY`, since the `Gateway`, `VirtualService`, and `DestinationRule` custom resources may reference an external host via a `ServiceEntry` name.
94
+
- Ensure that you have created a `ServiceEntry` custom resource for the specific external service that that the egress gateway is routing traffic to. Creating a `ServiceEntry` may be necessary even if the `outboundTrafficPolicy.mode` is set to `ALLOW_ANY`, since the `Gateway`, `VirtualService`, and `DestinationRule` custom resources may reference an external host via a `ServiceEntry` name.
95
95
96
96
- When configuring a `ServiceEntry` to be used by an Istio egress gateway, the `spec.resolution` must be set to `DNS`.
0 commit comments