You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: support/azure/azure-kubernetes/connectivity/tunnel-connectivity-issues.md
+34-10Lines changed: 34 additions & 10 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -263,8 +263,40 @@ To address scalability challenges in large clusters, we have implemented the Clu
263
263
**Why was this change made?**
264
264
Previously, the Konnectivity agent had a fixed replica count, which could create a bottleneck as the cluster grew. With the implementation of the Cluster Proportional Autoscaler (CPA), the replica count now dynamically adjusts based on node-scaling rules, ensuring optimal performance and resource usage.
265
265
266
-
**What should customers check for?**
267
-
Customers need to monitor for Out Of Memory (OOM) kills on their nodes, as the Konnectivity agents run on these nodes. Here are the steps to identify and troubleshoot OOMKills:
266
+
**How does the CPA work?**
267
+
The CPA uses a ladder configuration to determine the number of Konnectivity agent replicas based on the cluster size. The ladder configuration is defined in the konnectivity-agent-autoscaler configmap in the kube-system namespace. Here is an example of the ladder configuration:
268
+
269
+
```
270
+
nodesToReplicas": [
271
+
[1, 2],
272
+
[100, 3],
273
+
[250, 4],
274
+
[500, 5],
275
+
[1000, 6],
276
+
[5000, 10]
277
+
]
278
+
```
279
+
280
+
This configuration ensures that the number of replicas scales appropriately with the number of nodes in the cluster, providing optimal resource allocation and improved networking reliability.
281
+
282
+
**How do customers use the Cluster Proportional Autoscaler (CPA)?**
283
+
Customers can override default values by updating the konnectivity-agent-autoscaler configmap in the kube-system namespace. Here is a sample command to update the configmap:
284
+
285
+
```
286
+
kubectl edit configmap <pod-name> -n kube-system
287
+
```
288
+
This command opens the configmap in an editor where customers can make the necessary changes.
289
+
290
+
**What should customers check for?**
291
+
Customers need to monitor for Out Of Memory (OOM) kills on their nodes because misconfiguration of the CPA can lead to insufficient memory allocation for the Konnectivity agents. Here are the key reasons:
292
+
293
+
**High Memory Usage:** As the cluster grows, the memory usage of Konnectivity agents can increase significantly, especially during peak loads or when handling large numbers of connections. If the CPA configuration does not scale the replicas appropriately, the agents may run out of memory.
294
+
295
+
**Fixed Resource Limits:** If the resource requests and limits for the Konnectivity agents are set too low, they may not have enough memory to handle the workload, leading to OOM kills. Misconfigured CPA settings can exacerbate this issue by not providing enough replicas to distribute the load.
296
+
297
+
**Cluster Size and Workload Variability:** The CPU and memory needed by the Konnectivity agents can vary widely depending on the size of the cluster and the workload. If the CPA ladder configuration is not right-sized and adaptively resized for the cluster's usage patterns, it can lead to memory overcommitment and OOM kills.
298
+
299
+
Here are the steps to identify and troubleshoot OOMKills:
268
300
269
301
1. Check for OOMKills on Nodes: Use the following command to check for OOMKills on your nodes:
270
302
@@ -290,14 +322,6 @@ kubectl get pod <pod-name> -n kube-system -o yaml | grep -A5 "resources:"
**How do customers use the Cluster Proportional Autoscaler (CPA)?**
294
-
Customers can override default values by updating the konnectivity-agent-autoscaler configmap in the kube-system namespace. Here is a sample command to update the configmap:
295
-
296
-
```
297
-
kubectl edit configmap <pod-name> -n kube-system
298
-
```
299
-
This command opens the configmap in an editor where customers can make the necessary changes.
0 commit comments