Resolving sidecar proxies issues in Cloud Service Mesh

This section explains common Cloud Service Mesh sidecar proxies problems and how to resolve them. If you need additional assistance, see Getting support.

The istio-proxy container is killed because of a OOM event

In this section we assume that the istio-proxy container is not killed by a SystemOOM event, and the kubernetes node is not in MemoryPressure condition. The istio-proxy sidecar container has by default resource limits. If the istio-proxy container gets killed with Reason: OOMKilled it is necessary to understand why Envoy is consuming the memory.

If you are facing a production outage, a quick workaround is to raise the limits for all containers using IstioOperator:

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  values:
    global:
      proxy:
        resources:
          requests:
               memory: 128Mi
          limits:
               memory: 1Gi

If you are facing this issue with specific workloads, you can change the limit just on those workloads by adding the following annotations.

  • sidecar.istio.io/proxyMemory
  • sidecar.istio.io/proxyMemoryLimit

Please make sure you don't have limits that are lower of the default values.

The long term solution is to reduce the memory footprint of your istio-proxy sidecar containers. By default all sidecar proxies are programmed with the necessary configuration to reach any other workload instance in the mesh. Istio provides the custom resource definition Sidecar to limit the number of endpoints programmed to sidecar proxies, and therefore reduce the memory consumption of the istio-proxy container.