Installing and upgrading gateways

Anthos Service Mesh gives you the option to deploy and manage gateways as part of your service mesh. A gateway describes a load balancer operating at the edge of the mesh receiving incoming or outgoing HTTP/TCP connections. Gateways are Envoy proxies that provide you with fine-grained control over traffic entering and leaving the mesh. Gateways are primarily used to manage ingress traffic, but you can also configure gateways to manage other types of traffic. For example:

• Egress gateways: An egress gateway lets you configure a dedicated exit node for the traffic leaving the mesh, letting you limit which services can or should access external networks, or to enable secure control of egress traffic to add security to your mesh, for example.

• East-west gateways: A proxy for east-west traffic to allow service workloads to communicate across cluster boundaries in a multi-primary mesh on different networks. By default, this gateway will be public on the Internet.

This page describes best practices for deploying and upgrading the gateway proxies as well as examples of configuring your own istio-ingressgateway and istio-egressgateway gateway proxies. Things like traffic splitting, redirects, and retry logic are possible by applying a Gateway configuration to the gateway proxies. Then instead of adding application-layer traffic routing (L7) to the same API resource, you bind a Virtual Service to the Gateway. This lets you manage gateway traffic like any other data plane traffic in the service mesh.

You can deploy gateways in different ways and you can choose to use more than one topology within the same cluster. Refer to Gateway deployment topologies in the Istio documentation to learn more about these topologies.

Best practices for deploying gateways

1. Deploy and manage the control plane and gateways separately.
2. As a security best practice, we recommended that you deploy gateways in a different namespace from the control plane.
3. Use automatic sidecar injection (auto-injection) to inject the proxy configuration for the gateways just like you use to inject the sidecar proxies for your services.

These best practices:

• Let your namespace administrators manage gateways without needing elevated privileges to your entire cluster.
• Let your administrators use the same deployment tooling or mechanism that they use to manage Kubernetes applications to deploy and manage gateways.
• Gives administrators full control over the gateway Deployment, and also simplifies operations. When a new upgrade is available or a configuration has changed, administrators update gateway Pods by simply restarting them. This makes the experience of operating a gateway Deployment the same as operating sidecar proxies for your services.

Deploy gateways

To support users with existing deployment tools, Anthos Service Mesh supports the same ways to deploy a gateway as Istio: IstioOperator, Helm, and Kubernetes YAML. Each method produces the same result. Although you can choose the method you are most familiar with, we recommend that you use the Kubernetes YAML method because it is easier to modify and you can store hydrated manifests in source control.

1. Create a namespace for the gateway if you don't already have one. Replace GATEWAY_NAMESPACE with the name of your namespace.

   kubectl create namespace GATEWAY_NAMESPACE
   

2. You enable auto-injection on the gateway by applying a a revision label on the gateway namespace. The revision label is used by the sidecar injector webhook to associate injected proxies with a particular control plane revision. The revision label that you use depends on whether you deployed the Google-managed control plane or the in-cluster control plane.

   Select the tab below according to your installation type (either Google-managed or in-cluster).

   Google-managed

   For the Google-managed control plane, the revision label corresponds to a release channel:

   Revision label	Channel
   istio.io/rev=asm-managed	Regular
   istio.io/rev=asm-managed-rapid	Rapid
   istio.io/rev=asm-managed-stable	Stable

   Your gateway namespace can be on the same release channel as your services, or a different release channel. We recommend that you use the same release channel on a cluster. To see what release channel a namespace is using:

   kubectl get namespace NAMESPACE -L istio.io/rev
   

   In-cluster

   For in-cluster control planes, the istiod Service and Deployment typically have a revision label similar to istio.io/rev=asm-1118-4, where asm-1118-4 identifies the Anthos Service Mesh version. The revision becomes part of the istiod Service name, for example: istiod-asm-1118-4.istio-system

   Use the following command to locate the revision label on istiod for the in-cluster control plane:

   kubectl get deploy -n istio-system -l app=istiod -o jsonpath="{.items[*].metadata.labels.'istio\.io\/rev'}"'{"\n"}'
   

3. Enable the namespace for injection. Replace and REVISION with the value for the revision label.

   kubectl label namespace GATEWAY_NAMESPACE istio-injection- istio.io/rev=REVISION --overwrite
   

   You can ignore the message "istio-injection not found" in the output. That means that the namespace didn't previously have the istio-injection label, which you should expect in new installations of Anthos Service Mesh or new deployments. Because auto-injection fails if a namespace has both the istio-injection and the revision label, all kubectl label commands in the Anthos Service Mesh documentation include removing the istio-injection label.

4. If you installed Anthos Service Mesh using asmcli, change to the directory that you specified in --output_dir, then cd to the samples directory.

   Important: To ensure you are in the correct samples directory, run the ls command to list the directory contents and confirm there is a gateways directory (which you will access in the next step) and an online-boutique directory.

   If you didn't install using asmcli, copy the configuration files for the gateways from the anthos-service-mesh repository.

5. You can deploy the example gateway configuration located in the samples/gateways/ directory as is, or modify it as needed.

   Ingress

   kubectl apply -n GATEWAY_NAMESPACE -f samples/gateways/istio-ingressgateway
   

   Egress

   kubectl apply -n GATEWAY_NAMESPACE -f samples/gateways/istio-egressgateway
   

6. After you create the deployment, verify that the new services are working correctly:

   kubectl get pod,service -n GATEWAY_NAMESPACE
   

   Verify the output is similar to the following:

   NAME                                      READY   STATUS    RESTARTS   AGE
   pod/istio-ingressgateway-856b7c77-bdb77   1/1     Running   0          3s
   
   NAME                           TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)        AGE
   service/istio-ingressgateway   LoadBalancer   10.24.5.129    34.82.157.6      80:31904/TCP   3s

Important: Your gateway resources should be managed like any other Kubernetes application. The samples in the anthos-service-mesh-packages repo are meant for guidance and a quickstart and you should customize them according to your needs.

Gateway selectors

You apply a Gateway configuration to the istio-ingressgateway and istio-egressgateway proxies to manage inbound and outbound traffic for your mesh, letting you specify which traffic you want to enter or leave the mesh. The labels on a gateway deployment's Pods are used by Gateway configuration resources, so it's important that your Gateway selector matches these labels.

For example, in the above deployments, the istio=ingressgateway label is set on the gateway Pods. To apply a Gateway configuration to these deployments, you need to select the same label:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: gateway
spec:
  selector:
    istio: ingressgateway
...

For an example of a Gateway configuration and Virtual Service, see frontend.yaml in the Online Boutique sample application.

Upgrading Gateways

In-place Upgrades

For most use cases, you should upgrade your gateways following the in-place upgrade pattern. Because gateways utilize Pod injection, new gateway Pods that are created will automatically be injected with the latest configuration, which includes the version.

If you would like to change the control plane revision in use by the gateway, you can set the istio.io/rev label on the gateway Deployment, which will also trigger a rolling restart.

Google-managed control plane
Since Google manages the control plane upgrades for the Google-managed control plane, you can simply restart the gateway Deployment and the new pods will automatically be injected with the latest configuration and version.

kubectl rollout restart deployment istio-ingressgateway \
  -n GATEWAY_NAMESPACE

In-cluster control plane
To apply the same pattern to your gateways when you have the in-cluster control plane, you will need to change the control plane revision in use by the gateway. Set the istio.io/rev label on the gateway Deployment which will trigger a rolling restart.

1. Update the revision label on namespace or the gateway pod.
   • If you labeled the namespace for injection:
   • Set the istio.io/rev label on the namespace to the new revision value

kubectl label namespace GATEWAY_NAMESPACE \
  istio-injection- istio.io/rev=REVISION \
  --overwrite

OR

• If you enabled injection only for the gateway pod:
  • Set the istio.io/rev label on the Deployment to the new revision value
    • Kubernetes YAML

cat <<EOF > gateway-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: istio-ingressgateway
  namespace: GATEWAY_NAMESPACE
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  template:
    metadata:
      annotations:
        # This is required to tell Anthos Service Mesh to inject the gateway with the
        # required configuration.
        inject.istio.io/templates: gateway
      labels:
        istio: ingressgateway
        istio.io/rev: REVISION
    spec:
      containers:
      - name: istio-proxy
        image: auto # The image will automatically update each time the pod starts.
EOF

kubectl apply -f gateway-deployment.yaml

Canary Upgrades (advanced)

If you are using the in-cluster control plane and would like to more slowly control the rollout of a new control plane revision, you can follow the canary upgrade pattern. You can run multiple versions of a gateway Deployment and ensure everything works as expected with a subset of your traffic. For example, if you want to roll out a new revision, canary, create a copy of your gateway Deployment with the istio.io/rev=REVISION label set to the new revision and a new name, for example istio-ingressgateway-canary:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: istio-ingressgateway-canary
  namespace: GATEWAY_NAMESPACE
spec:
  selector:
    matchLabels:
      istio: ingressgateway
  template:
    metadata:
      annotations:
        inject.istio.io/templates: gateway
      labels:
        istio: ingressgateway
        istio.io/rev: REVISION # Set to the control plane revision you want to deploy
    spec:
      containers:
      - name: istio-proxy
        image: auto

When this Deployment is created, you will then have two versions of the gateway, both selected by the same Service:

kubectl get endpoints -o "custom-columns=NAME:.metadata.name,PODS:.subsets[*].addresses[*].targetRef.name"

NAME                   PODS
istio-ingressgateway   istio-ingressgateway-788854c955-8gv96,istio-ingressgateway-canary-b78944cbd-mq2qf

Note: Unlike application services deployed inside the mesh, you cannot use Istio traffic shifting to distribute the traffic between the gateway versions because their traffic is coming directly from external clients that Istio does not control. Instead, you can control the distribution of traffic by the number of replicas of each Deployment. If you use another load balancer in front of Istio, you may also use that to control the traffic distribution.

If you are satisfied that your applications are working as expected, run this command to transition to the new version by deleting the Deployment with the old istio.io/rev label set:

kubectl delete deploy/istio-ingressgateway -n GATEWAY_NAMESPACE

If you encountered an issue when testing your application with the new version of the gateway, run this command to transition back to the old version by deleting the Deployment with the new istio.io/rev label set:

kubectl delete deploy/istio-ingressgateway-canary -n GATEWAY_NAMESPACE