Enabling optional features on the in-cluster control plane

This page describes how to enable optional features on an in-cluster control plane. For information on the Google-managed control plane, see Configuring the Google-managed control plane.

When you install Anthos Service Mesh, the features that are enabled by default differ by platform. You enable optional features by including an overlay file when you install (or upgrade) Anthos Service Mesh. An overlay file is a YAML file containing an IstioOperator custom resource (CR) that you use to configure the control plane. You can override the default configuration and enable an optional feature or disable a default feature in an overlay file. Specify one feature per overlay file. You can layer on more overlays, and each overlay file overrides the configuration on the previous layers.

Caution: Using the istioctl install command and the install_asm script (which wraps istioctl install) overwrites the existing control plane configuration. When you enable optional features to an existing installation, make sure to specify all files that you want to keep in the original configuration, as well as the overlay files for the features that you want to enable. We recommended that you store the overlay files that you use for installation in source control so that you can apply the same configurations to your control plane with upgrades or when enabling additional optional features.

About the overlay files

The overlay files on this page are in the anthos-service-mesh package in GitHub. These files contain common customizations to the default configuration. You can use these files as they are, or you can make additional changes to them as needed.

• When you install Anthos Service Mesh using the istioctl install command, you can specify one or more overlay files with the -f command-line option. Although you can modify the configuration by specifying configuration parameters on the command-line by using the --set option to istioctl install, we recommend that you use an overlay file so you can store the file in your version-control system along with your other custom resource files. You need to keep these files for when you upgrade Anthos Service Mesh so that your control plane has the same configuration after the upgrade.

• When you install Anthos Service Mesh using the Google-provided install_asm script, you can specify one or more overlay files with the --option or the --custom_overlay options. If you don't need to make any changes to the files in the anthos-service-mesh repository, you can use --option, and the script fetches the file from GitHub for you. Otherwise, you can make changes to the overlay file, and then use the --custom_overlay option to pass it to the install_asm script.

Important: Don't specify more than one IstioOperator CR in an overlay file because istioctl install only applies the first CR that it finds in the file.

Don't include multiple CRs in one overlay file	Create separate overlay files for each CR

Downloading the anthos-service-mesh package

To download the anthos-service-mesh package:

The following steps use kpt to download the asm package from the GitHub repository. If you prefer, you can use git clone instead.

1. Install kpt if you haven't already:

   gcloud components install kpt
   

2. Download the package that contains the files:

   kpt pkg get \
   https://github.com/GoogleCloudPlatform/anthos-service-mesh-packages.git/asm@release-1.9-asm asm
   

   The following examples assume that the asm package is in your current working directory.

How to enable optional features

To enable a feature when you are installing Anthos Service Mesh, the exact command differs slightly depending on your platform and whether you are using the install_asm script or the istioctl install command.

The following examples are simplified to show only using the custom overlays to enable optional features. Replace OTHER_FLAGS with the command line options from the installation guide that you're following.

Important: The order of the files on the command line matters. Be sure to specify the configuration required for the default features first, and then the overlay files after that.

Using the install_asm script

The install_asm script provides two ways to enable an optional feature. The method that you use depends on whether you need to make changes to the overlay file.

• Use --option when you don't need to make any changes to the overlay file. With --option, install_asm fetches the file from the GitHub repository for you, so you must have an internet connection.

  ./install_asm \
    OTHER_FLAGS \
    --option OPTION_NAME
  

  Replace OPTION_NAME with the option you want to enable. For a list of options refer to the anthos-service-mesh package.

• Use --custom_overlay when you need to customize the overlay file.

  ./install_asm \
    OTHER_FLAGS \
    --custom_overlay PATH_TO_FILE
  

  Replace PATH_TO_FILE with the path to the overlay file you want to use.

Using the istioctl install command

You use the istioctl install command for multi-project installations on GKE and for platforms that aren't on Google Cloud, such as Anthos clusters on VMware.

• Use-f to include the overlay file. This can be an overlay file from the anthos-service-mesh package without customizations or a custom overlay file.

  istioctl install  \
   OTHER_FLAGS \
   -f PATH_TO_FILE
  

  Replace PATH_TO_FILE with the path to the overlay file you want to use.

YAML for optional features

The following sections provide the YAML to enable optional and supported features.

mTLS STRICT mode

The global.mtls.enabled configuration was removed from the IstioOperator CR to avoid issues with upgrades and to provide a more flexible installation. To enable STRICT mTLS, configure a peer authentication policy instead.

Direct Envoy to stdout

asm/istio/options/envoy-access-log.yaml

View on GitHub

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    accessLogFile: "/dev/stdout"

For more information, see Enable Envoy's access logging.

Cloud Trace

For installations on GKE, you can enable Cloud Trace. For detailed pricing information, refer to the Cloud Trace pricing page.

asm/istio/options/cloud-tracing.yaml

View on GitHub

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    enableTracing: true
  values:
    global:
      proxy:
        tracer: stackdriver

The default sampling rate is 1%, but you can override the default by specifying a tracing.sampling value. The value must be in the range of 0.0 to 100.0 with a precision of 0.01. For example, to trace 5 requests out of every 10,000, use 0.05.

The following example shows a sampling rate of 100% (which you would only do for demo or troubleshooting purposes).

apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  meshConfig:
    enableTracing: true
    defaultConfig:
      tracing:
        sampling: 100
  values:
    global:
      proxy:
        tracer: stackdriver

Note that currently tracer configuration is part of proxy bootstrap configuration, so pod needs to restart and get re-injected to pick up the tracer update. For example, you can use the following command the restart pods belong to a deployment:

kubectl rollout restart deployment -n NAMESPACE DEPLOYMENT_NAME

Trace context propagation

Although the sidecar proxies can automatically send trace spans, they need some hints to tie together the entire trace. Applications need to propagate the appropriate HTTP headers so that when the proxies send span information, the spans can be correlated correctly into a single trace.

To do this, an application needs to collect and propagate the following headers from the incoming request to any outgoing requests:

• x-request-id
• x-b3-traceid
• x-b3-spanid
• x-b3-parentspanid
• x-b3-sampled
• x-b3-flags
• x-ot-span-context
• x-cloud-trace-context
• traceparent
• grpc-trace-bin

For examples propagating the headers, see Trace context propagation.

Create a trace from client with custom ID

To create a trace from a client with a custom ID, use the curl command to create a request with an external client and force it to show a trace. For example:

curl $URL --header "x-client-trace-id: 105445aa7843bc8bf206b12000100000"

For more information about x-client-trace-id, refer to the Envoy documentation.

Note: If you are testing the Anthos Service Mesh sidecar containers sending Cloud Trace data using the Google microservices-demo, make sure that you disable the built-in Cloud Trace functionality. The microservices-demo is able to send Cloud Trace data without Istio. Not disabling this functionality will result in duplicated traces because both the application container and the sidecar container will send traces. There are several DISABLE_TRACING variables to configure in the microservices-demo release file.

Egress via egress gateways

asm/istio/options/egressgateways.yaml

View on GitHub

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    egressGateways:
      - name: istio-egressgateway
        enabled: true

For more information, see Egress Gateways.

Istio Container Network Interface

How you enable the Istio Container Network Interface (CNI) depends on the environment that Anthos Service Mesh is installed on.

1. Enable a network policy.

2. Choose the overlay file that matches your platform.

Enable CNI on GKE

asm/istio/options/cni-gcp.yaml

View on GitHub

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
      namespace: kube-system
  values:
    cni:
      cniBinDir: /home/kubernetes/bin
      excludeNamespaces:
        - istio-system
        - kube-system

Enable CNI on-premises

asm/istio/options/cni-onprem.yaml

View on GitHub

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    cni:
      enabled: true
      namespace: kube-system
  values:
    cni:
      cniBinDir: /opt/cni/bin
      excludeNamespaces:
        - istio-system
        - kube-system
        - gke-system

Enable an internal load balancer

For installations on GKE, you can enable an internal load balancer for the Istio ingress gateway.

asm/istio/options/internal-load-balancer.yaml

View on GitHub

---
apiVersion: install.istio.io/v1alpha1
kind: IstioOperator
spec:
  components:
    ingressGateways:
      - name: istio-ingressgateway
        enabled: true
        k8s:
          serviceAnnotations:
            cloud.google.com/load-balancer-type: "internal"
          service:
            ports:
              - name: status-port
                port: 15020
              - name: http2
                port: 80
                targetPort: 8080
              - name: https
                port: 443
                targetPort: 8443

External certificate management on the ingress gateway

For information on enabling external certificate management on the ingress gateway using Envoy SDS, see Secure Gateways.