Explore GKE Enterprise


Explore Google Kubernetes Engine (GKE) Enterprise edition by deploying a sample application using Terraform. It deploys a real hands-on environment with a GKE cluster, Anthos Service Mesh, and an application with multiple microservices. This tutorial introduces you to these features, letting you learn about GKE Enterprise deployed on Google Cloud with a fictional bank. You can then explore GKE Enterprise features that interest you by following the bank's GKE Enterprise story further in our follow-up tutorial.

If you want to learn more about GKE Enterprise features that power GKE Enterprise, see our technical overview. However, you don't need to be familiar with GKE Enterprise or Terraform to follow this tutorial. You should be familiar with basic Kubernetes concepts such as clusters; if you're not, see Kubernetes basics, the Google Kubernetes Engine (GKE) documentation, and Preparing an app for Anthos Service Mesh.

When you're ready for a real production installation, see our Setup section.

Your journey

You are the platform lead at the Bank of Anthos. Bank of Anthos started as a small business for payment processing on two servers almost ten years ago. Since then, it has grown into a successful commercial bank with thousands of employees and a growing engineering organization. Bank of Anthos now wants to expand its business further.

Throughout this period, you and your team have found yourself spending more time and money on maintaining infrastructure than on creating new business value. You have decades of cumulative experience invested in your existing stack; however, you know it's not the right technology to meet the scale of global deployment that the bank needs as it expands.

You've adopted GKE Enterprise to modernize your application and migrate successfully to Google Cloud to achieve your expansion goals.

Objectives

In this tutorial, you're introduced to some of the key features of GKE Enterprise through the following tasks:

  • Deploy your GKE Enterprise environment with a cluster, application, and enterprise features: Anthos Service Mesh, Config Sync, and Policy Controller.

  • Use the Google Cloud console to explore the GKE resources used by your application.

  • Use Anthos Service Mesh to observe application services.

What's deployed?

Deploying the Bank of Anthos on Google Cloud provisions your project with the following:

  • A GKE cluster running on Google Cloud: anthos-sample-cluster1.

  • Anthos Service Mesh installed on the cluster. You use Anthos Service Mesh to manage the service mesh on anthos-sample-cluster1.

  • Config Sync and Policy Controller to manage configuration and security policies on anthos-sample-cluster1.

  • Bank of Anthos application running on the cluster. This is a web-based banking application that uses a number of microservices written in various programming languages, including Java, Python, and JavaScript.

Costs

Deploying the Bank of Anthos application will incur pay-as-you-go charges for GKE Enterprise on Google Cloud as listed on our Pricing page, unless you have already purchased a subscription.

You are also responsible for other Google Cloud costs incurred while running the Bank of Anthos application, such as charges for Compute Engine VMs and load balancers.

We recommend cleaning up after finishing the tutorial or exploring the deployment to avoid incurring further charges.

Before you begin

Make sure to complete the following prerequisites.

Select or create a project

You can use an existing project, or create a new one for this tutorial.

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Google Cloud project.

  4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  5. Make sure that billing is enabled for your Google Cloud project.

Activate Cloud Shell

Cloud Shell is an interactive shell environment for Google Cloud that lets you manage your projects and resources from your web browser.

In the Google Cloud console, activate Cloud Shell.

Activate Cloud Shell

Grant IAM roles

If you are using an existing project, ensure that your Google Cloud account has the required IAM roles for this tutorial.

Grant roles to your Google Account. Run the following command once for each of the following IAM roles: roles/resourcemanager.projectIamAdmin, roles/iam.serviceAccountAdmin, roles/iam.serviceAccountUser, roles/iam.securityAdmin, roles/serviceusage.serviceUsageAdmin, roles/container.admin, roles/logging.logWriter, roles/gkehub.admin, roles/viewer, roles/monitoring.viewer

$ gcloud projects add-iam-policy-binding PROJECT_ID --member="user:EMAIL_ADDRESS" --role=ROLE
  • Replace PROJECT_ID with your project ID.
  • Replace EMAIL_ADDRESS with your email address.
  • Replace ROLE with each individual role.

Deploy Bank of Anthos using Terraform

Once all the prerequisites have been met, run the following commands on your Cloud Shell to deploy Bank of Anthos:

  1. Make sure that your default project is set to the project ID where you want to deploy the application. If it's not already set, run the following command in the Cloud Shell:

    gcloud config set project PROJECT_ID
    

    Replace PROJECT_ID with your Google Cloud project ID.

  2. Clone the Bank of Anthos GitHub repository to your Cloud Shell:

    git clone https://github.com/GoogleCloudPlatform/bank-of-anthos.git
    
  3. Set the TF_VAR_project environment variable to your project ID:

    export TF_VAR_project=PROJECT_ID
    

    Replace PROJECT_ID with your Google Cloud project ID.

  4. Change to the directory that hosts the installation scripts:

    cd bank-of-anthos/iac/tf-anthos-gke
    
  5. Initialize Terraform. To use the latest Google provider version, include the -upgrade flag:

    terraform init -upgrade
    
  6. (Optional) Review the resources that will be created:

    terraform plan
    
  7. Apply the Terraform configuration to create the necessary resources and deploy Bank of Anthos:

    terraform apply
    

If the preceding command completes successfully, the Bank of Anthos application and the resources it runs on are deployed in your project. If you encounter any deployment errors, see Troubleshoot.

Using the GKE Enterprise Overview

GKE Enterprise capabilities are built around the idea of the fleet: a logical grouping of Kubernetes clusters that can be managed together. The GKE Enterprise Overview in the Google Cloud console provides you with a high-level view of your entire fleet.

Go to the GKE Enterprise Overview

The Overview shows you the following information:

  • How many clusters there are in your fleet, and if they are healthy. In this example (provided you have no other existing fleet member clusters), the Clusters in this fleet section tells you that you have one GKE cluster.
  • Your fleet's resource utilization, including CPU, memory, and disk usage, aggregated by fleet and by cluster.
  • Any security concerns identified for your fleet, your fleet-wide Policy Controller coverage, and the synchronization status of your Config Sync packages.

Explore GKE resources

The GKE Clusters page shows you all the clusters in your project. Clusters that are registered to a fleet have their fleet listed in the Fleet column.

In this section, you'll take a closer look at Bank of Anthos' GKE resources.

Clusters

  1. In the Google Kubernetes Engine console, go to the Clusters page.

    Go to the Clusters page

  2. Click the newly deployed anthos-sample-cluster1 cluster. In the cluster details page that opens, you can view basic cluster details along with the cluster's networking and security configurations. You can also see which GKE features are enabled in this cluster in the Features section.

  3. Click the Nodes tab to view all the worker machines in your cluster. You can drill down even further to see the workload Pods running on each node, as well as a resource summary of the node (CPU, memory, storage).

You can find out more about GKE clusters and nodes in the GKE documentation.

Workloads

The GKE console has a Workloads view that shows an aggregated view of the workloads running on all your GKE clusters.

In the Google Kubernetes Engine console, go to the Workloads page.

Go to the Workloads page

The Overview tab shows a list of workloads and namespaces from the GKE cluster. You can filter by namespaces to see what workloads are running in each namespace.

Services & Ingress

The Services & Ingress view shows the project's Service and Ingress resources. A Service exposes a set of Pods as a network service with an endpoint, while an Ingress manages external access to the services in a cluster. However, rather than a regular Kubernetes Ingress, Bank of Anthos uses an Istio ingress gateway service for traffic to the bank, which Anthos Service Mesh users can use to add more complex traffic routing to their meshes' inbound traffic. You can see this in action when you use the service mesh observability features later in this tutorial.

  1. In the Google Kubernetes Engine console, go to the Services & Ingress page.

    Go to the Services & Ingress page

  2. To find the Bank of Anthos ingress gateway, scroll down the list of available services and find the service with the name frontend. An ingress gateway manages inbound traffic for your application's service mesh, so in this case we can use its details to visit the bank's web frontend.

  3. Click the IP address endpoint for the frontend service. This opens the Bank of GKE Enterprise web interface.

Observe services

In GKE Enterprise, service management and observability are provided by Anthos Service Mesh, a suite of tools powered by Istio that helps you monitor and manage a reliable service mesh. To find out more about Anthos Service Mesh and how it helps you manage microservices, see the Anthos Service Mesh documentation. If you're not familiar with using microservices with containers and what they can do for you, see Preparing an app for Anthos Service Mesh.

In our example, the cluster in the sample deployment has the microservice-based Bank of Anthos sample app running on it. The application also includes a loadgenerator utility that simulates a small amount of load to the cluster so that you can see metrics and traffic in the dashboard.

In this section, you'll use the GKE Service Mesh page to look at this application's services and traffic.

In the Google Kubernetes Engine console, go to the Service Mesh page.

Go to the Service Mesh page

The page displays the List view and Topology view side-by-side, which shows all your project's microservices, including system services. It might take some time for data on the Service Mesh page to populate. If you see a partial or no diagram in the topology view, try restarting the Pods in the cluster, or you can move along to the next section and check back later.

Use the list view

Each row in the list is one of the services that makes up the Bank of Anthos application. For example, the frontend service renders the application's web user interface, and the userservice service manages user accounts and authentication.

Each service listing shows up-to-date metrics, such as Server error rate and Requests per second, for that service. These metrics are collected out-of-the-box for services deployed on GKE Enterprise. You do not need to write any application code to see these statistics.

You can drill down from this view to see even more details about each service. For example, to learn more about the transactionhistory service:

  1. Click transactionhistory in the services list. The service details page shows all the telemetry available for this service.

  2. On the transactionhistory page, select Connected Services from the menu on the left. Here you can see both the Inbound and Outbound connections for the service. An unlocked lock icon indicates that some traffic has been observed on this port that is not encrypted using mutual TLS (mTLS). You can find out more about how this works in the Secure GKE Enterprise tutorial.

    Screenshot of Anthos Service Mesh Connected Services view

Use the topology view

The topology view lets you focus on how the services interact. As you can see from the legend, the graph shows the application's Anthos Service Mesh services, Istio services, Deployments and Pods. If you want to expand this view, click Toggle panel "Left Panel" on the List View.

Screenshot of Anthos Service Mesh topology view

Anthos Service Mesh automatically observes which services are communicating with each other to show service-to-service connections details:

  • Hold your mouse pointer over an item to see additional details, including outbound QPS from each service.

  • Drag nodes with your mouse to improve your view of particular parts of the graph.

  • Click service nodes for more service information.

  • Click Expand when you hold the pointer over a workload node to drill down for even more details, including the number of instances of a workload that are currently running.

Exploring GKE Enterprise further

While this tutorial has shown you many GKE Enterprise features, there's still lots more to see and do with our deployment. Visit our follow-up tutorial to try some hands-on tasks with GKE Enterprise, or continue to explore the sample yourself before cleaning up.

Troubleshoot

Refer to the following troubleshooting scenarios if you run into problems deploying the Bank of Anthos application.

Google Cloud APIs not enabled (code 403)

You might see errors similar to the following:

Error: Error creating Feature: failed to create a diff: failed to retrieve Feature resource: googleapi:
Error 403: GKE Hub API has not been used in project {project-number} before or it is disabled.
Enable it by visiting https://console.developers.google.com/apis/api/gkehub.googleapis.com/overview?project={project-number} then retry.
If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

As the error suggests, wait for a few minutes for an API to be fully enabled and run the deployment again using terraform apply.

Service Mesh page not displaying all services

After deploying the Bank of Anthos application, you might need to wait for a few minutes for the Service Mesh page to display your project's microservices. If the Service Mesh page doesn't show some or all of the services even after a few minutes, it means that some of the proxies for Anthos Service Mesh might have failed to automatically start along with the application workloads.

To fix this issue, restart the Pods in the anthos-sample-cluster1 cluster:

  • Get credentials for the cluster:

    gcloud container clusters get-credentials anthos-sample-cluster1 --zone us-central1
    
  • Delete the Pods:

    kubectl delete pod -n default --all
    
  • Check that the Pods have restarted:

    kubectl get pod -n default
    

The Service Mesh page should be populated with your project's microservices within a few minutes.

Clean up

After you've finished exploring the Bank of Anthos application, you can clean up the resources that you created on Google Cloud so they don't take up quota and you aren't billed for them in the future.

  • Option 1. You can delete the project. However, if you want to keep the project around, you can use Option 2 to delete the deployment.

  • Option 2. If you want to keep your current project, you can use terraform destroy to delete the sample application and cluster.

Delete the project (option 1)

The easiest way to avoid billing is to delete the project you created for this tutorial.

    Delete a Google Cloud project:

    gcloud projects delete PROJECT_ID

Delete the deployment (option 2)

This approach deletes the Bank of Anthos application and the cluster, but does not delete the project. Run the following commands on your Cloud Shell:

  1. Change to the directory that hosts the installation scripts:

    cd bank-of-anthos/iac/tf-anthos-gke
    
  2. Delete the sample and the cluster:

    terraform destroy
    
  3. Enter the project ID when prompted.

If you plan to redeploy, verify that all requirements are met as described in the Before you begin section.

What's next