View app latency

View app latency

Learn how to use Cloud Trace by doing the following:

1. Deploy a sample application to a Google Kubernetes Engine (GKE) cluster.

2. Create a trace by sending an HTTP request to the sample application.

3. Use the Cloud Trace interface to view the latency information of the trace you created.

Before you begin

Security constraints defined by your organization might prevent you from completing the following steps. For troubleshooting information, see Develop applications in a constrained Google Cloud environment.

Create a project with billing enabled:

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

5. Make sure that billing is enabled for your Google Cloud project.

Download and deploy your application

Note: When you execute this quickstart, a service account with the extension @developer.gserviceaccount.com is created. The service account is granted the Editor (roles/editor) role unless your project is in an organization with a policy that prohibits this action. If the quickstart fails, try granting the service account the Cloud Trace Agent (roles/cloudtrace.agent) role. For instructions, see Grant a single role.

To download and deploy the application, do the following:

1. To open the Cloud Shell, click Activate Cloud Shell in the Google Cloud console toolbar:

   

   After a few moments, a Cloud Shell session opens inside the Google Cloud console.

2. To download the source code from GitHub, run the following command:

   git clone https://github.com/GoogleCloudPlatform/python-docs-samples.git
   

3. Enable the Google Kubernetes Engine API by using the following command:

   gcloud services enable container.googleapis.com
   

4. To create the GKE cluster named cloud-trace-demo in the us-central1-c zone, run the following commands in the Cloud Shell:

   ZONE=us-central1-c
   gcloud container clusters create cloud-trace-demo \
      --zone $ZONE
   

   This command takes a few minutes to complete. After it completes successfully, your Google Cloud project contains the GKE cluster named cloud-trace-demo. You must have permission to create clusters that have external access in your Google Cloud project.

5. Update your GKE cluster credentials by running the following command:

   gcloud container clusters get-credentials cloud-trace-demo --zone $ZONE
   

6. Verify access to the cluster by running the following command:

   kubectl get nodes
   

   A sample output of this command is:

   NAME                                              STATUS   ROLES    AGE   VERSION
   gke-cloud-trace-demo-default-pool-063c0416-113s   Ready    <none>   78s   v1.22.12-gke.2300
   gke-cloud-trace-demo-default-pool-063c0416-1n27   Ready    <none>   79s   v1.22.12-gke.2300
   gke-cloud-trace-demo-default-pool-063c0416-frkd   Ready    <none>   78s   v1.22.12-gke.2300
   

7. Deploy the sample application by running the following command:

   cd python-docs-samples/trace/cloud-trace-demo-app-opentelemetry && ./setup.sh
   

   The script setup.sh configures three services of the application using a pre-built image. The workloads are named cloud-trace-demo-a, cloud-trace-demo-b, and cloud-trace-demo-c. The setup script waits for all resources to be provisioned, so the configuration might take several minutes to complete.

   A sample output of this command is:

   deployment.apps/cloud-trace-demo-a is created
   service/cloud-trace-demo-a is created
   deployment.apps/cloud-trace-demo-b is created
   service/cloud-trace-demo-b is created
   deployment.apps/cloud-trace-demo-c is created
   service/cloud-trace-demo-c is created
   
   Wait for load balancer initialization complete......
   Completed. You can access the demo at http://34.82.132.95/
   

Create a trace

A trace describes the time it takes an application to complete a single operation. Each trace consists of one or more spans. A span describes how long it takes to perform a complete sub-operation. For example, a trace might describe how long it takes to process an incoming request from a user and return a response. A span might describe how long a particular RPC call requires. For more information, see Cloud Trace's Data model.

To create a trace by sending a curl request to cloud-trace-demo-a, use the following command:

curl $(kubectl get svc -o=jsonpath='{.items[?(@.metadata.name=="cloud-trace-demo-a")].status.loadBalancer.ingress[0].ip}')

You can execute the curl command multiple times to generate multiple traces.

The output looks like the following:

Hello, I am service A
And I am service B
Hello, I am service C

View the trace data

In the Google Cloud console, select Trace, and then select  Overview, or click the following button:

Go to Trace Overview

Overview window

The Overview window is the default view in Trace. This window displays latency data and summary information, including an analysis report. If you created a new project, the most interesting pane of the Overview window is the pane labeled Recent traces:

This pane lists the most recent traces and their latency. To view the details of a trace, click its link.

Trace explorer window

In the Trace navigation pane, click Trace explorer:

This window displays a graph and a table. Each dot on the graph represents a trace. Each dot also corresponds to a row in the table. In the previous screenshot, multiple traces are listed, indicating that the curl command was executed multiple times.

To view a trace in detail, select a dot in the graph or a row in the table:

After you select a dot, the scatter plot is refreshed and the dot you selected is highlighted with a circle drawn around the dot. The dots that represent all other traces are dimmed. The Trace details pane displays information about the trace. The first row of the table is for the trace, and there exists one row for each span in the trace. To view details about a span, click the latency bar:

Analysis Reports window

Trace automatically creates daily reports. For this project, there isn't enough data to create a new report.

About the application

The sample application used in this quickstart is available in a GitHub repository. This repository contains information on how to use the application in environments other than the Cloud Shell. The sample application is written in Python, uses the Flask framework and OpenTelemetry packages, and executes on a Google Kubernetes Engine cluster.

Instrumentation

The file app.py in the GitHub repository, contains the instrumentation necessary to capture and send trace data to your Google Cloud project:

• The application imports several OpenTelemetry packages:

  View on GitHub

  from opentelemetry import trace
  from opentelemetry.exporter.cloud_trace import CloudTraceSpanExporter
  from opentelemetry.instrumentation.flask import FlaskInstrumentor
  from opentelemetry.instrumentation.requests import RequestsInstrumentor
  from opentelemetry.propagate import set_global_textmap
  from opentelemetry.propagators.cloud_trace_propagator import CloudTraceFormatPropagator
  from opentelemetry.sdk.trace import TracerProvider
  from opentelemetry.sdk.trace.export import BatchSpanProcessor
  

• The application instruments web requests with trace context and automatically traces Flask handlers and requests to other services:

  View on GitHub

  app = flask.Flask(__name__)
  FlaskInstrumentor().instrument_app(app)
  RequestsInstrumentor().instrument()

• The application configures the Cloud Trace exporter as a trace provider, which propagates trace context in the Cloud Trace format:

  View on GitHub

  def configure_exporter(exporter):
      """Configures OpenTelemetry context propagation to use Cloud Trace context
  
      Args:
          exporter: exporter instance to be configured in the OpenTelemetry tracer provider
      """
      set_global_textmap(CloudTraceFormatPropagator())
      tracer_provider = TracerProvider()
      tracer_provider.add_span_processor(BatchSpanProcessor(exporter))
      trace.set_tracer_provider(tracer_provider)
  
  
  configure_exporter(CloudTraceSpanExporter())
  tracer = trace.get_tracer(__name__)

• The following code snippet shows how to send requests in Python. OpenTelemetry implicitly propagates the trace context for you with your outgoing requests:

  View on GitHub

  if endpoint is not None and endpoint != "":
      data = {"body": keyword}
      response = requests.get(
          endpoint,
          params=data,
      )
      return keyword + "\n" + response.text
  else:
      return keyword, 200
  
  

How the application works

For clarity, in this section, cloud-trace-demo is omitted from the service names. For example, the service cloud-trace-demo-c is referenced as c.

This application creates three services named a, b, and c. Service a is configured to call service b, service b is configured to call service c. For details on the configuration of the services, see the YAML files in the GitHub repository.

When you issued a HTTP request to service a in this quickstart, you used the following curl command:

curl $(kubectl get svc -o=jsonpath='{.items[?(@.metadata.name=="cloud-trace-demo-a")].status.loadBalancer.ingress[0].ip}')

The curl command works as follows:

1. kubectl fetches the IP address of the service named cloud-trace-demo-a.
2. The curl command then sends the HTTP request to service a.
3. Service a receives the HTTP request and sends a request to service b.
4. Service b receives the HTTP request and sends a request to service c.
5. Service c receives the HTTP request from service b and returns the string Hello, I am service C to service b.
6. Service b receives the response from service c, appends it to the string And I am service B, and returns the result to service a.
7. Service a receives the response from service b and appends it to the string Hello, I am service A.
8. The response from service a is printed in the Cloud Shell.

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, follow these steps.

• If you created a new Google Cloud project for this quickstart, then delete the project to stop accruing charges. To delete your project, do the following:

  1. In the Google Cloud console, click Navigation Menu, and select Home.
  2. In the Project info pane, click Go to project settings.
  3. In the Settings window, click Shut down, and complete the remaining steps.

• If you didn't create a new Google Cloud project for this quickstart, then delete the Google Kubernetes Engine cluster named cloud-trace-demo by running the following command:

  gcloud container clusters delete cloud-trace-demo --zone $ZONE
  

What's next

• For information on languages and platforms supported, see Cloud Trace overview.
• For details on how to instrument your applications, see Instrument for Cloud Trace.
• For more information on the Overview window, see View traces overview.
• For more information on the Trace explorer window, see Find and view traces.
• For information on the waterfall graph and trace details, see View a trace.
• For information on Analysis Reports, see Create and view reports.
• To learn more about managing Google Kubernetes Engine clusters, see kubectl