This document describes the benefit of instrumenting your generative AI applications. It also summarizes how you can instrument an application that uses the LangGraph or Agent Development Kit (ADK) frameworks. Both frameworks let you collect text prompts. However, the ADK framework also lets you collect and view multimodal prompts and responses.
About generative AI agents
Applications that use generative AI rely on agents to complete tasks or achieve goals. Agents are applications that complete tasks or pursue goals on behalf of a user. For example, a generative AI agent can navigate a website and issue API commands. These API commands might retrieve information or perform an action.
Agents act autonomously and they use reasoning to determine how to decompose a goal or task into subtasks, and then to determine which tools to use to accomplish those subtasks. In the context of generative AI, tools determine how the agent can interact with its environment. For example, an agent might have a tool that lets it issue API commands.
To learn more about agents, see the following:
Why instrument your generative AI applications
Instrumenting your generative AI applications is the only way you can understand the reasoning used by the autonomous agents. This reasoning process isn't deterministic.
When you instrument a generative AI application, the telemetry data includes the decisions and actions of the agents. You can use this data to validate or improve your application.
How to instrument your generative AI applications
Instrumenting your application involves generating telemetry and sending it to where the data can be stored and queried and analyzed. For example, when your instrumentation sends telemetry to your Google Cloud project, you can use Google Cloud Observability to view and analyze that data.
We recommend that you instrument your application with OpenTelemetry, which is an open source project that provides a unified framework for application instrumentation. The examples we provide rely on OpenTelemetry.
For instrumentation information, see the following documents:
Collect and view multimodal prompts and responses describes how to configure a generative AI agent built with ADK to collect multimodal prompts and responses. It also describes how to do the following:
- View prompts and responses with the Trace Explorer page.
- Analyze prompts and responses with BigQuery.
- Evaluate prompts and responses with the Vertex AI SDK for Python.
Instrument a LangGraph ReAct Agent with OpenTelemetry describes how to use OpenTelemetry to instrument a generative AI application that uses the LangChain or LangGraph framework. It also links to a sample application that you can load and run.
Instrument ADK applications with OpenTelemetry describes how to enable built-in instrumentation that collects telemetry from the agent's key actions. The telemetry includes text prompts and agent responses. It also links to a sample application that you can load and run.
How to view your prompts and responses
Cloud Trace is configured to extract events from spans written by generative AI applications, provided the spans conform to the OpenTelemetry Semantic conventions for generative AI spans and include attributes or events that follow the Semantic conventions for generative AI events.
Cloud Trace can also display multimodal prompts and responses. In this scenario, your prompts and responses aren't attached to your trace data. Instead, they are stored in a Cloud Storage bucket. To learn more about this configuration, see Collect and view multimodal prompts and responses.