Generative playbooks

Note: This feature has limited access. Contact your Google Cloud account team to request access for this feature and to access the API reference documentation.

Generative playbooks provide a new way for creating Dialogflow CX agents using large language models (LLM). Rather than defining flows, pages, intents, and transitions; you provide natural language instructions and structured data in the form of playbooks. This can significantly reduce the agent creation and maintenance time, and enable brand new types of conversational experiences for your business.

If you still require the explicit control provided by flows in certain conversational scenarios, you can combine the power of playbooks and flows in a single hybrid agent.

Note: Vertex AI's text-bison is the current model used by generative playbooks.

Limitations

The following limitations apply:

• Only English is supported.
• Only the us-central1 and global regions are supported.
• Playbook agents don't support sending a call companion SMS from the Default Welcome Intent route in the Default Start Flow, but you can enable the call companion SMS option in standard flows.

Playbook overview

A playbook is the basic building block of a playbook agent. An agent typically has many playbooks, where each playbook is defined to handle specific tasks. The playbook data is provided to the LLM, so it has the information it needs to answer questions and execute tasks. Each playbook can provide information, send queries to external services, or defer conversation handling to a traditional flow or another playbook to handle sub-tasks.

Create playbook agents

When creating a Dialogflow agent, you can choose to have the agent start a conversation with either a playbook (playbook agent) or a flow (traditional agent).

To create a playbook agent:

1. Follow the steps to create an agent, selecting Build your own.
2. Select Generative as the agent type.
3. Click Save.

Notice that there are three new resource selectors in the left navigation. These selectors let you choose between:

• Flow resources
• Generative resources
• Shared resources

When you create a playbook agent, a default playbook is automatically created.

Playbook data

This section describes the data that defines a playbook.

Playbook name

The Playbook name is the display name for the playbook. Playbooks may reference each other by this name.

Playbook goal

A playbook goal is a high level description of what the playbook should accomplish.

For example:

Help customers to book flights and hotels.

Playbook steps

Playbook steps define the process that should be taken to accomplish the playbook goal.

Each step contains a natural language instruction that may contain any of the following:

• A basic instruction that the LLM can understand.
• An instruction to route the user to another playbook. Playbooks are referenced using the form ${PLAYBOOK: playbook_name}.
• An instruction to use a specific tool. Tools are referenced using the form ${TOOL: tool_name}.
• An instruction to route the user to a Dialogflow flow. Flows are referenced using the form ${FLOW: flow_name}.

Each step description starts with "- ", and you can define substeps using indentation.

For example:

- greet the customer and ask them how you can help.
    - If the customer wants to book flights, route them to ${PLAYBOOK: flight_booking}.
    - If the customer wants to book hotels, route them to  ${PLAYBOOK: hotel_booking}.
    - If the customer wants to know trending attractions, use the ${TOOL: attraction_tool} to show them the list.
- help the customer to pay for their booking by routing them to ${FLOW: make_payment}.

Note: When editing steps, you may see a warning indicating that a resource reference has no corresponding examples. You can still save and use the playbook with this warning present, but it's unlikely to work reliably until examples that use the resource are added.

Playbook parameters

Playbooks can accept and emit context information using explicitly defined parameters. Parameters are defined per-playbook using the Parameters tab once you've created a playbook.

Playbook parameters have a type, name, and description. After you define parameters, use them in playbook examples in order to show the playbook how to read, write, and use parameter values reliably. See playbook examples input and output and passing parameters for guidance.

Playbook input parameters

Input parameters allow playbooks to use values passed from flows and other playbooks. For example, a playbook might receive a user's preferred name as a parameter and use it to thank the user personally, or a playbook might receive an order identifier as a parameter and use it to retrieve order details using a tool.

Playbook input parameters are defined per-playbook, and playbooks don't have visibility of other Dialogflow CX parameter types by default. When a flow transitions to a playbook, page and session parameters are propagated to the playbook if the target playbook has an input parameter with the same name. To communicate information from a flow to a playbook during a transition, define playbook input parameters with the same name as a session or page parameter present before the transition.

Note: The default playbook may not accept input parameters. The default playbook serves as an entrypoint for user conversations, so it has no way to receive input parameters.

Create examples to control how the input parameter value should impact playbook actions. For example, if an input parameter should affect how the agent refers to the user, create examples that define a value for the parameter and then use the same value in utterance actions within the example. See passing parameters for details.

Playbook output parameters

Output parameters allow playbooks to emit information to be used by other flows or playbooks. For example, a playbook might collect an order number from a user and emit it through an output parameter, or a playbook might use a tool to book a flight and emit the confirmation number through an output parameter.

Create examples to control how the playbook should decide the value for each output parameter. For example, if an output parameter representing a confirmation number should derive its value from the output of a tool use, create examples where the output of the tool use matches the value of the playbook output parameter.

Passing parameters

Playbooks, unlike flows, do not support injecting parameter values with a particular syntax. Instead, playbooks rely on instructions and few-shot prompt examples to determine how parameter values should be used and how values should be decided when specifying parameter values.

Consider an agent designed for event ticket sales with the following playbooks:

1. A playbook named Ticket ordering which places orders using a tool named Ticket sales API.
   1. This playbook accepts an input parameter with type number and name event_id.
   2. The Ticket sales API tool expects a request including an event_id.
2. A playbook named Event selection which helps users select an event and then routes them to Ticket ordering with parameter event_id to purchase tickets.

In this example, to ensure that event_id is reliably passed from Event selection to Ticket ordering and from Ticket ordering to Ticket sales API, several examples are needed.

The Ticket ordering playbook should include multiple examples which:

• Have input parameter event_id specified with some realistic value, different in each example.
• Include a tool use action with a request body that includes the same realistic event_id value as specified in the input parameter.

The Event selection playbook should include multiple examples which:

• Include a user utterance where the user selects an event with some realistic event_id, different in each example.
• Include a playbook invocation of Ticket ordering which sets the event_id parameter to the same realistic event_id as decided by the user's selection.

In addition to adding examples, try adding specific instructions to the playbook steps, playbook goal, or tool details explaining how parameters should be used. For example, playbook Ticket ordering includes the following step:

- Use parameter event_id to send a buy_tickets request with ${TOOL: Ticket sales API}

With the described examples and instructions present, the Event selection Playbook correctly decides an event_id based on the user's selection, and passes it as an input parameter named event_id to the Ticket ordering playbook. Then, Ticket ordering passes the same event_id in the body of a request to the Ticket sales API. Playbooks depend on examples with distinct parameter values to help them infer how parameters should be used.

Playbook examples

Each playbook should have one or more examples. These examples are sample conversations between an end-user and the agent, including the dialogue and actions performed by the agent. These are effectively few-shot prompt examples for the LLM.

The console provides an interface for you to enter actions. For example:

Example input summary and output summary

In addition to input and output parameters, playbooks support receiving an input summary and emitting an output summary for exchanging information with other playbooks. Summaries are helpful for passing abstract contextual information between playbooks, while parameters are more helpful for passing structured, well defined fields between playbooks. Parameters are the only way to exchange data between flows and playbooks.

Add relevant input summaries to examples to condition the playbook to adjust its actions based on the input summaries at runtime. Add output summaries including relevant, accurate details about the example conversation to show the playbook what details are important to summarize.

Note: Examples for the default playbook may not define input parameters or an input summary. The default playbook serves as an entrypoint for user conversations, so it has no way to receive input parameters.

Example input and output parameters

In addition to an input summary and output summary, if a playbook defines input or output parameters, the playbook's examples should all also define those input or output parameters to help the language model infer how to use the parameter values reliably.

Each example for the playbook should specify and use values for each parameter defined in the playbook, as shown in this playbook with two input parameters and one output parameter:

Example playbook output state

For each example you create, select the playbook state that best represents the end state of the example conversation:

• OK: the playbook's goal was achieved.
• CANCELLED: the playbook stopped without achieving its goal due to circumstances in the conversation. For example, this state may be appropriate if the conversation includes a user changing their mind about what they'd like help with.
• FAILED: the playbook stopped without achieving its goal due to an error or unhandleable circumstance. For example, this state may be appropriate if the goal couldn't be reached due to network issues during a tool invocation.
• ESCALATED: the playbook stopped without achieving its goal because the user requested escalation to a human agent.

Retrieval strategy

Retrieval strategy controls whether or not each example is included in the Playbook's prompt.

• DEFAULT: the example may be omitted if the prompt nears the token limit.
• STATIC: the example is always included.
• NEVER: the example is never included in the prompt. The example will have no effect whatsoever on the playbook's performance.

Note: Retrieval strategy 'Never' may be useful for temporarily excluding an example for testing to understand its effects on the playbook's performance.

Create a playbook

To create a playbook:

1. Select the generative resources on the left navigation.
2. Click Playbooks.
3. Click Create new.
4. Provide data as described above.

Default playbook

When you create a generative agent, a default playbook is created automatically.

The default playbook is the starting point for conversations, so it has some important distinctions from other playbooks:

• The default playbook doesn't receive a summary of preceding conversation turns.
• The default playbook can't define or receive input parameters.

Playbook versions

You can save playbook versions, which are immutable snapshots of playbooks.

To save a playbook version:

1. Load the playbook in the console.
2. Click Version history.
3. Click Create version.
4. Provide a version name and click Save.

To view version history:

1. Load the playbook in the console.
2. Click Version history.
3. Click View version history.
4. The version history panel opens on the right side. You can click each version to see its content.

Tools

Playbook steps can reference tools that should be used to accomplish the step. For each tool used by playbooks, you provide the details for the tool by providing an OpenAPI schema.

HTTP API calls or data store queries are currently supported.

You can provide a session ID as a path or query parameter. For example:

parameters:
  - name: petId
    in: path
    description: ID of pet that needs to be updated
    required: true
    schema:
      $ref: '@dialogflow/sessionId'
  - name: petName
    in: query
    description: ID of pet that needs to be updated
    required: true
    schema:
      $ref: '@dialogflow/sessionId'

The following limitations apply to HTTP API calls:

• Except for session ID, query parameters are not supported.
• Request and response body must be empty or JSON.
• Advanced schema features like oneOf are not supported.

The following example shows how to reference a data store:

"dataStoreConnections": [
    {
       "dataStoreType": "DATA_STORE_TYPE",
       "dataStore": "projects/PROJECT_ID/locations/LOCATION_ID/collections/default_collection/dataStores/DATA_STORE_ID"
    }
]

The DATA_STORE_TYPE value can be one of the following:

• PUBLIC_WEB: A data store that contains public web content.
• UNSTRUCTURED: A data store that contains unstructured private data.
• STRUCTURED: A data store that contains structured data (for example FAQ).

The following example shows how to reference an HTTP API tool:

openapi: 3.0.2
info:
  title: Search Attraction Tool
  description: >-
    This API search for attractions for travel purposes
  version: 1.0
servers:
  - url: https://search-attraction.app
paths:
  /search:
    post:
      summary: Search for attractions given a query
      operationId: search
      requestBody:
        description: Query
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Query'
      responses:
        '200':
          description: Successfully got results (may be empty)
          content:
            application/json:
              schema:
                type: object
                properties:
                  results:
                    type: array
                    items:
                      type: string
components:
  schemas:
    Query:
      required:
        - text
      type: object
      properties:
        text:
          type: string

Best practices

The following best practices can help you build robust agents.

At least one example for each playbook

You should have at least one example for each playbook. Without enough examples, a playbook is likely to result in unpredictable behavior. If your agent is not responding or behaving in the manner you expect, missing or poorly defined examples are likely the cause. Try improving your examples or adding new ones.

Precision of instructions and examples

While it helps to write clear and descriptive instructional steps, it's really the quality and quantity of your examples that determine the accuracy of the agent's behavior. In other words, spend more time writing thorough examples than writing perfectly precise instructions.

Tool schema operationId field

When defining schemas for your tools, the operationId value is important. Your playbook instructions will reference this value. The following are naming recommendations for this field:

• Letters, numbers and underscores only.
• Must be unique among all operationIds described in the schema.
• Must be a meaningful name reflecting the functionality provided.

Tool schema validation

You should validate your tool schema. You can use the Swagger Editor to check your openAPI 3.0 schema syntax.

Generate a schema with Bard

Bard can generate a schema for you. For example, try "can you create an example openAPI 3.0 schema for Google Calendar".

Focused playbooks

Avoid creating very large and complex playbooks. Each playbook should accomplish a specific and clear task. If you have a complex playbook, consider breaking it down into smaller sub-playbooks.

Test cases

The existing test case feature has been enhanced to support playbooks.

A required expected test case result field has been added, which provides a list of actions in the form of playbook examples. The test case verifies that the actions were performed as expected.

When viewing a playbook or playbook version, you can click the Test cases tab above the playbook data to see test case results and to execute a test case on demand.

You can also compare test case results for different versions of a playbook. Select a test case, then click Compare.

To see all test cases for the agent, click Test cases in the left navigation.

Conversation history

The existing conversation history feature has been enhanced to support playbooks.

Information has been added for tool execution, playbook invocation, and flow invocation from a playbook agent.