Chat with the assistant

In your app, your users can chat about search results and uploaded content with the assistant.

The assistant can provide summaries and answer questions through natural language conversations with end users, and generate images.

After you get search results from the app, the assistant automatically provides a summary based on your search results. You can ask follow-up questions to the assistant about these results and get answers based on the data that your app searches across.

Chat about uploaded content

The assistant can analyze files that you upload, such as PDFs, images, and videos. You can ask the assistant follow-up questions based on the content and previous answers, and get suggested follow-up questions.

Chat about uploaded content limitations

The following limitations apply to uploading content to the assistant:

  • File size limit: 30MB
  • File token limit: 700,000 tokens

Chat about uploaded content in the app

To chat about uploaded content in your app, do the following:

  1. Go to your search app.
  2. Click the Upload icon in the search bar.
  3. Upload one or more files to the app.

  4. Enter a prompt or query in the Ask a follow-up search bar.

    The assistant generates an answer and the app displays related search results.

Upload content using the API

To upload content to the assistant using the API, do the following:

Run the following curl command:

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token" \
  -H "Content-Type: application/json" \
  -H "X-Goog-User-Project: PROJECT_ID" \
"https://discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/global/collections/default_collection/engines/APP_ID/sessions/-:addContextFile" \
-d '{
      "name": "projects/PROJECT_ID/locations/global/collections/default_collection/engines/APP_ID/sessions/-",
      "fileName": "FILE_NAME",
      "mimeType": "MIME_TYPE",
      "fileContents": "BASE64_ENCODED_FILE",
      "allowWithoutFile": true
    }'

Replace the following:

  • PROJECT_ID: the ID of your Google Cloud project.
  • APP_ID: the ID of the app.
  • FILE_NAME: the name of the file.
  • MIME_TYPE: the MIME type of the file.
  • BASE64_ENCODED_FILE: The base64 representation (ASCII string) of your data. This string should look similar to the following string: /9j/4QAYRXhpZgAA...9tAVx/zDQDlGxn//2Q==

Example command and response:

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token" \
  -H "Content-Type: application/json" \
  -H "X-Goog-User-Project: 1234" \
"https://discoveryengine.googleapis.com/v1alpha/projects/1234/locations/global/collections/default_collection/engines/myapp/sessions/-:addContextFile" \
-d '{
      "name": "projects/1234/locations/global/collections/default_collection/engines/myapp/sessions/-",
      "fileName": "Hello World.pdf",
      "mimeType": "application/pdf",
      "fileContents": JVBERi0xLjQKJdPr6eEKMSAwIG9iago8PC9UaXRsZSAoSGVsbG8gd29ybGQpCi9Qcm9kdWNlciAoU2tpYS9QREYgbTEzMCBHb29nbGUgRG9jcyBSZW5kZXJlcik+PgplbmRvYmoKMyAwIG9iago8PC..............KMDAwMDAxNDk0NCAwMDAwMCBuIAp0cmFpbGVyCjw8L1NpemUgMTIKL1Jvb3QgNyAwIFIKL0luZm8gMSAwIFI+PgpzdGFydHhyZWYKMTU0NjYKJSVFT0YK",
      "allowWithoutFile": true
    }'
------------------
{
  "session": "projects/1234/locations/global/collections/default_collection/engines/myapp/sessions/11157546467692424024",
  "fileId": "11157546467692423460",
  "tokenCount": "258"
}

Generate images using the assistant

The assistant can generate images based on text prompts, allowing end users to create custom visuals like artwork, UI mockups, and charts. All generated images include a digital watermark.

Users can specify details such as image style and aspect ratio in the prompt. After generating the initial image, users can upload existing images or enter follow-up text prompts to refine their generated image. Finally, users can export the finished image in PNG format.

Example prompts for image generation:

  • Generate a high-fidelity mockup of a mobile banking app with a clean, minimalist design, displaying a user's account balance and recent transactions.
  • Generate a surreal image of a giant clock melting in a desert landscape, symbolizing the passage of time.
  • Generate a photorealistic image of a group of people laughing and enjoying a cup of coffee together in a modern cafe.

Manage chats in the app

In the app, users can view and manage their recent and pinned chats that they had with the assistant.

To access and manage previous chats, do the following:

  1. Open the app
  2. Optional. If your chats are hidden, click the Menu icon.

  3. Under Recent, hover over the chat you want to manage, click the vertical ellipsis, and choose one of the following features:

    • Pin: Keep frequently used chats at the top of your list.
    • Rename: Customize chat names to make them easier to identify.
    • Delete: Remove chats from your pinned and recent lists after they're no longer needed.