Retrieval and generation output of Vertex AI RAG Engine

This page describes the fields in the output from Vertex AI RAG Engine. The output contains the following main parts:

  • retrieveContexts: Contains information about the text chunks retrieved from your source data.
  • generateContent: Contains the generated response and metadata that shows how the response is grounded in the retrieved text chunks.

retrieveContexts

The retrieveContexts response contains the following fields.

Fields

Field name Description
source_uri The URI of the original source file.
  • If you import the file from Cloud Storage or Google Drive, this field contains the original file URI.
  • If you upload the file, this field contains the file's display name.
source_display_name The file's display name.
text The text chunk that is relevant to the query.
score A score that represents the similarity between the query and the text chunk. The metric used depends on the vectorDB that you choose. For ragManagedDB, the score is the COSINE_DISTANCE.

Sample output

The following sample shows the format of the retrieveContexts response:

contexts {
    source_uri: "gs://sample_folder/hello_world.txt"
    source_display_name: "hello_world.txt"
    text: "Hello World!"
    score: 0.60545359030757784
  }

generateContent

Most of the fields defined for the generateContent API can found in the Response body.

Fields

The grounding_metadata object contains the following fields:

  • text: The response generated by Gemini.
  • grounding_chunks: A list of text chunks returned by Vertex AI RAG Engine that are relevant to the query. Each chunk object contains the following field:
    • retrieved_context: An object containing the text chunk used to ground the generated content. It contains the following fields:
      • uri: The source_uri of the original data.
      • title: The source_display_name of the original file.
      • text: The text chunk used to ground the Gemini response.
  • grounding_supports: A list that describes the relationship between segments of the generated response and the grounding chunks. Each entry contains the following fields:
    • segment: An object that describes a segment of the generated response that is grounded in the source data. It contains the following fields:
      • start_index: The starting character index of the grounded text segment. If this field is omitted, the index is 0.
      • end_index: The ending character index of the grounded text segment.
      • text: The text of the grounded segment.
    • grounding_chunk_indices: A list of indices that point to the chunks in grounding_chunks used to ground the text segment. The index starts at 0. A segment can be grounded by more than one chunk.
    • confidence_scores: A list of confidence scores. Each score indicates how strongly the text segment is grounded on a corresponding chunk in grounding_chunk_indices. The maximum score is 1.0. Only chunks with a confidence score of 0.6 or higher are included.

Sample output

The following sample shows the format of the generateContent response, focusing on the grounding_metadata:

candidates {
  content {
    role: "model"
    parts {
      text: "The rectangle is red and the background is white. The rectangle appears to be on some type of document editing software. \n"
    }
  }
  grounding_metadata {
    grounding_chunks {
      retrieved_context {
        uri: "a.txt"
        title: "a.txt"
        text: "Okay ,  I  see  a  red  rectangle  on  a  white  background .  It looks like it\'s on some sort of document editing software. It has those small squares and circles around it, indicating that it\'s a selected object ."
      }
    }
    grounding_chunks {
      retrieved_context {
        uri: "b.txt"
        title: "b.txt"
        text: "The  video  is  identical  to  the  last  time  I  described  it .  It shows a blue rectangle on a white background."
      }
    }
    grounding_chunks {
      retrieved_context {
        uri: "c.txt"
        title: "c.txt"
        text: "Okay ,  I  remember  the  rectangle  was  blue  in  the  past  session . Now it is red.\n The  red  rectangle  is  still  there .  It \' s  still  in  the same position on the white background, with the same handles around it. Nothing new is visible since last time.\n You \' re  welcome .  The  red  rectangle  is  still  the  only  thing visible."
      }
    }
    grounding_supports {
      segment {
        end_index: 49
        text: "The rectangle is red and the background is white."
      }
      grounding_chunk_indices: 2
      grounding_chunk_indices: 0
      confidence_scores: 0.958192229
      confidence_scores: 0.992316723
    }
    grounding_supports {
      segment {
        start_index: 50
        end_index: 120
        text: "The rectangle appears to be on some type of document editing software."
      }
      grounding_chunk_indices: 0
      confidence_scores: 0.98374176
    }
  }
}

What's next