Rank and rerank documents

Note: This feature is a Preview offering, subject to the "Pre-GA Offerings Terms" of the GCP Service Specific Terms. Pre-GA products and features may have limited support, and changes to pre-GA products and features may not be compatible with other pre-GA versions. For more information, see the launch stage descriptions. Further, by using this feature, you agree to the Generative AI Preview terms and conditions ("Preview Terms"). For this feature, you can process personal data as outlined in the Cloud Data Processing Addendum, subject to applicable restrictions and obligations in the Agreement (as defined in the Preview Terms).

This page describes how to use the ranking API to rank a set of documents based on a query.

The ranking API takes a list of documents and reranks those documents based on how relevant the documents are to a query. Compared to embeddings, which look only at the semantic similarity of a document and a query, the ranking API can give you precise scores for how well a document answers a given query. The ranking API can be used to improve the quality of search results after retrieving an initial set of candidate documents.

The ranking API is stateless so there's no need to index documents before calling the API. All you need to do is pass in the query and documents. This makes the API well suited for reranking documents from Vector Search and other search solutions.

Use cases

The primary use case of the ranking API is to improve the quality of search results.

However, the ranking API can be valuable for any scenario where you need to find what pieces of content are most relevant to a user's query. For example, the ranking API can assist you in the following:

• Finding the right content to give to an LLM for grounding

• Improving the relevance of an existing search experience

• Identifying relevant sections of a document

The following flow outlines how you might use the ranking API to improve the quality of results for chunked documents:

1. Use Document AI Layout Parser API to split a set of documents into chunks.

2. Use an embeddings API to create embeddings for each of the chunks.

3. Load the embeddings into Vector Search or another search solution.

4. Query your search index and retrieve the most relevant chunks.

5. Rerank the relevant chunks using the ranking API.

Input data

Key Term: The ranking API uses the term record to indicate a document. A record is made up of an ID, a title, and the content of a document. Unlike the documents that are contained in Vertex AI Search data stores, the records input to the ranking API are in JSON format and have not been indexed by Vertex AI Search.

The ranking API requires the following inputs:

• The query for which you're ranking the records.

  For example:

  "query": "Why is the sky blue?"
  

• A set of records that are relevant to the query. The records are provided as an array of objects. Each record can include a unique ID, a title, and the content of the document. For each record include either a title, content, or both. If the length of the title and content together exceed 512 tokens, the additional content is truncated. You can include up to 200 records per request.

  For example, a record array looks something like this. In reality, many more records would be included in the array and the content would be much longer:

  "records": [
     {
         "id": "1",
         "title": "The Color of the Sky: A Poem",
         "content": "A canvas stretched across the day,\nWhere sunlight learns to dance and play.\nBlue, a hue of scattered light,\nA gentle whisper, soft and bright."
     },
     {
         "id": "2",
         "title": "The Science of a Blue Sky",
         "content": "The sky appears blue due to a phenomenon called Rayleigh scattering. Sunlight is comprised of all the colors of the rainbow. Blue light has shorter wavelengths than other colors, and is thus scattered more easily."
     }
  ]
  

• Optional: The maximum number of records that you want the ranking API to return. By default, all records are returned; however, you can use thetop_n field to return fewer records. All records are ranked regardless of what value is set.

  For example, this returns the top 10 ranked records:

  "top_n": 10
  

• Optional: A setting that specifies whether you want just the ID of the record returned by the API or if you want the record title and content returned as well. By default, the full record is returned. The main reason to set this is if you want to reduce the size of the response payload.

  For example, setting to true returns only the record ID, not the title or content:

  "ignore_record_details_in_response": true,
  

• Optional: The model name. The model is semantic-ranker-512@latest. This model is used whether or not the model name is set.

  "model": "semantic-ranker-512@latest"
  

Output data

The ranking API returns a ranked list of records with following outputs:

• Score: a float value between 0 and 1 that indicates relevance of the record.

• ID: the unique ID of the record.

• If requested, the full object: the ID, title, and content.

  For example:

  {
   "records": [
       {
           "id": "2",
           "score": 0.98,
           "title": "The Science of a Blue Sky",
           "content": "The sky appears blue due to a phenomenon called Rayleigh scattering. Sunlight is comprised of all the colors of the rainbow. Blue light has shorter wavelengths than other colors, and is thus scattered more easily."
       },
       {
           "id": "1",
           "score": 0.64,
           "title": "The Color of the Sky: A Poem",
           "content": "A canvas stretched across the day,\nWhere sunlight learns to dance and play.\nBlue, a hue of scattered light,\nA gentle whisper, soft and bright."
       }
   ]
  }
  

Rank (or rerank) a set of records according to a query

Typically, you'll supply the ranking API with a query and a set of records that are relevant to that query and have already been ranked by some other method such as a keyword search or a vector search. Then, you use the ranking API to improve the quality of the ranking and determine a score that indicates the relevance of each record to the query.

1. Obtain the query and resulting records. Ensure that each record has an ID and either a title, content, or both.

   The model supports up to 512 tokens per record. If the combined length of the title and content is more than 512 tokens, the extra content is truncated.

2. Run the following curl command:

   curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
   -H "Content-Type: application/json" \
   -H "X-Goog-User-Project: PROJECT_ID" \
   "https://discoveryengine.googleapis.com/v1alpha/projects/PROJECT_ID/locations/global/rankingConfigs/default_ranking_config:rank" \
   -d '{
   "model": "semantic-ranker-512@latest",
   "query": "QUERY",
   "records": [
       {
           "id": "RECORD_ID_1",
           "title": "TITLE_1",
           "content": "CONTENT_1"
       },
       {
           "id": "RECORD_ID_2",
           "title": "TITLE_2",
           "content": "CONTENT_2"
       },
       {
           "id": "RECORD_ID_3",
           "title": "TITLE_3",
           "content": "CONTENT_3"
       }
   ]
   }'
   

   Replace the following:

   • PROJECT_ID: the ID of your Google Cloud project..
   • QUERY: the query against which the records are ranked and scored.
   • RECORD_ID_n: a unique string that identifies the record.
   • TITLE_n: the title of the record.
   • CONTENT_n: the content of the record.
   Note: You can test the API by typing your own values into these placeholders and pasting the command into Cloud Shell. However, in reality, the records array would be larger.

   For general information about this method, see rankingConfigs.rank.

   Click for an example curl command and response.

   curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
   -H "Content-Type: application/json" \
   curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
   -H "Content-Type: application/json" \
   -H "X-Goog-User-Project: my-project-123" \
   "https://discoveryengine.googleapis.com/v1alpha/projects/my-project-123/locations/global/rankingConfigs/default_ranking_config:rank" \
   -d '{
   {
   "model": "semantic-ranker-512@latest",
   "query": "what is Google gemini?",
   "records": [
       {
           "id": "1",
           "title": "Gemini",
           "content": "The Gemini zodiac symbol often depicts two figures standing side-by-side."
       },
       {
           "id": "2",
           "title": "Gemini",
           "content": "Gemini is a cutting edge large language model created by Google."
       },
       {
           "id": "3",
           "title": "Gemini Constellation",
           "content": "Gemini is a constellation that can be seen in the night sky."
       }
   ]
   }
   {
   "records": [
       {
           "id": "2",
           "title": "Gemini",
           "content": "Gemini is a cutting edge large language model created by Google.",
           "score": 0.97
       },
       {
           "id": "3",
           "title": "Gemini Constellation",
           "content": "Gemini is a constellation that can be seen in the night sky.",
           "score": 0.18
       },
       {
           "id": "1",
           "title": "Gemini",
           "content": "The Gemini zodiac symbol often depicts two figures standing side-by-side.",
           "score": 0.05
       }
   ]
   }