Searching for Data Objects

Preview

This feature is subject to the "Pre-GA Offerings Terms" in the General Service Terms section of the Service Specific Terms. Pre-GA features are available "as is" and might have limited support. For more information, see the launch stage descriptions.

The purpose of the Search API is to find Data Objects that are similar to a given query and return a list of ranked results (ranked by similarity). The Search API also supports filtering.

The Search API offers a variety of ways to search for Data Objects: vector search, full text search, and semantic search. Additionally, multiple searches of any type can be combined together to achieve hybrid search.

Vector search

Vector search lets you provide your own query vector. This is the required method for searching embedding fields that don't have an embedding-config. If multiple search_vector fields are provided, results are combined using equal weights.

The following example demonstrates how to perform a vector search on a collection with the ID movies.

curl -X POST \
'https://vectorsearch.googleapis.com/v1alpha/projects/PROJECT_ID/locations/LOCATION/collections/movies/dataObjects:search' \
  -H 'Bearer $(gcloud auth print-access-token)' \
  -H 'Content-Type: application/json' \
  -d '{ \
    "vector_search": { \
      "search_field": "plot_embedding", \
      "vector": { \
        "values": [ \
          0.42426406871192845, \
          0.565685424949238, \
          0.7071067811865475 \
        ] \
      }, \
      "filter": { \
        "genre": { \
          "$eq": "Thriller" \
        } \
      }, \
      "top_k": 5, \
      "output_fields": { \
        "data_fields": "*", \
        "vector_fields": "*", \
        "metadata_fields": "*" \
      } \
    } \
  }'

Text search

This performs full-text search without sparse vectors. The default "word" query dialect treats the entire input as individual search terms with an implicit AND operator. You can set enhanced_query to true to expand search terms, handle stemming, stop word removal, and allow for additional search operators:

• OR: A case-sensitive disjunction operator that matches documents containing at least one of the specified terms. It only applies to the two adjacent terms.

• ": (double quotes) for phrase search.

• -: The negation operator. It excludes documents containing any terms it is placed before.

Semantic search

This search converts your text query into embeddings to find results based on semantic meaning. It uses the embedding-config defined in your schema to generate the query embedding. If multiple search_vector fields are provided, results are combined using equal weights.

Note: Only vector fields that are managed by an auto-embedding specification can be used in a semantic search.

Searching using hybrid search

Use batch_search_data_objects to execute multiple searches in parallel (vector search, text search, and semantic search) and optionally combine and rank the results using a Ranker.

The following Rankers are available:

• ReciprocalRankFusion: Merges result sets using the Reciprocal Rank Fusion (RFF) algorithm.

• VertexRanker: Uses the Vertex Ranking API to merge and rank results.