查詢公開索引以取得最近鄰

建立並部署索引後,即可執行查詢來取得最鄰近的項目。

以下提供比對查詢範例,說明如何使用 K 近鄰演算法 (k-NN) 找出最鄰近的項目。

公開端點的查詢範例

Python

如要瞭解如何安裝或更新 Python 適用的 Vertex AI SDK,請參閱「安裝 Python 適用的 Vertex AI SDK」。 詳情請參閱 Python API 參考說明文件

def vector_search_find_neighbors(
    project: str,
    location: str,
    index_endpoint_name: str,
    deployed_index_id: str,
    queries: List[List[float]],
    num_neighbors: int,
) -> List[
    List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]
]:
    """Query the vector search index.

    Args:
        project (str): Required. Project ID
        location (str): Required. The region name
        index_endpoint_name (str): Required. Index endpoint to run the query
        against.
        deployed_index_id (str): Required. The ID of the DeployedIndex to run
        the queries against.
        queries (List[List[float]]): Required. A list of queries. Each query is
        a list of floats, representing a single embedding.
        num_neighbors (int): Required. The number of neighbors to return.

    Returns:
        List[List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]] - A list of nearest neighbors for each query.
    """
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)

    # Create the index endpoint instance from an existing endpoint.
    my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_name
    )

    # Query the index endpoint for the nearest neighbors.
    return my_index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=queries,
        num_neighbors=num_neighbors,
    )

指令列

您可以在「Deploy」中找到下列 publicEndpointDomainName,格式為 <number>.<region>-<number>.vdb.vertexai.goog


  $ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer `gcloud auth print-access-token`" https://1957880287.us-central1-181224308459.vdb.vertexai.goog/v1/projects/181224308459/locations/us-central1/indexEndpoints/3370566089086861312:findNeighbors -d '{deployed_index_id: "test_index_public1", queries: [{datapoint: {datapoint_id: "0", feature_vector: [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]}, neighbor_count: 5}]}'
  

這個 curl 範例示範如何從 http(s) 用戶端呼叫,但公開端點支援 restful 和 grpc_cli 的雙重通訊協定。


  $ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer `gcloud auth print-access-token`" https://1957880287.us-central1-181224308459.vdb.vertexai.goog/v1/projects/${PROJECT_ID}/locations/us-central1/indexEndpoints/${INDEX_ENDPOINT_ID}:readIndexDatapoints -d '{deployed_index_id:"test_index_public1", ids: ["606431", "896688"]}'
  

這個 curl 範例示範如何使用權杖和數值限制進行查詢。


  $ curl -X POST -H "Content-Type: application/json" -H "Authorization: Bearer `gcloud auth print-access-token`"  https://${PUBLIC_ENDPOINT_DOMAIN}/v1/projects/${PROJECT_ID}/locations/${LOCATION}/indexEndpoints/${INDEX_ENDPOINT_ID}:findNeighbors -d '{deployed_index_id:"${DEPLOYED_INDEX_ID}", queries: [{datapoint: {datapoint_id:"x", feature_vector: [1, 1], "sparse_embedding": {"values": [111.0,111.1,111.2], "dimensions": [10,20,30]}, numeric_restricts: [{namespace: "int-ns", value_int: -2, op: "GREATER"}, {namespace: "int-ns", value_int: 4, op: "LESS_EQUAL"}, {namespace: "int-ns", value_int: 0, op: "NOT_EQUAL"}], restricts: [{namespace: "color", allow_list: ["red"]}]}}]}'
  

控制台

按照這些操作說明,從控制台查詢部署至公開端點的索引。

  1. 在 Google Cloud 控制台的 Vertex AI 專區中,前往「Deploy and Use」(部署及使用) 專區。選取「Vector Search」

    前往 Vector Search

  2. 選取要查詢的索引。「Index info」(索引資訊) 頁面隨即開啟。
  3. 向下捲動至「已部署的索引」部分,然後選取要查詢的已部署索引。「已部署的索引資訊」頁面隨即開啟。
  4. 在「查詢索引」部分,選取要依密集嵌入值、稀疏嵌入值、混合嵌入值 (密集和稀疏嵌入) 或特定資料點查詢。
  5. 輸入所選查詢類型的查詢參數。舉例來說,如果您要透過稠密型嵌入查詢,請輸入要查詢的嵌入向量。
  6. 使用提供的 curl 指令執行查詢,或透過 Cloud Shell 執行查詢。
  7. 如果使用 Cloud Shell,請選取「在 Cloud Shell 中執行」
  8. 在 Cloud Shell 中執行。
  9. 結果會傳回最鄰近的項目。

混合查詢

混合型搜尋會同時使用稠密和稀疏嵌入,根據關鍵字搜尋和語意搜尋的組合進行搜尋。

Python

如要瞭解如何安裝或更新 Python 適用的 Vertex AI SDK,請參閱「安裝 Python 適用的 Vertex AI SDK」。 詳情請參閱 Python API 參考說明文件

def vector_search_find_neighbors_hybrid_queries(
    project: str,
    location: str,
    index_endpoint_name: str,
    deployed_index_id: str,
    num_neighbors: int,
) -> List[
    List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]
]:
    """Query the vector search index using example hybrid queries.

    Args:
        project (str): Required. Project ID
        location (str): Required. The region name
        index_endpoint_name (str): Required. Index endpoint to run the query
        against.
        deployed_index_id (str): Required. The ID of the DeployedIndex to run
        the queries against.
        num_neighbors (int): Required. The number of neighbors to return.

    Returns:
        List[List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]] - A list of nearest neighbors for each query.
    """
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)

    # Create the index endpoint instance from an existing endpoint.
    my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_name
    )

    # Query hybrid datapoints, sparse-only datapoints, and dense-only datapoints.
    hybrid_queries = [
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            dense_embedding=[1, 2, 3],
            sparse_embedding_dimensions=[10, 20, 30],
            sparse_embedding_values=[1.0, 1.0, 1.0],
            rrf_ranking_alpha=0.5,
        ),
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            dense_embedding=[1, 2, 3],
            sparse_embedding_dimensions=[10, 20, 30],
            sparse_embedding_values=[0.1, 0.2, 0.3],
        ),
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            sparse_embedding_dimensions=[10, 20, 30],
            sparse_embedding_values=[0.1, 0.2, 0.3],
        ),
        aiplatform.matching_engine.matching_engine_index_endpoint.HybridQuery(
            dense_embedding=[1, 2, 3]
        ),
    ]

    return my_index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=hybrid_queries,
        num_neighbors=num_neighbors,
    )

使用篩選和擁擠度條件的查詢

篩選向量比對結果可將近鄰結果限制在特定類別。篩選器也可以指定要從結果中排除的類別。

每個擁擠鄰近項目的限制可限制從索引資料中任何單一擁擠標記傳回的結果數量,進而增加結果多樣性。

Python

如要瞭解如何安裝或更新 Python 適用的 Vertex AI SDK,請參閱「安裝 Python 適用的 Vertex AI SDK」。 詳情請參閱 Python API 參考說明文件

def vector_search_find_neighbors_filtering_crowding(
    project: str,
    location: str,
    index_endpoint_name: str,
    deployed_index_id: str,
    queries: List[List[float]],
    num_neighbors: int,
    filter: List[aiplatform.matching_engine.matching_engine_index_endpoint.Namespace],
    numeric_filter: List[
        aiplatform.matching_engine.matching_engine_index_endpoint.NumericNamespace
    ],
    per_crowding_attribute_neighbor_count: int,
) -> List[
    List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]
]:
    """Query the vector search index with filtering and crowding.

    Args:
        project (str): Required. Project ID
        location (str): Required. The region name
        index_endpoint_name (str): Required. Index endpoint to run the query
        against.
        deployed_index_id (str): Required. The ID of the DeployedIndex to run
        the queries against.
        queries (List[List[float]]): Required. A list of queries. Each query is
        a list of floats, representing a single embedding.
        num_neighbors (int): Required. The number of neighbors to return.
        filter (List[Namespace]): Required. A list of Namespaces for filtering
        the matching results. For example,
        [Namespace("color", ["red"], []), Namespace("shape", [], ["square"])]
        will match datapoints that satisfy "red color" but not include
        datapoints with "square shape".
        numeric_filter (List[NumericNamespace]): Required. A list of
        NumericNamespaces for filtering the matching results. For example,
        [NumericNamespace(name="cost", value_int=5, op="GREATER")] will limit
        the matching results to datapoints with cost greater than 5.
        per_crowding_attribute_neighbor_count (int): Required. The maximum
        number of returned matches with the same crowding tag.

    Returns:
        List[List[aiplatform.matching_engine.matching_engine_index_endpoint.MatchNeighbor]] - A list of nearest neighbors for each query.
    """
    # Initialize the Vertex AI client
    aiplatform.init(project=project, location=location)

    # Create the index endpoint instance from an existing endpoint.
    my_index_endpoint = aiplatform.MatchingEngineIndexEndpoint(
        index_endpoint_name=index_endpoint_name
    )

    # Query the index endpoint for the nearest neighbors.
    return my_index_endpoint.find_neighbors(
        deployed_index_id=deployed_index_id,
        queries=queries,
        num_neighbors=num_neighbors,
        filter=filter,
        numeric_filter=numeric_filter,
        per_crowding_attribute_neighbor_count=per_crowding_attribute_neighbor_count,
    )

影響成效的查詢時間設定

使用向量搜尋時,下列查詢時間參數可能會影響延遲時間、可用性和費用。這項指引適用於大多數情況。不過,請務必實驗各種設定,確保這些設定符合您的用途。

如需參數定義,請參閱「索引設定參數」。

參數 關於 效能影響
approximateNeighborsCount

告知演算法要從每個分片擷取的近似結果數量。

approximateNeighborsCount 的值一律應大於 setNeighborsCount 的值。如果 setNeighborsCount 的值很小,建議將 approximateNeighborsCount 設為該值的 10 倍。如果 setNeighborsCount 值較大,可以使用較小的乘數。

這個欄位對應的 REST API 名稱為 approximate_neighbor_count

增加 approximateNeighborsCount 的值可能會對效能造成以下影響:

  • 喚回度:提升
  • 延遲時間:可能增加
  • 可用性:沒有影響
  • 費用:可能會增加,因為搜尋期間處理的資料量較多

降低 approximateNeighborsCount 的值可能會對效能造成以下影響:

  • 喚回度:降低
  • 延遲:可能減少
  • 可用性:沒有影響
  • 成本:可減少搜尋期間處理的資料量,進而降低成本
setNeighborCount

指定查詢要傳回的結果數量。

這個欄位對應的 REST API 名稱為 neighbor_count

在大多數用途中,值小於或等於 300 仍可維持效能。 如為較大的值,請針對特定用途進行測試。

fractionLeafNodesToSearch 控制搜尋最鄰近項目時要造訪的分葉節點百分比。這與 leafNodeEmbeddingCount 相關,因為每個分葉節點的嵌入越多,每個分葉檢查的資料就越多。

這個欄位對應的 REST API 名稱為 fraction_leaf_nodes_to_search_override

提高 fractionLeafNodesToSearch 的值可能會對效能造成以下影響:

  • 喚回度:提升
  • 延遲時間:增加
  • 可用性:沒有影響
  • 費用:可能會增加,因為延遲時間越長,佔用的機器資源就越多

降低 fractionLeafNodesToSearch 的值可能會對效能造成以下影響:

  • 喚回度:降低
  • 延遲時間:縮短
  • 可用性:沒有影響
  • 費用:可降低,因為延遲時間越短,佔用的機器資源就越少