Method: catalog.search

Searches Data Catalog for multiple resources like entries, tags that match a query.

This is a custom method (https://cloud.google.com/apis/design/custom_methods) and does not return the complete resource, only the resource identifier and high level fields. Clients can subsequentally call Get methods.

Note that Data Catalog search queries do not guarantee full recall. Query results that match your query may not be returned, even in subsequent result pages. Also note that results returned (and not returned) can vary across repeated search queries.

See Data Catalog Search Syntax for more information.

HTTP request

POST https://datacatalog.googleapis.com/v1beta1/catalog:search

Request body

The request body contains data with the following structure:

JSON representation
{
  "corpus": string,
  "scope": {
    object (Scope)
  },
  "query": string,
  "additionalQuery": string,
  "pageSize": integer,
  "pageToken": string,
  "orderBy": string,
  "resultsView": [
    enum (ResultsView)
  ],
  "labels": {
    string: string,
    ...
  }
}
Fields
corpus

string

scope

object (Scope)

Required. The scope of this search request. A scope that has empty includeOrgIds, includeProjectIds AND false includeGcpPublicDatasets is considered invalid. Data Catalog will return an error in such a case.

query

string

Optional. The query string in search query syntax. An empty query string will result in all data assets (in the specified scope) that the user has access to. Query strings can be simple as "x" or more qualified as:

  • name:x
  • column:x
  • description:y

Note: Query tokens need to have a minimum of 3 characters for substring matching to work correctly. See Data Catalog Search Syntax for more information.

additionalQuery

string

Extra filtering condition to apply on top of user provided query string. Clients should use this field to specify additional criteria to search for. For example, if a query should only return BigQuery tables, clients can specify "type:table.bigquery".

pageSize

integer

Number of results in the search page. If <=0 then defaults to 10. Max limit for pageSize is 1000. Throws an invalid argument for pageSize > 1000.

pageToken

string

Optional. Pagination token returned in an earlier SearchCatalogResponse.next_page_token, which indicates that this is a continuation of a prior SearchCatalogRequest call, and that the system should return the next page of data. If empty, the first page is returned.

orderBy

string

Specifies the ordering of results, currently supported case-sensitive choices are:

  • relevance, only supports descending
  • last_modified_timestamp [asc|desc], defaults to descending if not specified

If not specified, defaults to relevance descending.

resultsView[]

enum (ResultsView)

Describes various options on search. Example: Whether to return snippets or not.

labels

map (key: string, value: string)

The labels on this Search API call. These labels will be passed into the billing system, and costs generated by the search API call will have these labels.

Response body

If successful, the response body contains data with the following structure:

Response message for catalog.search.

JSON representation
{
  "results": [
    {
      object (SearchCatalogResult)
    }
  ],
  "totalSize": integer,
  "nextPageToken": string,
  "clientRequestId": string,
  "unreachable": [
    string
  ]
}
Fields
results[]

object (SearchCatalogResult)

Search results.

totalSize

integer

The (approximate) total number of entries matched for the query.

nextPageToken

string

The token that can be used to retrieve the next page of results.

clientRequestId

string

The UUID (the hex format of the random uint64 value) for the client to log the user actions. The same ID will also be used by the external client to report back the user action on the search results.

unreachable[]

string

Unreachable locations. Search result does not include data from those locations. Users can get additional information on the error by repeating the search request with a more restrictive parameter -- setting the value for SearchDataCatalogRequest.scope.restricted_locations.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

Scope

The criteria that select the subspace used for query matching.

JSON representation
{
  "includeOrgIds": [
    string
  ],
  "includeProjectIds": [
    string
  ],
  "includeGcpPublicDatasets": boolean,
  "restrictedLocations": [
    string
  ],
  "includeLocations": [
    string
  ]
}
Fields
includeOrgIds[]

string

The list of organization IDs to search within. To find your organization ID, follow instructions in https://cloud.google.com/resource-manager/docs/creating-managing-organization.

includeProjectIds[]

string

The list of project IDs to search within. To learn more about the distinction between project names/IDs/numbers, go to https://cloud.google.com/docs/overview/#projects.

includeGcpPublicDatasets

boolean

If true, include Google Cloud Platform (GCP) public datasets in the search results. Info on GCP public datasets is available at https://cloud.google.com/public-datasets/. By default, GCP public datasets are excluded.

restrictedLocations[]

string

Optional. The list of locations to search within. 1. If empty, search will be performed in all locations; 2. If any of the locations are NOT in the valid locations list, error will be returned; 3. Otherwise, search only the given locations for matching results. Typical usage is to leave this field empty. When a location is unreachable as returned in the SearchCatalogResponse.unreachable field, users can repeat the search request with this parameter set to get additional information on the error.

Valid locations: * asia-east1 * asia-east2 * asia-northeast1 * asia-northeast2 * asia-northeast3 * asia-south1 * asia-southeast1 * australia-southeast1 * eu * europe-north1 * europe-west1 * europe-west2 * europe-west3 * europe-west4 * europe-west6 * global * northamerica-northeast1 * southamerica-east1 * us * us-central1 * us-east1 * us-east4 * us-west1 * us-west2

includeLocations[]
(deprecated)

string

Optional. This field is deprecated. Use restrictedLocations instead.

ResultsView

Represents additional details to include in search results. Example: STATS include aggregate-level statistics about search results.

Enums
RESULTS_VIEW_UNSPECIFIED View unspecified.
SNIPPETS Returns snippets alongside results.
STATS

Deprecated. Returns aggregate-level statistics about search results.

SearchCatalogResult

A result that appears in the response of a search request. Each result captures details of one entry that matches the search.

JSON representation
{
  "searchResultType": enum (SearchResultType),
  "searchResultSubtype": string,
  "relativeResourceName": string,
  "linkedResource": string,
  "attributeSections": {
    string: {
      object (AttributeSection)
    },
    ...
  },
  "modifyTime": string,

  // Union field system can be only one of the following:
  "integratedSystem": enum (IntegratedSystem),
  "userSpecifiedSystem": string
  // End of list of possible types for union field system.
}
Fields
searchResultType

enum (SearchResultType)

Type of the search result. This field can be used to determine which Get method to call to fetch the full resource.

searchResultSubtype

string

Sub-type of the search result. This is a dot-delimited description of the resource's full type, and is the same as the value callers would provide in the "type" search facet. Examples: entry.table, entry.dataStream, tagTemplate.

relativeResourceName

string

The relative resource name of the resource in URL format. Examples:

  • projects/{project_id}/locations/{locationId}/entryGroups/{entryGroupId}/entries/{entryId}
  • projects/{project_id}/tagTemplates/{tagTemplateId}
linkedResource

string

The full name of the cloud resource the entry belongs to. See: https://cloud.google.com/apis/design/resource_names#full_resource_name. Example:

  • //bigquery.googleapis.com/projects/projectId/datasets/datasetId/tables/tableId
attributeSections

map (key: string, value: object (AttributeSection))

Attribute sections for the search result. It captures detailed information such as snippets, values for different attributes for the search result. List of attributes currently returned for each matching result are title, description, and shardCount for sharded tables. They are populated based on the corresponding metadata of the given entry. If the search term also matches within the attribute value, the snippet field will be populated to highlight the matching part.

modifyTime

string (Timestamp format)

Last-modified timestamp of the entry from the managing system.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

Union field system. The source system of the entry. Only applicable when search_result_type is ENTRY. system can be only one of the following:
integratedSystem

enum (IntegratedSystem)

Output only. This field indicates the entry's source system that Data Catalog integrates with, such as BigQuery or Cloud Pub/Sub.

userSpecifiedSystem

string

This field indicates the entry's source system that Data Catalog does not integrate with.

SearchResultType

The different types of resources that can be returned in search.

Enums
SEARCH_RESULT_TYPE_UNSPECIFIED Default unknown type.
ENTRY An Entry.
TAG_TEMPLATE A TagTemplate.
ENTRY_GROUP An EntryGroup.