Use Data Catalog to search for model and dataset resources

This page explains how you can search for and manage your Vertex AI model and dataset resources by using Data Catalog. You can access all your managed resources across different projects using Dataplex's Data Catalog feature.

Data Catalog is a fully managed, scalable metadata management service within Dataplex. The service automatically catalogs the following metadata about Vertex AI models and datasets:

• Name and fully-qualified name of model and dataset resources,
• Location (region),
• Creation date and last modification date,
• Model Alias and Model Version ID Dataset data item count,
• Description.

Vertex AI model and dataset metadata is automatically synced to Data Catalog at regular intervals, typically every few hours. You can use Data Catalog to discover and understand your Vertex AI metadata. Use Data Catalog to aid with the following activities:

• Search for models with specific model alias, and model name across all projects and locations that you have access to. For example, to search for a specific model alias use model_version_alias=my-model-alias.
• Search for datasets using dataset name, dataset ID across all projects and locations that you have access to.

With Data Catalog, you can curate metadata by attaching tags to Vertex AI resources. Each tag can have multiple metadata fields, and can be based on a predefined or custom tag template.

For example, you could attach the following tag to a column that contains deployment environment name:

deployed_to:Test

To learn more about Data Catalog see Data Catalog overview

Note: Data Catalog refers to the resources in Vertex AI and in other Cloud services as assets. From now on we'll refer to Vertex AI resources as Vertex AI assets. The term assets is used to refer to both the resources and Data Catalog metadata for the resources.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

5. Make sure that billing is enabled for your Google Cloud project.

6. Enable the Data Catalog API for the project.

   Enable the Data Catalog API

7. Check permissions.

   You need certain Identity and Access Management (IAM) roles and permissions to search for and attach tags to Vertex AI assets. For details, see Required IAM roles and permissions for Data Catalog.

Create tag templates

Tag temples are reusable structures that you can use to rapidly create new tags. Templates help you avoid duplication of work and incomplete tags. Create as many tag templates as you need.

To learn more, see Tags and tag templates.

Attach tags to Vertex AI assets

Attaching tags to Vertex AI assets lets you do the following:

• Add business metadata to the assets.
• Search for assets by business metadata and other custom metadata.

To learn more, see Tags and tag templates.

Search for Vertex AI assets

Use the Dataplex search page in the Google Cloud console to search for Vertex AI assets.

1. Go to the Dataplex search page.

   Go to Dataplex

2. In the Filters panel, under Systems, select Vertex AI.

   Dataplex displays all known Vertex AI assets.

3. Optional: Do one of the following to narrow down the search:

   • Use the faceted search on the Search page. Select checkboxes under Data types, Projects, and Tags.

   • In the search field, append a search parameter after system=vertex_ai. Separate parameters with spaces.

     For example, to view only models, enter the following text in the search field, and then press Enter.

     system=vertex_ai type=model
     

     You can also use parentheses and the logical operators and and or for complex expressions. To learn more about the expressions that you can use in the search field, see Data Catalog search syntax.

4. In the results table, click the name of an asset to view the metadata for that asset.

5. Optional: Do any of the following:

   • Click ADD OVERVIEW to add a rich text description of the asset.
   • Click ATTACH TAGS to add a tag to the asset.

Required IAM roles and permissions for Data Catalog

Viewing, tagging, and searching Vertex AI resources in Data Catalog have different permission requirements.

Permission to view public and private tags

To get the permissions that you need to view public and private tags on Vertex AI resources, ask your administrator to grant you the following IAM roles:

• roles/datacatalog.tagTemplateViewer
• roles/aiplatform.viewer

Permission to search Vertex AI resources

To get the permissions that you need to search Vertex AI resources in Data Catalog, ask your administrator to grant you the following IAM role:

• roles/aiplatform.user

Permission to attach tags to Vertex AI resources

To get the permissions that you need to attach tags to Vertex AI resources in Data Catalog, ask your administrator to grant you the following IAM roles:

• roles/datacatalog.tagTemplateUser
• roles/datacatalog.tagEditor
• roles/aiplatform.admin

For more information, see Predefined roles.

What's next

• What is Data Catalog
• Roles to search Google Cloud resources