[[["わかりやすい","easyToUnderstand","thumb-up"],["問題の解決に役立った","solvedMyProblem","thumb-up"],["その他","otherUp","thumb-up"]],[["わかりにくい","hardToUnderstand","thumb-down"],["情報またはサンプルコードが不正確","incorrectInformationOrSampleCode","thumb-down"],["必要な情報 / サンプルがない","missingTheInformationSamplesINeed","thumb-down"],["翻訳に関する問題","translationIssue","thumb-down"],["その他","otherDown","thumb-down"]],["最終更新日 2025-08-17 UTC。"],[],[],null,["# Manage pipelines\n================\n\nThis document describes how to manage\n[BigQuery pipelines](/bigquery/docs/pipelines-introduction),\nincluding how to schedule and delete pipelines.\n\nThis document also describes how to view and manage pipeline metadata in\n[Dataplex Universal Catalog](/dataplex/docs/introduction).\n\nPipelines are powered by [Dataform](/dataform/docs/overview).\n\nBefore you begin\n----------------\n\n1. [Create a BigQuery pipeline](/bigquery/docs/create-pipelines).\n2. To manage pipeline metadata in Dataplex Universal Catalog, ensure that the [Dataplex API](/dataplex/docs/enable-api) is enabled in your Google Cloud project.\n\n### Required roles\n\n\nTo get the permissions that\nyou need to manage pipelines,\n\nask your administrator to grant you the\nfollowing IAM roles:\n\n- To delete pipelines: [Dataform Admin](/iam/docs/roles-permissions/dataform#dataform.Admin) (`roles/dataform.Admin`) on the pipeline\n- To view and run pipelines: [Dataform Viewer](/iam/docs/roles-permissions/dataform#dataform.Viewer) (`roles/dataform.Viewer`) on the project\n\n\nFor more information about granting roles, see [Manage access to projects, folders, and organizations](/iam/docs/granting-changing-revoking-access).\n\n\nYou might also be able to get\nthe required permissions through [custom\nroles](/iam/docs/creating-custom-roles) or other [predefined\nroles](/iam/docs/roles-overview#predefined).\n\nTo manage pipeline metadata in Dataplex Universal Catalog,\nensure that you have the required\n[Dataplex Universal Catalog roles](/dataplex/docs/iam-roles)\n\nFor more information about Dataform IAM, see\n[Control access with IAM](/dataform/docs/access-control).\n| **Note:** When you create a pipeline, BigQuery grants you the Dataform Admin ([`roles/dataform.admin`](/iam/docs/understanding-roles#dataform.admin)) role on that pipeline. All users with the [Dataform Admin (`roles/dataform.admin`) role](/dataform/docs/access-control#dataform.admin) granted on the Google Cloud project have owner access to all pipelines created in the project.\n\nView all pipelines\n------------------\n\nTo view a list of all pipelines in your project, do the following:\n\n1. In the Google Cloud console, go to the\n **BigQuery** page.\n\n [Go to BigQuery](https://console.cloud.google.com/bigquery)\n2. In the **Explorer** pane,\n click more_vert\n expand **Pipelines**.\n\nView past manual runs\n---------------------\n\nTo view past manual runs of a selected pipeline, follow these steps:\n\n1. In the Google Cloud console, go to the\n **BigQuery** page.\n\n [Go to BigQuery](https://console.cloud.google.com/bigquery)\n2. In the **Explorer** pane, expand your project and the **Pipelines** folder,\n and then select a pipeline.\n\n3. Click **Executions**.\n\n4. Optional: To refresh the list of past runs, click **Refresh**.\n\nConfigure alerts for failed pipeline runs\n-----------------------------------------\n\nEach pipeline has a corresponding Dataform repository ID.\nEach BigQuery pipeline run is logged in\n[Cloud Logging](/logging/docs) using the corresponding\nDataform repository ID. You can use Cloud Monitoring to observe\ntrends in Cloud Logging logs for\nBigQuery pipeline runs and to notify\nyou when conditions you describe occur.\n\nTo receive alerts when a BigQuery pipeline run fails,\nyou can create a log-based alerting policy for the corresponding\nDataform repository ID. For instructions, see\n[Configure alerts for failed workflow invocations](/dataform/docs/monitor-runs#configure-alerts-failed-workflow-invocations).\n\nTo find the Dataform repository ID of your pipeline, do the following:\n\n1. In the Google Cloud console, go to the\n **BigQuery** page.\n\n [Go to BigQuery](https://console.cloud.google.com/bigquery)\n2. In the **Explorer** pane, expand your project and the **Pipelines** folder,\n and then select a pipeline.\n\n3. Click **Settings**.\n\n The Dataform repository ID of your pipeline is displayed at\n the bottom of the **Settings** tab.\n\nDelete a pipeline\n-----------------\n\nTo permanently delete a pipeline, follow these steps:\n\n1. In the Google Cloud console, go to the\n **BigQuery** page.\n\n [Go to BigQuery](https://console.cloud.google.com/bigquery)\n2. In the **Explorer** pane, expand your project and the **Pipelines** folder.\n Find the pipeline that you want to delete.\n\n3. Click more_vert\n **View actions** next to the pipeline, and then click **Delete**.\n\n4. Click **Delete**.\n\nManage metadata in Dataplex Universal Catalog\n---------------------------------------------\n\nDataplex Universal Catalog lets you store and manage metadata for\npipelines. Pipelines are available in Dataplex Universal Catalog\nby default, without additional configuration.\n\nYou can use Dataplex Universal Catalog to manage pipelines\nin all [pipeline locations](/bigquery/docs/locations).\nManaging pipelines in Dataplex Universal Catalog\nis subject to [Dataplex Universal Catalog quotas and limits](/dataplex/docs/quotas)\nand [Dataplex Universal Catalog pricing](/dataplex/pricing).\n\nDataplex Universal Catalog automatically retrieves\nthe following metadata from pipelines:\n\n- Data asset name\n- Data asset parent\n- Data asset location\n- Data asset type\n- Corresponding Google Cloud project\n\nDataplex Universal Catalog logs pipelines as\n[entries](/dataplex/docs/ingest-custom-sources#entries) with the following\nentry values:\n\nSystem entry group\n: The [system entry group](/dataplex/docs/ingest-custom-sources#entry-groups)\n for pipelines is `@dataform`. To view details of pipeline entries\n in Dataplex Universal Catalog, you need to view the `dataform` system entry group.\n For instructions about how to view a list of all entries in an entry group, see\n [View details of an entry group](/dataplex/docs/ingest-custom-sources#entry-group-details)\n in the Dataplex Universal Catalog documentation.\n\nSystem entry type\n: The [system entry type](/dataplex/docs/ingest-custom-sources#entry-types)\n for pipelines is `dataform-code-asset`. To view details of\n pipelines,you need to view the `dataform-code-asset` system entry type,\n filter the results with an aspect-based filter,\n and [set the `type` field inside `dataform-code-asset` aspect to `WORKFLOW`](/dataplex/docs/search-syntax#aspect-search).\n Then, select an entry of the selected pipeline.\n For instructions about how to view details of a selected entry type, see\n [View details of an entry type](/dataplex/docs/ingest-custom-sources#entry-type-details)\n in the Dataplex Universal Catalog documentation.\n For instructions about how to view details of a selected entry, see\n [View details of an entry](/dataplex/docs/search-assets#view-entry-details)\n in the Dataplex Universal Catalog documentation.\n\nSystem aspect type\n: The [system aspect type](/dataplex/docs/enrich-entries-metadata#aspect-types)\n for pipelines is `dataform-code-asset`. To\n provide additional context to pipelines in Dataplex Universal Catalog\n by annotating data pipeline entries with\n [aspects](/dataplex/docs/enrich-entries-metadata#aspects),\n view the `dataform-code-asset` aspect type,\n filter the results with an aspect-based filter,\n and [set the `type` field inside `dataform-code-asset` aspect to `WORKFLOW`](/dataplex/docs/search-syntax#aspect-search).\n For instructions about how to annotate entries with aspects, see\n [Manage aspects and enrich metadata](/dataplex/docs/enrich-entries-metadata)\n in the Dataplex Universal Catalog documentation.\n\nType\n: The type for data canvases is `WORKFLOW`.\n This type lets you filter pipelines in the `dataform-code-asset`\n system entry type and the `dataform-code-asset` aspect type by using the\n `aspect:dataplex-types.global.dataform-code-asset.type=WORKFLOW`\n query in an [aspect-based filter](/dataplex/docs/search-syntax#aspect-search).\n\nFor instructions about how to search for assets in Dataplex Universal Catalog, see\n[Search for data assets in Dataplex Universal Catalog](/dataplex/docs/search-assets)\nin the Dataplex Universal Catalog documentation.\n\nWhat's next\n-----------\n\n- Learn more about [BigQuery pipelines](/bigquery/docs/pipelines-introduction).\n- Learn how to [create pipelines](/bigquery/docs/create-pipelines).\n- Learn how to [schedule pipelines](/bigquery/docs/schedule-pipelines)."]]
