[[["易于理解","easyToUnderstand","thumb-up"],["解决了我的问题","solvedMyProblem","thumb-up"],["其他","otherUp","thumb-up"]],[["很难理解","hardToUnderstand","thumb-down"],["信息或示例代码不正确","incorrectInformationOrSampleCode","thumb-down"],["没有我需要的信息/示例","missingTheInformationSamplesINeed","thumb-down"],["翻译问题","translationIssue","thumb-down"],["其他","otherDown","thumb-down"]],["最后更新时间 (UTC)：2025-08-19。"],[[["\u003cp\u003eCloud Data Fusion's Dataplex Source plugin allows reading data from Dataplex entities (tables) located on Cloud Storage or BigQuery assets, treating data in Cloud Storage as tables with SQL filtering capabilities.\u003c/p\u003e\n"],["\u003cp\u003eUsing this plugin requires a Cloud Data Fusion instance version 6.6 or later, and the source data must reside in a Dataplex zone and asset.\u003c/p\u003e\n"],["\u003cp\u003eTo read from Cloud Storage, a metastore must be configured for the lake and the data must be in JSON, Avro, Parquet, or ORC formats, as CSV is not supported.\u003c/p\u003e\n"],["\u003cp\u003eSpecific IAM roles, including Dataplex Developer, Dataplex Data Reader, Dataproc Metastore Metadata User, Cloud Dataplex Service Agent, and Dataplex Metadata Reader, are required to manage roles and utilize this plugin.\u003c/p\u003e\n"],["\u003cp\u003eSample pipelines, such as SAP source to Dataplex sink and Dataplex source to BigQuery sink, are available in the Cloud Data Fusion UI under the Hub section.\u003c/p\u003e\n"]]],[],null,["# Process data with Cloud Data Fusion\n\n[Cloud Data Fusion](/data-fusion) provides a Dataplex Universal Catalog Source plugin\nto read data from Dataplex Universal Catalog entities (tables) residing on\nCloud Storage or BigQuery assets. The Dataplex Universal Catalog Source\nplugin lets you treat data in Cloud Storage assets as tables and filter\nthe data with SQL queries.\n\nBefore you begin\n----------------\n\n- [Create a Cloud Data Fusion instance](/data-fusion/docs/how-to/create-instance),\n if you don't have one. This plugin is available in instances that run in\n Cloud Data Fusion version 6.6 or later.\n\n- The source data must already be part of a Dataplex Universal Catalog\n [zone](/dataplex/docs/add-zone) and an [asset](/dataplex/docs/manage-assets)\n (either a Cloud Storage bucket or a BigQuery dataset).\n\n- To use tables from Cloud Storage, you must configure a metastore\n for your lake.\n\n- For data to be read from Cloud Storage entities,\n Dataproc Metastore must be attached to the lake.\n\n- CSV data in Cloud Storage entities isn't supported.\n\n- In the Dataplex Universal Catalog project, enable Private Google Access on the\n subnetwork, which is usually set to `default`, or set `internal_ip_only` to\n `false`.\n\n### Limitations\n\n- For Cloud Storage assets: this plugin does not support reading from\n CSV files. It supports reading from JSON, Avro, Parquet, and ORC formats.\n\n- For Cloud Storage assets: **Partition Start Date** and **Partition\n End Date** aren't applicable.\n\n### Required roles\n\n\nTo get the permissions that\nyou need to manage roles,\n\nask your administrator to grant you the\nfollowing IAM roles on the Dataproc service agent and the Cloud Data Fusion service agent (service-\u003cvar translate=\"no\"\u003eCUSTOMER_PROJECT_NUMBER\u003c/var\u003e@gcp-sa-datafusion.iam.gserviceaccount.com):\n\n- [Dataplex Developer](/iam/docs/roles-permissions/dataplex#dataplex.developer) (`roles/dataplex.developer`)\n- [Dataplex Data Reader](/iam/docs/roles-permissions/dataplex#dataplex.dataReader) (`roles/dataplex.dataReader`)\n- [Dataproc Metastore Metadata User](/iam/docs/roles-permissions/metastore#metastore.metadataUser) (`roles/metastore.metadataUser`)\n- [Cloud Dataplex Service Agent](/iam/docs/roles-permissions/dataplex#dataplex.serviceAgent) (`roles/dataplex.serviceAgent`)\n- [Dataplex Metadata Reader](/iam/docs/roles-permissions/dataplex#dataplex.metadataReader) (`roles/dataplex.metadataReader`)\n\n\nFor more information about granting roles, see [Manage access to projects, folders, and organizations](/iam/docs/granting-changing-revoking-access).\n\n\nYou might also be able to get\nthe required permissions through [custom\nroles](/iam/docs/creating-custom-roles) or other [predefined\nroles](/iam/docs/roles-overview#predefined).\n\nAdd the plugin to your pipeline\n-------------------------------\n\n1. In the Google Cloud console, go to the Cloud Data Fusion **Instances** page.\n\n [Go to Instances](https://console.cloud.google.com/data-fusion/locations/-/instances)\n\n This page lets you manage your instances.\n2. Click **View instance** to open your instance in the Cloud Data Fusion\n UI.\n\n3. Go to the **Studio** page, expand the **Source** menu, and click **Dataplex**.\n\nConfigure the plugin\n--------------------\n\nAfter you add this plugin to your pipeline on the **Studio** page, click\nthe Dataplex Universal Catalog source to configure its properties.\n\nFor more information about configurations, see the\n[Dataplex Source](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/1766817793/Google+Dataplex+Batch+Source) reference.\n\nOptional: Get started with a sample pipeline\n--------------------------------------------\n\nSample pipelines are available, including an SAP source to\nDataplex Universal Catalog sink pipeline and a Dataplex Universal Catalog source to\nBigQuery sink pipeline.\n\nTo use a sample pipeline, open your instance in the Cloud Data Fusion UI,\nclick **Hub \\\u003e Pipelines**, and select one of the\nDataplex Universal Catalog pipelines. A dialog opens to help you create the\npipeline.\n\nWhat's next\n-----------\n\n- [Ingest data with Cloud Data Fusion](/dataplex/docs/ingest-with-data-fusion) using the Dataplex Universal Catalog Sink plugin."]]
