[[["容易理解","easyToUnderstand","thumb-up"],["確實解決了我的問題","solvedMyProblem","thumb-up"],["其他","otherUp","thumb-up"]],[["難以理解","hardToUnderstand","thumb-down"],["資訊或程式碼範例有誤","incorrectInformationOrSampleCode","thumb-down"],["缺少我需要的資訊/範例","missingTheInformationSamplesINeed","thumb-down"],["翻譯問題","translationIssue","thumb-down"],["其他","otherDown","thumb-down"]],["上次更新時間：2025-09-04 (世界標準時間)。"],[[["\u003cp\u003eCloud Data Fusion supports Pub/Sub as a streaming data source in real-time pipelines, enabling the processing of data from Pub/Sub subscriptions.\u003c/p\u003e\n"],["\u003cp\u003eTo read from a Pub/Sub streaming source, users must have the \u003ccode\u003ePub/Sub Editor\u003c/code\u003e IAM role or specific permissions including \u003ccode\u003epubsub.snapshots.create\u003c/code\u003e, \u003ccode\u003epubsub.snapshots.delete\u003c/code\u003e, \u003ccode\u003epubsub.snapshots.seek\u003c/code\u003e, \u003ccode\u003epubsub.subscriptions.consume\u003c/code\u003e, and \u003ccode\u003epubsub.topics.attachSubscription\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eIn Cloud Data Fusion version 6.9.1 and later, pipelines with a single Pub/Sub streaming source do not need Windower plugins or Spark checkpointing, and data is processed at least once using Pub/Sub snapshots.\u003c/p\u003e\n"],["\u003cp\u003eUpgrading pipelines with Pub/Sub streaming sources from version 6.9.0 or earlier to 6.9.1 or later requires a manual process involving stopping the old pipeline, upgrading the instance, duplicating and updating the pipeline, then running and deleting the old one.\u003c/p\u003e\n"],["\u003cp\u003eThe Pub/Sub streaming source will create a snapshot at the beginning of each batch and remove it at the end of the batch, which will incur storage cost, and these snapshots can be monitored using Cloud Audit Logs.\u003c/p\u003e\n"]]],[],null,["# Read from a Pub/Sub streaming source\n\nCloud Data Fusion supports Pub/Sub sources in streaming data pipelines.\n\nBefore you begin\n----------------\n\n- [Create a Cloud Data Fusion instance](/data-fusion/docs/how-to/create-instance).\n\n### Roles and permissions\n\n\nTo get the permissions that\nyou need to read from a Pub/Sub streaming source,\n\nask your administrator to grant you the\n\n\n[Pub/Sub Editor](/iam/docs/roles-permissions/pubsub#pubsub.editor) (`roles/pubsub.editor`)\nIAM role on the service account used to access the Pub/Sub subscription.\n\n\nFor more information about granting roles, see [Manage access to projects, folders, and organizations](/iam/docs/granting-changing-revoking-access).\n\n\nThis predefined role contains\n\nthe permissions required to read from a Pub/Sub streaming source. To see the exact permissions that are\nrequired, expand the **Required permissions** section:\n\n\n#### Required permissions\n\nThe following permissions are required to read from a Pub/Sub streaming source:\n\n- ` pubsub.snapshots.create `\n- ` pubsub.snapshots.delete `\n- ` pubsub.snapshots.seek `\n- ` pubsub.subscriptions.consume `\n- ` pubsub.topics.attachSubscription`\n\n\nYou might also be able to get\nthese permissions\nwith [custom roles](/iam/docs/creating-custom-roles) or\nother [predefined roles](/iam/docs/roles-overview#predefined).\n\nYou grant the role on the service account you specified in the plugin properties\nfor accessing Pub/Sub. If none is specified, grant the\nrole on the Dataproc service account.\n| **Note:** In versions 6.9.1 and later, the predefined Pub/Sub Subscriber role isn't enough, as `pubsub.snapshots.create` and `pubsub.snapshots.delete` permission are required.\n\nFor more information about granting roles, see\n[Manage access](/iam/docs/granting-changing-revoking-access).\n\nAdd a Pub/Sub source to your streaming data pipeline\n----------------------------------------------------\n\n1. Go to your instance:\n\n\n 1. In the Google Cloud console, go to the Cloud Data Fusion page.\n\n 2. To open the instance in the Cloud Data Fusion Studio,\n click **Instances** , and then click **View instance**.\n\n [Go to Instances](https://console.cloud.google.com/data-fusion/locations/-/instances)\n\n \u003cbr /\u003e\n\n2. In the Cloud Data Fusion web interface, click **Studio**.\n\n3. Select **Data Pipeline - Realtime**.\n\n4. In the **Source** menu, select **Pub/Sub**. A Pub/Sub\n streaming source node appears in the pipeline.\n\n5. On the Pub/Sub node, click **Properties** to configure the\n source. For more information, see\n [Pub/Sub Streaming Source](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/464912525/Google+Cloud+Pub+Sub+Streaming+Source).\n\n| **Note:** Cloud Data Fusion supports Pub/Sub pull subscriptions.\n\nSupport for a single Pub/Sub source with no Windower plugins\n------------------------------------------------------------\n\nCloud Data Fusion version 6.9.1 supports real time pipelines with a single\nPub/Sub streaming source and no Windower plugins.\n\n- The Pub/Sub streaming source has built-in support and data is processed at least once. Enabling Spark checkpointing isn't required.\n- The Pub/Sub streaming source creates a [Pub/Sub snapshot](/pubsub/docs/replay-overview#seek_to_a_snapshot) at the beginning of each batch and removes it at the end of each batch.\n- Creating Pub/Sub snapshots has a cost associated with it. For more information, see [Pub/Sub pricing](/pubsub/pricing#storage_costs).\n- You can monitor snapshot creation in [Cloud Audit Logs](/pubsub/docs/audit-logging).\n\nUpgrade a pipeline with a Pub/Sub streaming source\n--------------------------------------------------\n\nCloud Data Fusion supports [direct application upgrades](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477560983/Lifecycle+Microservices#Upgrade-an-Application)\nfor streaming pipelines with a Pub/Sub streaming source created in\n6.9.1 or later.\n\nCloud Data Fusion doesn't support upgrades for data pipelines with a\nPub/Sub streaming source in version 6.9.0 or earlier. Instead,\nupgrade those pipelines to 6.9.1:\n\n1. Stop publishing the data to the topic when the instance upgrade is planned.\n2. Wait for the pipeline to finish processing the published data.\n3. After the data is processed completely, stop the pipeline.\n4. [Upgrade the instance](/data-fusion/docs/how-to/upgrading).\n5. Duplicate the existing pipeline and update to the newest plugins.\n6. Deploy the pipeline.\n7. Run the new pipeline to read data.\n\n The new version automatically uses snapshot instead of Spark checkpointing.\n8. Delete the old pipeline.\n\nWhat's next\n-----------\n\n- Refer to the CDAP [Pub/Sub Streaming Source](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/464912525/Google+Cloud+Pub+Sub+Streaming+Source)."]]
