Para receber as permissões necessárias para ler de uma fonte de streaming do Pub/Sub,
peça ao administrador para conceder a você o
papel do IAM de Editor do Pub/Sub (roles/pubsub.editor)
na conta de serviço usada para acessar a assinatura do Pub/Sub.
Para mais informações sobre a concessão de papéis, consulte Gerenciar o acesso a projetos, pastas e organizações.
Esse papel predefinido contém
as permissões necessárias para ler de uma fonte de streaming do Pub/Sub. Para conferir as permissões exatas
necessárias, expanda a seção Permissões necessárias:
Permissões necessárias
As seguintes permissões são necessárias para ler de uma fonte de streaming do Pub/Sub:
Você concede o papel na conta de serviço especificada nas propriedades do plug-in
para acessar o Pub/Sub. Se nenhuma for especificada, conceda a função na conta de serviço do Dataproc.
Para mais informações sobre como conceder papéis, consulte
Gerenciar o acesso.
Adicionar uma origem do Pub/Sub ao pipeline de dados de streaming
Acesse sua instância:
No console Google Cloud , acesse a página do Cloud Data Fusion.
Para abrir a instância no Cloud Data Fusion Studio, clique em Instâncias e em Ver instância.
Fazer upgrade de um pipeline com uma origem de streaming do Pub/Sub
O Cloud Data Fusion oferece suporte a upgrades diretos de aplicativos
para pipelines de streaming com uma origem de streaming do Pub/Sub criada na
versão 6.9.1 ou mais recente.
O Cloud Data Fusion não oferece suporte a upgrades de pipelines de dados com uma
fonte de streaming do Pub/Sub na versão 6.9.0 ou anterior. Em vez disso, faça upgrade desses pipelines para a versão 6.9.1:
Pare de publicar os dados no tópico quando o upgrade da instância for planejado.
Aguarde a conclusão do processamento dos dados publicados pelo pipeline.
Depois que os dados forem totalmente processados, pare o pipeline.
[[["Fácil de entender","easyToUnderstand","thumb-up"],["Meu problema foi resolvido","solvedMyProblem","thumb-up"],["Outro","otherUp","thumb-up"]],[["Difícil de entender","hardToUnderstand","thumb-down"],["Informações incorretas ou exemplo de código","incorrectInformationOrSampleCode","thumb-down"],["Não contém as informações/amostras de que eu preciso","missingTheInformationSamplesINeed","thumb-down"],["Problema na tradução","translationIssue","thumb-down"],["Outro","otherDown","thumb-down"]],["Última atualização 2025-09-04 UTC."],[[["\u003cp\u003eCloud Data Fusion supports Pub/Sub as a streaming data source in real-time pipelines, enabling the processing of data from Pub/Sub subscriptions.\u003c/p\u003e\n"],["\u003cp\u003eTo read from a Pub/Sub streaming source, users must have the \u003ccode\u003ePub/Sub Editor\u003c/code\u003e IAM role or specific permissions including \u003ccode\u003epubsub.snapshots.create\u003c/code\u003e, \u003ccode\u003epubsub.snapshots.delete\u003c/code\u003e, \u003ccode\u003epubsub.snapshots.seek\u003c/code\u003e, \u003ccode\u003epubsub.subscriptions.consume\u003c/code\u003e, and \u003ccode\u003epubsub.topics.attachSubscription\u003c/code\u003e.\u003c/p\u003e\n"],["\u003cp\u003eIn Cloud Data Fusion version 6.9.1 and later, pipelines with a single Pub/Sub streaming source do not need Windower plugins or Spark checkpointing, and data is processed at least once using Pub/Sub snapshots.\u003c/p\u003e\n"],["\u003cp\u003eUpgrading pipelines with Pub/Sub streaming sources from version 6.9.0 or earlier to 6.9.1 or later requires a manual process involving stopping the old pipeline, upgrading the instance, duplicating and updating the pipeline, then running and deleting the old one.\u003c/p\u003e\n"],["\u003cp\u003eThe Pub/Sub streaming source will create a snapshot at the beginning of each batch and remove it at the end of the batch, which will incur storage cost, and these snapshots can be monitored using Cloud Audit Logs.\u003c/p\u003e\n"]]],[],null,["# Read from a Pub/Sub streaming source\n\nCloud Data Fusion supports Pub/Sub sources in streaming data pipelines.\n\nBefore you begin\n----------------\n\n- [Create a Cloud Data Fusion instance](/data-fusion/docs/how-to/create-instance).\n\n### Roles and permissions\n\n\nTo get the permissions that\nyou need to read from a Pub/Sub streaming source,\n\nask your administrator to grant you the\n\n\n[Pub/Sub Editor](/iam/docs/roles-permissions/pubsub#pubsub.editor) (`roles/pubsub.editor`)\nIAM role on the service account used to access the Pub/Sub subscription.\n\n\nFor more information about granting roles, see [Manage access to projects, folders, and organizations](/iam/docs/granting-changing-revoking-access).\n\n\nThis predefined role contains\n\nthe permissions required to read from a Pub/Sub streaming source. To see the exact permissions that are\nrequired, expand the **Required permissions** section:\n\n\n#### Required permissions\n\nThe following permissions are required to read from a Pub/Sub streaming source:\n\n- ` pubsub.snapshots.create `\n- ` pubsub.snapshots.delete `\n- ` pubsub.snapshots.seek `\n- ` pubsub.subscriptions.consume `\n- ` pubsub.topics.attachSubscription`\n\n\nYou might also be able to get\nthese permissions\nwith [custom roles](/iam/docs/creating-custom-roles) or\nother [predefined roles](/iam/docs/roles-overview#predefined).\n\nYou grant the role on the service account you specified in the plugin properties\nfor accessing Pub/Sub. If none is specified, grant the\nrole on the Dataproc service account.\n| **Note:** In versions 6.9.1 and later, the predefined Pub/Sub Subscriber role isn't enough, as `pubsub.snapshots.create` and `pubsub.snapshots.delete` permission are required.\n\nFor more information about granting roles, see\n[Manage access](/iam/docs/granting-changing-revoking-access).\n\nAdd a Pub/Sub source to your streaming data pipeline\n----------------------------------------------------\n\n1. Go to your instance:\n\n\n 1. In the Google Cloud console, go to the Cloud Data Fusion page.\n\n 2. To open the instance in the Cloud Data Fusion Studio,\n click **Instances** , and then click **View instance**.\n\n [Go to Instances](https://console.cloud.google.com/data-fusion/locations/-/instances)\n\n \u003cbr /\u003e\n\n2. In the Cloud Data Fusion web interface, click **Studio**.\n\n3. Select **Data Pipeline - Realtime**.\n\n4. In the **Source** menu, select **Pub/Sub**. A Pub/Sub\n streaming source node appears in the pipeline.\n\n5. On the Pub/Sub node, click **Properties** to configure the\n source. For more information, see\n [Pub/Sub Streaming Source](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/464912525/Google+Cloud+Pub+Sub+Streaming+Source).\n\n| **Note:** Cloud Data Fusion supports Pub/Sub pull subscriptions.\n\nSupport for a single Pub/Sub source with no Windower plugins\n------------------------------------------------------------\n\nCloud Data Fusion version 6.9.1 supports real time pipelines with a single\nPub/Sub streaming source and no Windower plugins.\n\n- The Pub/Sub streaming source has built-in support and data is processed at least once. Enabling Spark checkpointing isn't required.\n- The Pub/Sub streaming source creates a [Pub/Sub snapshot](/pubsub/docs/replay-overview#seek_to_a_snapshot) at the beginning of each batch and removes it at the end of each batch.\n- Creating Pub/Sub snapshots has a cost associated with it. For more information, see [Pub/Sub pricing](/pubsub/pricing#storage_costs).\n- You can monitor snapshot creation in [Cloud Audit Logs](/pubsub/docs/audit-logging).\n\nUpgrade a pipeline with a Pub/Sub streaming source\n--------------------------------------------------\n\nCloud Data Fusion supports [direct application upgrades](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/477560983/Lifecycle+Microservices#Upgrade-an-Application)\nfor streaming pipelines with a Pub/Sub streaming source created in\n6.9.1 or later.\n\nCloud Data Fusion doesn't support upgrades for data pipelines with a\nPub/Sub streaming source in version 6.9.0 or earlier. Instead,\nupgrade those pipelines to 6.9.1:\n\n1. Stop publishing the data to the topic when the instance upgrade is planned.\n2. Wait for the pipeline to finish processing the published data.\n3. After the data is processed completely, stop the pipeline.\n4. [Upgrade the instance](/data-fusion/docs/how-to/upgrading).\n5. Duplicate the existing pipeline and update to the newest plugins.\n6. Deploy the pipeline.\n7. Run the new pipeline to read data.\n\n The new version automatically uses snapshot instead of Spark checkpointing.\n8. Delete the old pipeline.\n\nWhat's next\n-----------\n\n- Refer to the CDAP [Pub/Sub Streaming Source](https://cdap.atlassian.net/wiki/spaces/DOCS/pages/464912525/Google+Cloud+Pub+Sub+Streaming+Source)."]]
