[[["わかりやすい","easyToUnderstand","thumb-up"],["問題の解決に役立った","solvedMyProblem","thumb-up"],["その他","otherUp","thumb-up"]],[["わかりにくい","hardToUnderstand","thumb-down"],["情報またはサンプルコードが不正確","incorrectInformationOrSampleCode","thumb-down"],["必要な情報 / サンプルがない","missingTheInformationSamplesINeed","thumb-down"],["翻訳に関する問題","translationIssue","thumb-down"],["その他","otherDown","thumb-down"]],["最終更新日 2025-08-28 UTC。"],[[["\u003cp\u003eThis page details how to monitor and debug Dataproc workflows, which are instantiated WorkflowTemplates modeled as operations.\u003c/p\u003e\n"],["\u003cp\u003eYou can list workflows in your project using the \u003ccode\u003egcloud dataproc operations list\u003c/code\u003e command with specific filters, such as by operation type or template ID.\u003c/p\u003e\n"],["\u003cp\u003eWorkflowMetadata, accessible via the \u003ccode\u003eoperation.metadata\u003c/code\u003e field, provides diagnostic information about workflow failures, including details on the template, version, and a graph of nodes (jobs).\u003c/p\u003e\n"],["\u003cp\u003eWorkflow timeouts can be set to automatically cancel a workflow if its jobs do not complete within the specified period, preventing the need for manual intervention.\u003c/p\u003e\n"],["\u003cp\u003eYou can set, change, or remove workflow template timeouts using the \u003ccode\u003egcloud\u003c/code\u003e command, and the API, and it will only apply to the new workflows instantiated, not existing ones.\u003c/p\u003e\n"]]],[],null,["This page provides information to help you monitor and debug [Dataproc\nworkflows](/dataproc/docs/concepts/workflows/overview).\n\nList workflows\n\nAn instantiated\n[WorkflowTemplate](/dataproc/docs/reference/rest/v1/projects.locations.workflowTemplates#resource-workflowtemplate) is a called a \"workflow\" and is modeled as an \"operation.\"\n\nRun the following `gcloud` command to list your project's workflows: \n\n```\ngcloud dataproc operations list \\\n --region=region \\\n --filter=\"operationType = WORKFLOW\"\n``` \n\n```\n...\nOPERATION_NAME DONE\nprojects/.../operations/07282b66-2c60-4919-9154-13bd4f03a1f2 True\nprojects/.../operations/1c0b0fd5-839a-4ad4-9a57-bbb011956690 True\n```\n\nHere's a sample request to list all workflows started from a\n\"terasort\" template: \n\n```\ngcloud dataproc operations list \\\n --region=region \\\n --filter=\"labels.goog-dataproc-workflow-template-id=terasort\"\n``` \n\n```\n...\nOPERATION_NAME DONE\nprojects/.../07282b66-2c60-4919-9154-13bd4f03a1f2 True\nprojects/.../1c0b0fd5-839a-4ad4-9a57-bbb011956690 True\n```\n\nNote that only the UUID portion of `OPERATION_NAME` is used in subsequent\nqueries.\n\nUsing WorkflowMetadata\n\nThe `operation.metadata` field provides information to help you diagnose\nworkflow failures.\n\nHere's a sample\n[`WorkflowMetadata`](/dataproc/docs/reference/rpc/google.cloud.dataproc.v1#workflowmetadata),\nincluding a graph of nodes (jobs), embedded in an operation: \n\n```\n{\n \"name\": \"projects/my-project/regions/us-central1/operations/671c1d5d-9d24-4cc7-8c93-846e0f886d6e\",\n \"metadata\": {\n \"@type\": \"type.googleapis.com/google.cloud.dataproc.v1.WorkflowMetadata\",\n \"template\": \"terasort\",\n \"version\": 1,\n \"createCluster\": {\n \"operationId\": \"projects/my-project/regions/us-central1/operations/8d472870-4a8b-4609-9f7d-48daccb028fc\",\n \"Done\": true\n },\n \"graph\": {\n \"nodes\": [\n {\n \"stepId\": \"teragen\",\n \"jobId\": \"teragen-vtrprwcgepyny\",\n \"state\": \"COMPLETED\"\n },\n {\n \"stepId\": \"terasort\",\n \"prerequisiteStepIds\": [\n \"teragen\"\n ],\n \"jobId\": \"terasort-vtrprwcgepyny\",\n \"state\": \"FAILED\",\n \"error\": \"Job failed\"\n },\n {\n \"stepId\": \"teravalidate\",\n \"prerequisiteStepIds\": [\n \"terasort\"\n ],\n \"state\": \"FAILED\",\n \"error\": \"Skipped, node terasort failed\"\n }\n ]\n },\n \"deleteCluster\": {\n \"operationId\": \"projects/my-project/regions/us-central1/operations/9654c67b-2642-4142-a145-ca908e7c81c9\",\n \"Done\": true\n },\n \"state\": \"DONE\",\n \"clusterName\": \"terasort-cluster-vtrprwcgepyny\"\n },\n \"done\": true,\n \"error\": {\n \"message\": \"Workflow failed\"\n }\n}\nDone!\n```\n\nRetrieve a template\n\nAs shown in the previous example, the `metadata` contains the template id\nand version. \n\n```\n\"template\": \"terasort\",\n\"version\": 1,\n```\n\nIf a template is not deleted, instantiated template versions can be retrieved\nby a describe-with-version request. \n\n```\ngcloud dataproc workflow-templates describe terasort \\\n --region=region \\\n --version=1\n```\n\nList cluster operations started by a template: \n\n```\ngcloud dataproc operations list \\\n --region=region \\\n --filter=\"labels.goog-dataproc-workflow-instance-id = 07282b66-2c60-4919-9154-13bd4f03a1f2\"\n``` \n\n```\n...\nOPERATION_NAME DONE\nprojects/.../cf9ce692-d6c9-4671-a909-09fd62041024 True\nprojects/.../1bbaefd9-7fd9-460f-9adf-ee9bc448b8b7 True\n```\n\nHere's a sample request to list jobs submitted from a template: \n\n```\ngcloud dataproc jobs list \\\n --region=region \\\n --filter=\"labels.goog-dataproc-workflow-template-id = terasort\"\n``` \n\n```\n...\nJOB_ID TYPE STATUS\nterasort2-ci2ejdq2ta7l6 pyspark DONE\nterasort2-ci2ejdq2ta7l6 pyspark DONE\nterasort1-ci2ejdq2ta7l6 pyspark DONE\nterasort3-3xwsy6ubbs4ak pyspark DONE\nterasort2-3xwsy6ubbs4ak pyspark DONE\nterasort1-3xwsy6ubbs4ak pyspark DONE\nterasort3-ajov4nptsllti pyspark DONE\nterasort2-ajov4nptsllti pyspark DONE\nterasort1-ajov4nptsllti pyspark DONE\nterasort1-b262xachbv6c4 pyspark DONE\nterasort1-cryvid3kreea2 pyspark DONE\nterasort1-ndprn46nesbv4 pyspark DONE\nterasort1-yznruxam4ppxi pyspark DONE\nterasort1-ttjbhpqmw55t6 pyspark DONE\nterasort1-d7svwzloplbni pyspark DONE\n```\n\nList jobs submitted from a workflow instance: \n\n```\ngcloud dataproc jobs list \\\n --region=region \\\n --filter=\"labels.goog-dataproc-workflow-instance-id = 07282b66-2c60-4919-9154-13bd4f03a1f2\"\n``` \n\n```\n...\nJOB_ID TYPE STATUS\nterasort3-ci2ejdq2ta7l6 pyspark DONE\nterasort2-ci2ejdq2ta7l6 pyspark DONE\nterasort1-ci2ejdq2ta7l6 pyspark DONE\n```\n\nWorkflow timeouts\n\nYou can set a workflow timeout that will cancel the workflow if the workflow's\njobs do not finish within the timeout period. The timeout period applies to the\n[DAG (Directed Acyclic Graph)](https://en.wikipedia.org/wiki/Directed_acyclic_graph)\nof jobs in the workflow (the sequence of jobs in the workflow), not to the\nentire workflow operation. The timeout period starts when the first workflow job\nstarts---it does not include the time taken to create a\n[managed cluster](/dataproc/docs/concepts/workflows/using-workflows#configuring_or_selecting_a_cluster).\nIf any job is running at the end of the timeout period, all running jobs are\nstopped, the workflow is ended, and if the workflow was running on a\nmanaged cluster, the cluster is deleted.\n\n**Benefit:** Use this feature to avoid having to manually end a\nworkflow that does not complete due to stuck jobs.\n\nSet a workflow template timeout\n\nYou can set a workflow template timeout period when you\n[create a workflow template](/dataproc/docs/concepts/workflows/using-workflows#creating_a_template). You can also add a workflow timeout\nto an existing workflow template by [updating the workflow template](#updating_a_workflow_template_timeout).\nWorkflow timeouts apply to any workflow [instantiated](/dataproc/docs/concepts/workflows/using-workflows#running_a_workflow) after the timeout is set on the template. Existing workflows are not affected by a new or updated template. \n\ngcloud\n\nTo set a workflow timeout on a new template, use the `--dag-timeout` flag with the\n[gcloud dataproc workflow-templates create](/sdk/gcloud/reference/dataproc/workflow-templates/create) command. You can use \"s\", \"m\", \"h\", and \"d\" suffixes\nto set second, minute, hour, and day duration values, respectively. The timeout\nduration must be from 10 minutes (\"10m\") to 24 hours (\"24h\" or \"1d\"). \n\n```\ngcloud dataproc workflow-templates create template-id (such as \"my-workflow\") \\\n --region=region \\\n --dag-timeout=duration (from \"10m\" to \"24h\" or \"1d\"\") \\\n ... other args ...\n```\n\nAPI\n\nTo set a workflow timeout, complete the `WorkflowTemplate`\n[`dagTimeout`](/dataproc/docs/reference/rest/v1/projects.regions.workflowTemplates#WorkflowTemplate.FIELDS.dag_timeout)\nfield as part of a [workflowTemplates.create](/dataproc/docs/reference/rest/v1/projects.regions.workflowTemplates/create) request.\n\nConsole\n\nCurrently, the Google Cloud console does not support creating a\n[workflow template](https://console.cloud.google.com/dataproc/workflows/templates).\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\nUpdate a workflow template timeout\n\nYou can update an existing workflow template to change, add, or remove\na workflow timeout.\nWorkflow timeouts apply to any workflow [instantiated](/dataproc/docs/concepts/workflows/using-workflows#running_a_workflow) after the timeout is set on the template. Existing workflows are not affected by a new or updated template. \n\ngcloud\n\n**Adding or changing a workflow timeout**\n\nTo add or change a workflow timeout on an existing template, use the\n`--dag-timeout` flag with the\n[gcloud dataproc workflow-templates set-dag-timeout](/sdk/gcloud/reference/dataproc/workflow-templates/set-dag-timeout)\ncommand. You can use \"s\", \"m\", \"h\", and \"d\" suffixes to set second, minute,\nhour, and day duration values, respectively. The timeout duration must be from\n10 minutes (\"10m\") to 24 hours (\"24h\"). \n\n```\ngcloud dataproc workflow-templates set-dag-timeout template-id (such as \"my-workflow\") \\\n --region=region \\\n --dag-timeout=duration (from \"10m\" to \"24h\" or \"1d\")\n```\n\n**Removing a workflow timeout**\n\nTo remove a workflow timeout from an existing template, use the\n[gcloud dataproc workflow-templates remove-dag-timeout](/sdk/gcloud/reference/dataproc/workflow-templates/remove-dag-timeout)\ncommand. \n\n```\ngcloud dataproc workflow-templates remove-dag-timeout template-id (such as \"my-workflow\") \\\n --region=region\n```\n\nAPI\n\n**Adding or changing a workflow timeout**\n\nTo add or change a workflow timeout on an existing template,\n[update the workflow template](/dataproc/docs/concepts/workflows/using-workflows#updating_a_workflow_template)\nby filling in the template's\n[`dagTimeout`](/dataproc/docs/reference/rest/v1/projects.regions.workflowTemplates#WorkflowTemplate.FIELDS.dag_timeout)\nfield with the new or changed timeout value.\n\n**Removing a workflow timeout**\n\nTo remove a workflow timeout from an existing template,\n[update the workflow template](/dataproc/docs/concepts/workflows/using-workflows#updating_a_workflow_template)\nby removing the template's\n[`dagTimeout`](/dataproc/docs/reference/rest/v1/projects.regions.workflowTemplates#WorkflowTemplate.FIELDS.dag_timeout)\nfield.\n\n\nConsole\n\nCurrently, the Google Cloud console does not support updating a\n[workflow template](https://console.cloud.google.com/dataproc/workflows/templates).\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e"]]
