Workflow Templates—Monitoring & Debugging

This page provides information to help you monitor and debug Cloud Dataproc workflows.

Listing workflows

An instantiated WorkflowTemplate is a called a "workflow" and is modeled as an "operation."

Run the following gcloud command to list your project's workflows:

gcloud beta dataproc operations list --filter "operationType = WORKFLOW"

OPERATION_NAME                                                DONE
projects/.../operations/07282b66-2c60-4919-9154-13bd4f03a1f2  True
projects/.../operations/1c0b0fd5-839a-4ad4-9a57-bbb011956690  True

Here'a a sample request to list all workflows started from a "terasort" template:

gcloud beta dataproc operations list \
   --filter "labels.goog-dataproc-workflow-template-id=terasort

OPERATION_NAME                                     DONE
projects/.../07282b66-2c60-4919-9154-13bd4f03a1f2  True
projects/.../1c0b0fd5-839a-4ad4-9a57-bbb011956690  True

Note that only the UUID portion of OPERATION_NAME is used in subsequent queries.

Using WorkflowMetadata

The operation.metadata field provides information to help you diagnose workflow failures.

Here’s sample workflow metadata, including a graph of nodes (jobs), embedded in an operation:

{
  "name": "projects/my-project/regions/global/operations/671c1d5d-9d24-4cc7-8c93-846e0f886d6e",
  "metadata": {
    "@type": "type.googleapis.com/google.cloud.dataproc.v1beta2.WorkflowMetadata",
    "template": "terasort",
    "version": 1,
    "createCluster": {
      "operationId": "projects/my-project/regions/global/operations/8d472870-4a8b-4609-9f7d-48daccb028fc",
      "isDone": true
    },
    "graph": {
      "nodes": [
        {
          "stepId": "teragen",
          "jobId": "teragen-vtrprwcgepyny",
          "state": "COMPLETED"
        },
        {
          "stepId": "terasort",
          "prerequisiteStepIds": [
            "teragen"
          ],
          "jobId": "terasort-vtrprwcgepyny",
          "state": "FAILED",
          "error": "Job failed"
        },
        {
          "stepId": "teravalidate",
          "prerequisiteStepIds": [
            "terasort"
          ],
          "state": "FAILED",
          "error": "Skipped, node terasort failed"
        }
      ]
    },
    "deleteCluster": {
      "operationId": "projects/my-project/regions/global/operations/9654c67b-2642-4142-a145-ca908e7c81c9",
      "isDone": true
    },
    "state": "DONE",
    "clusterName": "terasort-cluster-vtrprwcgepyny"
  },
  "done": true,
  "error": {
    "message": "Workflow failed"
  }
}
Done!

Retrieving a template

As shown in the previous example, the metadata contains the template id and version.

    "template": "terasort",
    "version": 1,

If a template is not deleted, instantiated template versions can be retrieved by a get-with-version request.

gcloud beta dataproc workflow-templates describe \
  terasort --version 1

List cluster operations started by a template:

gcloud beta dataproc operations list \
  --filter  "labels.goog-dataproc-workflow-instance-id = \
             07282b66-2c60-4919-9154-13bd4f03a1f2"

OPERATION_NAME                                     DONE
projects/.../cf9ce692-d6c9-4671-a909-09fd62041024  True
projects/.../1bbaefd9-7fd9-460f-9adf-ee9bc448b8b7  True

Here's a sample request to list jobs submitted from a template:

gcloud beta dataproc jobs list \
  --filter "labels.goog-dataproc-workflow-template-id = terasort"

JOB_ID                TYPE     STATUS
terasort2-ci2ejdq2ta7l6  pyspark  DONE
terasort2-ci2ejdq2ta7l6  pyspark  DONE
terasort1-ci2ejdq2ta7l6  pyspark  DONE
terasort3-3xwsy6ubbs4ak  pyspark  DONE
terasort2-3xwsy6ubbs4ak  pyspark  DONE
terasort1-3xwsy6ubbs4ak  pyspark  DONE
terasort3-ajov4nptsllti  pyspark  DONE
terasort2-ajov4nptsllti  pyspark  DONE
terasort1-ajov4nptsllti  pyspark  DONE
terasort1-b262xachbv6c4  pyspark  DONE
terasort1-cryvid3kreea2  pyspark  DONE
terasort1-ndprn46nesbv4  pyspark  DONE
terasort1-yznruxam4ppxi  pyspark  DONE
terasort1-ttjbhpqmw55t6  pyspark  DONE
terasort1-d7svwzloplbni  pyspark  DONE

List jobs submitted from a workflow instance:

gcloud beta dataproc jobs list \
  --filter  "labels.goog-dataproc-workflow-instance-id = \
             07282b66-2c60-4919-9154-13bd4f03a1f2"

JOB_ID                TYPE     STATUS
terasort3-ci2ejdq2ta7l6  pyspark  DONE
terasort2-ci2ejdq2ta7l6  pyspark  DONE
terasort1-ci2ejdq2ta7l6  pyspark  DONE
Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Dataproc Documentation