Monitor workflows

This page provides information to help you monitor workflow deployments and executions.

Access workflow deployment and deletion logs

You can access error logs related to a workflow's deployment and deletion in the Google Cloud console.

  1. In the Google Cloud console, go to the Workflows page.

    Go to Workflows

  2. Click the workflow's name to display its Workflow details page.

  3. Click the Logs tab.

  4. To filter the logs by severity, in the Default list, select the type of log to display.

Access workflow execution results

You can access workflow execution results in the Google Cloud console or by using the gcloud CLI.

Console

  1. In the Google Cloud console, go to the Workflows page.

    Go to Workflows

  2. To access a workflow's execution results, click the workflow's name to go to its Workflow details page.

  3. For details about a particular execution, on the Executions tab, click the execution ID in the list to go to its Execution details page.

  4. On the Summary tab, each execution has the following information:

    • Execution state: indicates the workflow's end state, including the current or final workflow step.
    • Execution start: when the execution began.
    • Execution end: when the execution ended.
    • Execution duration: total time elapsed. This can be an indication of network errors or connectivity problems.
    • Workflow name: the workflow name.
    • Workflow revision: the current revision at the time of execution.
    • Call logging level: the level of call logging applied during the execution. In this document, see Call logging.
    • Input: the runtime arguments passed to the workflow, if any.
    • Output: the workflow's output. If the execution failed, includes the exception that led to the execution's failure. In this document, see Execution error messages.

  5. To view the workflow execution history as a list of step entries, click the Steps tab. For more information, see View history of execution steps.

  6. To view the logs for a workflow execution, click the Logs tab.

  7. To filter the execution logs, use the Filter field at the top of the table. For example, to display only failed execution attempts, enter failed in the filter's text field.

gcloud

  1. To see a full list of a workflow's executions, enter the following command:

    gcloud workflows executions list WORKFLOW_NAME

    Replace WORKFLOW_NAME with your workflow's name. Copy the execution ID of the execution you're interested in.

  2. To view a workflow's execution logs, enter the following command:

    gcloud workflows executions describe \
    --workflow=WORKFLOW_NAME \
    EXECUTION_ID

    Replace the following:

    • WORKFLOW_NAME: the workflow's name
    • EXECUTION_ID: the execution's unique ID

    This command returns output similar to the following: 

    argument: 'null'
endTime: '2022-07-19T12:40:07.070039707Z'
error:
 context: |-
    The argument of 'in' must be a dict or an array; got: null
    in step "checkSearchTermInInput", routine "main", line: 12
 payload: "{"message":"The argument of 'in' must be a dict or an array; got: null"
    
    ,"tags":["TypeError"]}"
 stackTrace:
    elements:
    - position:
       column: '26'
       length: '24'
       line: '12'
       routine: main
       step: checkSearchTermInInput
name: projects/1051295516635/locations/us-central1/workflows/myFirstWorkflow/executions/17ffc89c-0a27-4d2f-8356-e681d949a3d3
startTime: '2022-07-19T12:40:07.024823663Z'
state: FAILED
status:
 currentSteps:
 - routine: main
    step: checkSearchTermInInput
workflowRevisionId: 000001-ac2
    The output contains the following information:

    • argument: the runtime arguments passed to the workflow, if any
    • endTime: when the execution ended
    • error: the error message thrown as a part of the exception that resulted in the execution's failure
    • name: the full name of the execution, including the name of the project, the location of the workflow, the name of the workflow, and the execution ID
    • startTime: when the execution began
    • state: indicates the workflow's end state
    • status: the current or final workflow step of the execution
    • workflowRevisionID: the current revision at the time of the execution

Execution error maps

When a workflow throws an error during execution that isn't caught in a try/except block, the execution fails, and an error map (a JSON dictionary) describing the error is returned.

Errors thrown during workflow execution contain tags to help you identify what caused the error. For example, the error returned from a connector can have two keys (tags and message) similar to the following:

{'tags': ['SystemError'], 'message': 'an error has occurred'}

There can be more than one tag. To check for a specific tag, you can use an expression. For example:

${'SystemError' in e.tags}

Access error data returned as a string

Some connectors and HTTP APIs will serialize errors as strings before returning the errors. You can use standard library functions to restore a payload to the original error. For example, to convert an error string to a map, you can use the json.decode and text.encode functions:

json.decode(text.encode(ERROR_FROM_API))

Error tags

The following table describes the meaning of different error tags.

Tag Description
AuthError Raised when generating credentials for an HTTP request fails.
ConnectionError Raised when a connection is successfully established with the endpoint but there is a problem with the connection during data transfer. The connection is terminated before a full response is received and a message might not have been delivered to the endpoint. Retries might not be idempotent.
ConnectionFailedError Raised when a connection is not established with the API endpoint; for example, due to an incorrect domain name, DNS resolution issues, or other network problems. Retries are idempotent.
HttpError Raised when an HTTP request fails with an HTTP error status. When this exception is raised, the response is a map with the following elements:
  • tags—list with HttpError string
  • message—human-readable error message
  • code—HTTP response status code
  • headers—response headers
  • body—response body
IndexError Raised when a sequence subscript is an out of range integer.
KeyError Raised when a map key is not found in the set of existing keys.
OperationError Raised when a long-running operation finishes unsuccessfully.
ParallelNestingError Raised when the maximum depth that parallel steps can be nested is exceeded.
RecursionError Raised when the interpreter detects that the maximum call stack depth is exceeded.
ResourceLimitError Raised when some resource limit is exhausted. When raised internally, this type of error cannot be caught and causes immediate execution failure.
ResponseTypeError Raised when a long-running operation returns a response of the wrong type.
SystemError Raised when the interpreter finds an internal error.
TimeoutError Raised when a system function times out at the system level.
TypeError Raised when an operation or function is applied to an object of incompatible type. The associated value is a string giving details about the type mismatch.
UnhandledBranchError Raised when one or more branches or iterations encounters an unhandled runtime error up to a maximum number.
ValueError Raised when an operation or function receives an argument that has the correct type but an incorrect value, and the situation is not described by a more precise exception, such as an IndexError.
ZeroDivisionError Raised when the second argument of a division or modulo operation is zero. The associated value is a string indicating the type of the operands and the operation.

You can also raise custom errors using the raise syntax.

Check the status of executions

There are several commands to help you check the status of a workflow execution.

  • To retrieve a list of a workflow's execution attempts and their IDs, enter the following command:

    gcloud workflows executions list WORKFLOW_NAME

    Replace WORKFLOW_NAME with the name of the workflow.

    The command returns a NAME value that is similar to the following:

    projects/PROJECT_NUMBER/locations/REGION/workflows/WORKFLOW_NAME/executions/EXECUTION_ID

    Copy the execution ID to use in the next command.

  • To check the status of an execution attempt and wait for the attempt to finish, enter the following command:

    gcloud workflows executions wait EXECUTION_ID

    Replace EXECUTION_ID with the execution attempt's ID.

    The command waits for the execution attempt to finish and then returns the results.

  • To wait until the last execution is complete and then return the result of the completed execution, enter the following command:

    gcloud workflows executions wait-last

    If you made a previous execution attempt in the same gcloud session, the command waits for the prior execution attempt to finish and then returns the results of the completed execution. If no previous attempt exists, gcloud returns the following error:

    ERROR: (gcloud.workflows.executions.wait-last) [NOT FOUND] There are no cached executions available.

  • To get the status of the last execution, enter the following command:

    gcloud workflows executions describe-last

    If you made a previous execution attempt in the same gcloud session, the command returns the results of the last execution even if it is running. If no previous attempt exists, gcloud returns the following error:

    ERROR: (gcloud.beta.workflows.executions.describe-last) [NOT FOUND] There are no cached executions available.

Filter executions

You can apply filters to the list of workflow executions returned by the workflows.executions.list method.

You can filter on the following fields:

  • createTime
  • disableOverflowBuffering
  • duration
  • endTime
  • executionId
  • label
  • startTime
  • state
  • stepName
  • workflowRevisionId

For example, to filter on a label (labels."fruit":"apple"), you can make an API request similar to the following:

GET https://workflowexecutions.googleapis.com/v1/projects/MY_PROJECT/locations/MY_LOCATION/workflows/MY_WORKFLOW/executions?view=full&filter=labels.%22fruit%22%3A%22apple%22"

Where:

  • view=full specifies a view defining which fields should be filled in the returned executions; in this case, all the data
  • labels.%22fruit%22%3A%22apple%22is the URL-encoded filter syntax

For more information, see AIP-160 Filtering.

Send logs to Cloud Logging

Workflows automatically generates execution logs for workflow executions in Cloud Logging.

You can also enable call logging. Or, you can create custom logs that use the sys.log function in your source. Call logging and custom logs let you control when logs are sent to Logging during a workflow execution and can be particularly helpful when debugging your workflow.

For details, including the engine_call and executions_system logging proto files, see this GitHub repository.

Execution logs

Each workflow execution automatically triggers at least two execution logs: one at the start of an execution and one at the end.

For more information about the Workflows platform logs that are available in Logging, see Google Cloud platform logs.

Call logging

You can set a flag so that each call step during the execution of your workflow is logged, and step names, function names, function arguments, and call responses are returned. Or, you can log any exceptions that are caught, or that stop a call.

Only explicit call steps are logged; for example, calls to subworkflows or library functions. Calls from within expressions, or within standard library functions (for example, http.post in sys.log), and inside connectors are not logged.

HTTP Authorization request headers are redacted from the logs for HTTP calls.

When applying call logging to a workflow definition or to the execution of a workflow, you can specify the level of logging required. The execution log level takes precedence over any workflow log level, unless the execution log level is not specified (the default); in that case, the workflow log level applies.

Note that the log entry size limit set by Cloud Logging also applies to call logging.

Custom logs

To create a log entry in Logging during a workflow execution, define a step in the workflow that makes a call to the standard library sys.log function:

YAML

  - step1:
      assign:
          - varA: "Hello"
          - varB: "World"
  - logStep:
      call: sys.log
      args:
          text: TEXT
          severity: SEVERITY 
  - step2:
      return: ${varA + " " + varB}

JSON

    [
      {
        "step1": {
          "assign": [
            {
              "varA": "Hello"
            },
            {
              "varB": "World"
            }
          ]
        }
      },
      {
        "logStep": {
          "call": "sys.log",
          "args": {
            "text": "TEXT",
            "severity": "SEVERITY"
          }
        }
      },
      {
        "step2": {
          "return": "${varA + " " + varB}"
        }
      }
    ]

When creating a log entry, define the following:

  • TEXT: required. The text to be logged. If you need to log the values of a map, use ${json.encode_to_string(myMap)}.
  • SEVERITY: optional. The severity level of the log entry. For example, INFO, WARNING, or CRITICAL.

For more information, see the sys.log function reference.

Required permissions

To apply call logging or send custom logs to Logging, a workflow must be associated with a service account that includes the logging.logEntries.create permission (for example, the roles/logging.logWriter role). If you need to change the service account updated with your workflow, see Update a workflow. To learn more about creating service accounts and assigning roles, see Manage access to projects, folders, and organizations.

View workflow logs

You can view logs in Workflows or in Logging. To view the logs for a single workflow, use the Logs tab in Workflows. To get an aggregate view of the logs for all of your workflows, use the Logs Explorer page in Logging.

View logs in Workflows

To view a workflow's logs in Workflows, do the following:

  1. In the Google Cloud console, go to the Workflows page:

    Go to Workflows

  2. To access a workflow's logs, click the workflow's name to go to its Details page.

  3. To view the logs, click Logs.

  4. To filter the logs by severity, in the Default list, select the type of log to display. By default, logs of all severity levels are shown.

The Logs tab on a workflow's Details page displays the following types of logs:

  • Logs sent to Logging

  • Audit logs of any operations performed on the workflow, such as updates to the workflow's definition

View logs in Logging

To view logs in Logging, do the following:

  1. In the Google Cloud console, go to the Logs Explorer page:

    Go to Logs Explorer

  2. In the Query builder, click Resource and enter workflow. Select Cloud Workflow from the list and click Add.

    Workflow logging

  3. Click Run query.

To learn more about viewing logs in Logging, see Use the Logs Explorer.

What's next