Configure custom status events to describe runnables

This document explains how to configure custom status events, which describe a job's runnables, when you create and run a Batch job. To learn about status events, see View a job's history through status events.

Custom status events let you provide additional details in a task's history about the progress of its runnables, which can help make a job easier to analyze and troubleshoot. For example, you can configure custom status events that describe when a runnable starts, a runnable ends, a barrier runnable is reached, or an important event happens during the progression of your code.

Before you begin

Configure custom status events

Configure custom status events by using one or more of following options when you are creating a job:

  • Describe a runnable's state by defining its display name. You can do this when you create a job using the gcloud CLI or Batch API.

  • Indicate important runtime events by writing a structured task log with the batch/custom/event field for each event. You can do this when using any method to create a job as part of the definitions of script and container runnables.

Describe a runnable's state

You can configure custom status events that describe a runnable's state by defining a runnable's display name (displayName field). The resulting custom status events vary slightly for different types of runnables:

  • If you define a display name for a container runnable or script runnable, then Batch automatically adds two types of custom status events. The first custom status event indicates whenever a task starts this runnable. The second custom status event indicates whenever a tasks finishes this runnable and the corresponding exit code.

  • If you define a display name for a barrier runnable, then Batch automatically adds a custom status event that indicates whenever a task reaches this barrier.

To create and run a job with custom status events that describes a runnable's state, define the displayName field for one or more runnables using the gcloud CLI or Batch API.

...
"runnables": [
  {
    "displayName":DISPLAY_NAME,
    ...
  }
]
...

For example, a job with custom status events that describes each runnable's state can have a JSON configuration file similar to the following:

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "displayName":"DISPLAY_NAME1",
            "script": {
              "text": "echo Hello world from script 1 for task ${BATCH_TASK_INDEX}"
            }
          },
          {
            "displayName":"DISPLAY_NAME2",
            "barrier": {}
          },
          {
            "displayName":"DISPLAY_NAME3",
            "script": {
              "text": "echo Hello world from script 2 for task ${BATCH_TASK_INDEX}"
            }
          }
        ]
      },
      "taskCount": 3
    }
  ],
  "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}

Replace DISPLAY_NAME1, DISPLAY_NAME2, and DISPLAY_NAME3 with the name of the runnable, which must be unique within the job—for example, script 1, barrier 1, and script 2.

After the example job has finished running, the resulting custom status events for each task are similar to the following:

statusEvents:
  ...
  - description: 'script at index #0 with display name [DISPLAY_NAME1] started.'
    eventTime: '...'
    type: RUNNABLE_EVENT
  - description: 'script at index #0 with display name [DISPLAY_NAME1] finished with exit
      code 0.'
    eventTime: '...'
    type: RUNNABLE_EVENT
  - description: 'barrier at index #2 with display name [DISPLAY_NAME2] reached.'
    eventTime: '...'
    type: BARRIER_REACHED_EVENT
  - description: 'script at index #2 with display name [DISPLAY_NAME3] started.'
    eventTime: '...'
    type: RUNNABLE_EVENT
  - description: 'script at index #2 with display name [DISPLAY_NAME3] finished with exit
      code 0.'
    eventTime: '...'
    type: RUNNABLE_EVENT
  ...

Indicate important runtime events

You can configure custom status events that indicate when an important event happens while a runnable is running by configuring that runnable to write a structured task log that defines a string for the Batch custom status event (batch/custom/event) field.

If a container runnable or script runnable writes a structured task log that defines the batch/custom/event JSON field, it produces a custom status event at that time. Although you might configure the structured task log to include additional fields, the custom status event only includes the string for the batch/custom/event field.

To create and run a job with custom status events that indicate when an important event happens, configure one or more runnables to write a structured log by printing JSON and define the batch/custom/event field as part of the log.

...
"runnables": [
  {
    ...
    "echo '{\"batch/custom/event\":\"EVENT_DESCRIPTION\"}'"
    ...
  }
]
...
"logsPolicy": {
  "destination": "CLOUD_LOGGING"
}
...

For example, a job with custom status events that indicate when an important event happens can have a JSON configuration file similar to the following:

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "script": {
              "text": "sleep 30; echo '{\"batch/custom/event\": \"EVENT_DESCRIPTION\"}'; sleep 30"
            }
          }
        ]
      },
      "taskCount": 3
    }
  ],
  "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}

Replace EVENT_DESCRIPTION with a description for the custom status event—for example, halfway done.

After the example job has finished running, the resulting custom status event for each task is similar to the following:

statusEvents:
  ...
  - description: EVENT_DESCRIPTION
    eventTime: '...'
    type: RUNNABLE_CUSTOM_EVENT
  ...

What's next