Create and run a job that sends Pub/Sub status notifications

This document explains how to create a Batch job that sends Pub/Sub notifications. You can use Pub/Sub to get notifications when job or task state changes, or when a job or task enters a specific state. For more information, see Monitor jobs using notifications.

Before you begin

If you haven't used Batch before, review Get started with Batch and enable Batch by completing the prerequisites for projects and users.
To get the permissions that you need to create and run a job that sends notifications, ask your administrator to grant you the following IAM roles:
- Batch Job Editor (roles/batch.jobsEditor) on the project.
- Service Account User (roles/iam.serviceAccountUser) on the job's service account, which by default is the default Compute Engine service account.
- Pub/Sub Editor (roles/pubsub.editor) on the Pub/Sub topic or project.
For more information about granting roles, see Manage access.

You might also be able to get the required permissions through custom roles or other predefined roles.
To ensure that the job's service account has the necessary permissions to publish Pub/Sub notifications, ask your administrator to grant the job's service account the Pub/Sub Publisher (roles/pubsub.publisher) IAM role on your Pub/Sub topic.

By default, the job's service account is the default Compute Engine service account.
Enable Pub/Sub status notifications for your job.
Optionally, configure Pub/Sub to stream notifications to BigQuery.

Create and run a job that sends notifications

You can create a Batch job that sends Pub/Sub notifications by doing the following:

Use the Google Cloud CLI or REST API to create a job that includes the notifications field and one or more jobNotification objects in the main body of the JSON file:

  "notifications": [
    {
      "pubsubTopic": "projects/PROJECT_ID/topics/TOPIC_ID",
      "message": {
        ATTRIBUTES
      }
    },
  ]

Replace the following:

PROJECT_ID: the project ID of the project that contains the Pub/Sub topic.
TOPIC_ID: the Pub/Sub topic ID of the topic you created when you enabled Pub/Sub notifications.
ATTRIBUTES: the attributes you specify depends on whether you want to receive notifications about a job or all tasks in the job:
- For notifications about all job state changes, specify the following:
```
"type": "JOB_STATE_CHANGED"
```
- For notifications about a specific job state change, specify the following:
```
"type": "JOB_STATE_CHANGED",
"newJobState": "JOB_STATE"
```
  Replace JOB_STATE with one of the following job states:
  - QUEUED
  - SCHEDULED
  - RUNNING
  - SUCCEEDED
  - FAILED
  For more information about job states, see Job lifecycle.
- For notifications about all task state changes, specify the following:
```
"type": "TASK_STATE_CHANGED"
```
- For notifications about specific task state changes, specify the following:
```
"type": "TASK_STATE_CHANGED",
"newTaskState": "TASK_STATE"
```
  Replace TASK_STATE with one of the following task states:
  - PENDING
  - ASSIGNED
  - RUNNING
  - SUCCEEDED
  - FAILED
  For more information about task states, see Job lifecycle.

For example, suppose you want to receive notifications about all job state changes and any time a task fails. To do so, you would add the following to your job's configuration JSON file:

  "notifications": [
    {
      "pubsubTopic": "projects/example-projecet/topics/example-topic",
      "message": {
        "type": "JOB_STATE_CHANGED"
      }
    },
    {
      "pubsubTopic": "projects/example-project/topics/example-topic",
      "message": {
        "type": "TASK_STATE_CHANGED",
        "newTaskState": "FAILED"
      }
    }
  ]

What's next

If you have issues creating or running a job, see Troubleshooting.
View jobs and tasks.
Learn about more job creation options.