This document explains how to create a Batch job that sends Pub/Sub notifications. You can use Pub/Sub to get notifications when job or task state changes, or when a job or task enters a specific state. For more information, see Monitor jobs using notifications.
Before you begin
- If you haven't used Batch before, review Get started with Batch and enable Batch by completing the prerequisites for projects and users.
-
To get the permissions that you need to create and run a job that sends notifications, ask your administrator to grant you the following IAM roles:
-
Batch Job Editor (
roles/batch.jobsEditor
) on the project. -
Service Account User (
roles/iam.serviceAccountUser
) on the job's service account, which by default is the default Compute Engine service account. -
Pub/Sub Editor (
roles/pubsub.editor
) on the Pub/Sub topic or project.
For more information about granting roles, see Manage access.
You might also be able to get the required permissions through custom roles or other predefined roles.
-
Batch Job Editor (
-
To ensure that the job's service account has the necessary permissions to publish Pub/Sub notifications, ask your administrator to grant the job's service account the Pub/Sub Publisher (
roles/pubsub.publisher
) IAM role on your Pub/Sub topic.By default, the job's service account is the default Compute Engine service account.
- Enable Pub/Sub status notifications for your job.
- Optionally, configure Pub/Sub to stream notifications to BigQuery.
Create and run a job that sends notifications
You can create a Batch job that sends Pub/Sub notifications by doing the following:
Use the Google Cloud CLI or REST API to
create a job that
includes the notifications
field and one or more
jobNotification
objects
in the main body of the JSON file:
"notifications": [
{
"pubsubTopic": "projects/PROJECT_ID/topics/TOPIC_ID",
"message": {
ATTRIBUTES
}
},
]
Replace the following:
PROJECT_ID
: the project ID of the project that contains the Pub/Sub topic.TOPIC_ID
: the Pub/Sub topic ID of the topic you created when you enabled Pub/Sub notifications.ATTRIBUTES
: the attributes you specify depends on whether you want to receive notifications about a job or all tasks in the job:For notifications about all job state changes, specify the following:
"type": "JOB_STATE_CHANGED"
For notifications about a specific job state change, specify the following:
"type": "JOB_STATE_CHANGED", "newJobState": "JOB_STATE"
Replace
JOB_STATE
with one of the following job states:QUEUED
SCHEDULED
RUNNING
SUCCEEDED
FAILED
For more information about job states, see Job lifecycle.
For notifications about all task state changes, specify the following:
"type": "TASK_STATE_CHANGED"
For notifications about specific task state changes, specify the following:
"type": "TASK_STATE_CHANGED", "newTaskState": "TASK_STATE"
Replace
TASK_STATE
with one of the following task states:PENDING
ASSIGNED
RUNNING
SUCCEEDED
FAILED
For more information about task states, see Job lifecycle.
For example, suppose you want to receive notifications about all job state changes and any time a task fails. To do so, you would add the following to your job's configuration JSON file:
"notifications": [
{
"pubsubTopic": "projects/example-projecet/topics/example-topic",
"message": {
"type": "JOB_STATE_CHANGED"
}
},
{
"pubsubTopic": "projects/example-project/topics/example-topic",
"message": {
"type": "TASK_STATE_CHANGED",
"newTaskState": "FAILED"
}
}
]
What's next
- If you have issues creating or running a job, see Troubleshooting.
- View jobs and tasks.
- Learn about more job creation options.