Specify the network for a job

This document explains how to specify the network for the VMs that run a job.

You can control connections for the VMs that run a job by specifying a network with the desired access. For example, you might specify a network that allows a job to access required resources or limits access to improve security. Alternatively, if you don't have any networking requirements and don't want to configure networking for a job, skip specifying the network to use the default networking configuration instead.

For more information about networking concepts and when to configure networking, see Batch networking overview.

Before you begin

If you haven't used Batch before, review Get started with Batch and enable Batch by completing the prerequisites for projects and users.
To get the permissions that you need to create a job that runs on a specific network, ask your administrator to grant you the following IAM roles:
- Batch Job Editor (roles/batch.jobsEditor) on the project
- Service Account User (roles/iam.serviceAccountUser) on the job's service account, which by default is the default Compute Engine service account
- To identify the network and subnet: Compute Network Viewer (roles/compute.networkViewer) on the project
For more information about granting roles, see Manage access.

You might also be able to get the required permissions through custom roles or other predefined roles.
Identify the network that you want to use for the job. The network you specify for a job must meet the following requirements:
- The network is a Virtual Private Cloud (VPC) network that is in the same project as the job or is a Shared VPC network that is hosted by or shared with the project for the job.
- The network includes a subnetwork (subnet) in the location where you want to run the job.
- The network allows the access required for your job. For example, if your job requires communication between VMs—such as a job that uses MPI libraries to communicate between tightly coupled tasks—make sure the network has a firewall rule allowing connections between the job's VMs.
  
  Learn how to Configure VPC firewall rules for common use cases.
For more information, see Create and manage VPC networks.
If you want a job to run on a subnet of a Shared VPC network that is hosted by another project, your project's Batch service agent must be granted permission to use that subnet.
To ensure that your project's Batch service agent has the necessary permissions to create a job that runs on a subnet of a Shared VPC network, ask your administrator to grant your project's Batch service agent the Compute Network User (roles/compute.networkUser) IAM role on the Shared VPC subnet.

Caution: If you grant the Batch service agent the Compute Network User (roles/compute.networkUser) IAM role on a Shared VPC subnet, you allow all principals in the project that are allowed to create Batch jobs to do so using this Shared VPC subnet.

For more information, see the documentation for setting up Shared VPC for service accounts.

Create a job that runs on a specific network

Specify the network for a job when you are creating it. Specifically, you need to specify a VPC network and a subnet that is located where you want to run this job.

If you want to use a VM instance template while creating this job, you must specify the network in the VM instance template. Otherwise, use the following steps to specify the network for a job by using the gcloud CLI or Batch API.

gcloud

To create a job that runs on a specific network using the gcloud CLI, select one of the following options:

Use gcloud flags to specify the network for a job
Use JSON fields to specify the network for a job

Use gcloud flags to specify the network for a job

To create a job and use gcloud flags to specify the network for the job, complete the following steps:

Create a JSON file that specifies your job's configuration details.

For example, to create a basic script job, create a JSON file with the following contents.

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "script": {
              "text": "echo Hello world! This is task ${BATCH_TASK_INDEX}. This job has a total of ${BATCH_TASK_COUNT} tasks."
            }
          }
        ]
      },
      "taskCount": 3
    }
  ],
  "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}

Create the job by using the gcloud batch jobs submit command. To specify the network the job, include the --network and --subnetwork flags.
```
gcloud batch jobs submit JOB_NAME \
    --location LOCATION \
    --config JSON_CONFIGURATION_FILE \
    --network projects/HOST_PROJECT_ID/global/networks/NETWORK \
    --subnetwork projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET
```
Replace the following:
- JOB_NAME: the name for this job.
- LOCATION: the location for this job.
- JSON_CONFIGURATION_FILE: the path for the JSON file with the job's configuration details.
- HOST_PROJECT_ID: the project ID of the project for the network you specify:
  - If you are using a Shared VPC network, specify the host project.
  - Otherwise, specify the current project.
- NETWORK: the name of a VPC network in the current project or a Shared VPC network that is hosted by or shared with the current project.
- REGION: the region where the subnet and the VMs for the job are located:
  - If you include the allowedLocations field to specify the allowed location for the VMs for the job, you must specify the same region here.
  - Otherwise, the region must be the same as the location you select for the job (LOCATION).
- SUBNET: the name of a subnet that is part of the VPC network and is located in the same region as the VMs for the job.

Use JSON fields to specify the network for a job

To create a job and use fields in the JSON configuration file to specify the network for the job, complete the following steps:

Create a JSON file that specifies your job's configuration details. To specify the network for the job, include the network and subnetwork fields.

For example, to create a basic script job that runs on a specific network, create a JSON file with the following contents.
```
{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "script": {
              "text": "echo Hello world! This is task ${BATCH_TASK_INDEX}. This job has a total of ${BATCH_TASK_COUNT} tasks."
            }
          }
        ]
      },
      "taskCount": 3
    }
  ],
  "allocationPolicy": {
    "network": {
      "networkInterfaces": [
        {
          "network": "projects/HOST_PROJECT_ID/global/networks/NETWORK",
          "subnetwork": "projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET"
        }
      ]
    }
  },
  "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}
```
Replace the following:
- HOST_PROJECT_ID: the project ID of the project for the network you specify:
  - If you are using a Shared VPC network, specify the host project.
  - Otherwise, specify the current project.
- NETWORK: the name of a VPC network in the current project or a Shared VPC network that is hosted by or shared with the current project.
- REGION: the region where the subnet and the VMs for the job are located:
  - If you include the allowedLocations field to specify the allowed location for the VMs for the job, you must specify the same region here.
  - Otherwise, the region must be the same as the location you select for the job (LOCATION).
- SUBNET: the name of a subnet that is part of the VPC network and is located in the same region as the VMs for the job.
Create the job by using the gcloud batch jobs submit command.
```
gcloud batch jobs submit JOB_NAME \
    --location LOCATION \
    --config JSON_CONFIGURATION_FILE
```
Replace the following:
- JOB_NAME: the name for this job.
- LOCATION: the location for this job.
- JSON_CONFIGURATION_FILE: the path for the JSON file with the job's configuration details.

API

To create a job using the Batch API, use the jobs.create method and specify your job's configuration details. To specify the network for the job, include the network and subnetwork fields.

For example, to create a basic script job that runs on a specific network, make the following POST request:

POST https://batch.googleapis.com/v1/projects/PROJECT_ID/locations/LOCATION/jobs?job_id=JOB_NAME

{
  "taskGroups": [
    {
      "taskSpec": {
        "runnables": [
          {
            "script": {
              "text": "echo Hello world! This is task ${BATCH_TASK_INDEX}. This job has a total of ${BATCH_TASK_COUNT} tasks."
            }
          }
        ]
      },
      "taskCount": 3
    }
  ],
  "allocationPolicy": {
    "network": {
      "networkInterfaces": [
        {
          "network": "projects/HOST_PROJECT_ID/global/networks/NETWORK",
          "subnetwork": "projects/HOST_PROJECT_ID/regions/REGION/subnetworks/SUBNET"
        }
      ]
    }
  },
  "logsPolicy": {
    "destination": "CLOUD_LOGGING"
  }
}

Replace the following:

PROJECT_ID: the project ID of your project.
LOCATION: the location for this job.
JOB_NAME: the name for this job.
HOST_PROJECT_ID: the project ID of the project for the network you specify:
- If you are using a Shared VPC network, specify the host project.
- Otherwise, specify the current project (PROJECT_ID).
NETWORK: the name of a VPC network in the current project or a Shared VPC network that is hosted by or shared with the current project.
REGION: the region where the subnet and the VMs for the job are located:
- If you include the allowedLocations field to specify the allowed location for the VMs for the job, you must specify the same region here.
- Otherwise, the region must be the same as the location you select for the job (LOCATION).
SUBNET: the name of a subnet that is part of the VPC network and is located in the same region as the VMs for the job.

What's next

If you have issues creating or running a job, see Troubleshooting.
Learn more about networking.
Learn more about creating a job.
Learn how to view jobs and tasks.