Try Gemini 1.5 Pro, our most advanced multimodal model in Vertex AI, and see what you can build with a 1M token context window. Try Gemini 1.5 Pro, our most advanced multimodal model in Vertex AI, and see what you can build with a 1M token context window.

Use Private Service Connect to access Vertex AI batch predictions from on-premises

Batch predictions are asynchronous requests that request predictions directly from the model resource without the need to deploy the model to an endpoint.

In this tutorial, you use High-Availability VPN (HA VPN) to send batch prediction requests to a trained model privately, between two Virtual Private Cloud networks that can serve as a basis for multi-cloud and on-premises private connectivity.

This tutorial is intended for enterprise network administrators, data scientists, and researchers who are familiar with Vertex AI, Virtual Private Cloud (VPC), the Google Cloud console, and the Cloud Shell. Familiarity with Vertex AI Workbench is helpful but not required.

Architectural diagram of using Private Service Connect to access batch predictions.

Objectives

Create two Virtual Private Cloud (VPC) networks, as shown in the preceding diagram:
- One (vertex-networking-vpc) is for accessing Google APIs for batch prediction.
- The other (onprem-vpc) represents an on-premises network.
Deploy HA VPN gateways, Cloud VPN tunnels, and Cloud Routers to connect vertex-networking-vpc and onprem-vpc.
Build a Vertex AI batch prediction model and upload it to a Cloud Storage bucket.
Create a Private Service Connect (PSC) endpoint to forward private requests to the Vertex AI batch prediction REST API.
Configure a Cloud Router custom route advertisement in vertex-networking-vpc to announce routes for the Private Service Connect endpoint to onprem-vpc.
Create a Compute Engine VM instance in onprem-vpc to represent a client application (on-prem-client) that sends batch prediction requests privately over HA VPN.

Costs

In this document, you use the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

When you finish the tasks that are described in this document, you can avoid continued billing by deleting the resources that you created. For more information, see Clean up.

Before you begin

In the Google Cloud console, go to the project selector page.

Go to project selector
Select or create a Google Cloud project.
Make sure that billing is enabled for your Google Cloud project.
Open Cloud Shell to execute the commands listed in this tutorial. Cloud Shell is an interactive shell environment for Google Cloud that lets you manage your projects and resources from your web browser.
In Cloud Shell, set the current project to your Google Cloud project ID and store the same project ID into the projectid shell variable:
```
  projectid="PROJECT_ID"
  gcloud config set project ${projectid}
```
Replace PROJECT_ID with your project ID. If necessary, you can locate your project ID in the Google Cloud console. For more information, see Find your project ID.
If you're not the project owner, ask the project owner to grant you the Project IAM Admin (roles/resourcemanager.projectIamAdmin) role. You must have this role to grant IAM roles in the next step.
Grant roles to your Google Account. Run the following command once for each of the following IAM roles: roles/compute.instanceAdmin.v1, roles/compute.networkAdmin, roles/compute.securityAdmin, roles/dns.admin, roles/iap.tunnelResourceAccessor, roles/notebooks.admin, roles/iam.serviceAccountAdmin, roles/iam.serviceAccountUser, roles/servicedirectory.editor, roles/serviceusage.serviceUsageAdmin, roles/storage.admin, roles/aiplatform.admin, roles/aiplatform.user, roles/resourcemanager.projectIamAdmin
```
gcloud projects add-iam-policy-binding PROJECT_ID --member="user:EMAIL_ADDRESS" --role=ROLE
```
- Replace PROJECT_ID with your project ID.
- Replace EMAIL_ADDRESS with your email address.
- Replace ROLE with each individual role.

Enable the DNS, Artifact Registry, IAM, Compute Engine, Notebooks, and Vertex AI APIs:

gcloud services enable dns.googleapis.com artifactregistry.googleapis.com iam.googleapis.com compute.googleapis.com notebooks.googleapis.com aiplatform.googleapis.com

Create the VPC networks

In this section, you create two VPC networks: one for accessing Google APIs for batch prediction, and the other to simulate an on-premises network. In each of the two VPC networks, you create a Cloud Router and Cloud NAT gateway. A Cloud NAT gateway provides outgoing connectivity for Compute Engine virtual machine (VM) instances without external IP addresses.

Create the vertex-networking-vpc VPC network:

gcloud compute networks create vertex-networking-vpc \
  --subnet-mode custom

In the vertex-networking-vpc network, create a subnet named workbench-subnet, with a primary IPv4 range of 10.0.1.0/28:

gcloud compute networks subnets create workbench-subnet \
  --range=10.0.1.0/28 \
  --network=vertex-networking-vpc \
  --region=us-central1 \
  --enable-private-ip-google-access

Create the VPC network to simulate the on-premises network (onprem-vpc):
```
gcloud compute networks create onprem-vpc \
  --subnet-mode custom
```

In the onprem-vpc network, create a subnet named onprem-vpc-subnet1, with a primary IPv4 range of 172.16.10.0/29:

gcloud compute networks subnets create onprem-vpc-subnet1 \
  --network onprem-vpc \
  --range 172.16.10.0/29 \
  --region us-central1

Verify that the VPC networks are correctly configured

In the Google Cloud console, go to the Networks in current project tab on the VPC networks page.

Go to VPC networks
In the list of VPC networks, verify that the two networks have been created: vertex-networking-vpc and onprem-vpc.
Click the Subnets in current project tab.
In the list of VPC subnets, verify that the workbench-subnet and onprem-vpc-subnet1 subnets have been created.

Configure hybrid connectivity

In this section you create two HA VPN gateways that are connected to each other. One resides in the vertex-networking-vpc VPC network. The other resides in the onprem-vpc VPC network. Each gateway contains a Cloud Router and a pair of VPN tunnels.

Create the HA VPN gateways

In Cloud Shell, create the HA VPN gateway for the vertex-networking-vpc VPC network:

gcloud compute vpn-gateways create vertex-networking-vpn-gw1 \
   --network vertex-networking-vpc \
   --region us-central1

Create the HA VPN gateway for the onprem-vpc VPC network:

gcloud compute vpn-gateways create onprem-vpn-gw1 \
   --network onprem-vpc \
   --region us-central1

In the Google Cloud console, go to the Cloud VPN Gateways tab on the VPN page.

Go to VPN
Verify that the two gateways (vertex-networking-vpn-gw1 and onprem-vpn-gw1) have been created and that each gateway has two interface IP addresses.

Create Cloud Routers and Cloud NAT gateways

In each of the two VPC networks, you create two Cloud Routers: one general and one regional. In each of the regional Cloud Routers, you create a Cloud NAT gateway. Cloud NAT gateways provide outgoing connectivity for Compute Engine virtual machine (VM) instances that don't have external IP addresses.

In Cloud Shell, create a Cloud Router for the vertex-networking-vpc VPC network:

gcloud compute routers create vertex-networking-vpc-router1 \
   --region us-central1\
   --network vertex-networking-vpc \
   --asn 65001

Create a Cloud Router for the onprem-vpc VPC network:

gcloud compute routers create onprem-vpc-router1 \
   --region us-central1\
   --network onprem-vpc\
   --asn 65002

Create a regional Cloud Router for the vertex-networking-vpc VPC network:

gcloud compute routers create cloud-router-us-central1-vertex-nat \
  --network vertex-networking-vpc \
  --region us-central1

Configure a Cloud NAT gateway on the regional Cloud Router:

gcloud compute routers nats create cloud-nat-us-central1 \
  --router=cloud-router-us-central1-vertex-nat \
  --auto-allocate-nat-external-ips \
  --nat-all-subnet-ip-ranges \
  --region us-central1

Create a regional Cloud Router for the onprem-vpc VPC network:

gcloud compute routers create cloud-router-us-central1-onprem-nat \
  --network onprem-vpc \
  --region us-central1

Configure a Cloud NAT gateway on the regional Cloud Router:

gcloud compute routers nats create cloud-nat-us-central1-on-prem \
  --router=cloud-router-us-central1-onprem-nat \
  --auto-allocate-nat-external-ips \
  --nat-all-subnet-ip-ranges \
  --region us-central1

In the Google Cloud console, go to the Cloud Routers page.

Go to Cloud Routers
In the Cloud Routers list, verify that the following routers have been created:
- cloud-router-us-central1-onprem-nat
- cloud-router-us-central1-vertex-nat
- onprem-vpc-router1
- vertex-networking-vpc-router1
You may need to refresh the Google Cloud console browser tab to see the new values.
In the Cloud Routers list, click cloud-router-us-central1-vertex-nat.
In the Router details page, verify that the cloud-nat-us-central1 Cloud NAT gateway has been created.
Click the back arrow to return to the Cloud Routers page.
In the router list, click cloud-router-us-central1-onprem-nat.
In the Router details page, verify that the cloud-nat-us-central1-on-prem Cloud NAT gateway has been created.

Create VPN tunnels

In Cloud Shell, in the vertex-networking-vpc network, create a VPN tunnel called vertex-networking-vpc-tunnel0:

gcloud compute vpn-tunnels create vertex-networking-vpc-tunnel0 \
  --peer-gcp-gateway onprem-vpn-gw1 \
  --region us-central1 \
  --ike-version 2 \
  --shared-secret [ZzTLxKL8fmRykwNDfCvEFIjmlYLhMucH] \
  --router vertex-networking-vpc-router1 \
  --vpn-gateway vertex-networking-vpn-gw1 \
  --interface 0

In the vertex-networking-vpc network, create a VPN tunnel called vertex-networking-vpc-tunnel1:

gcloud compute vpn-tunnels create vertex-networking-vpc-tunnel1 \
  --peer-gcp-gateway onprem-vpn-gw1 \
  --region us-central1 \
  --ike-version 2 \
  --shared-secret [bcyPaboPl8fSkXRmvONGJzWTrc6tRqY5] \
  --router vertex-networking-vpc-router1 \
  --vpn-gateway vertex-networking-vpn-gw1 \
  --interface 1

In the onprem-vpc network, create a VPN tunnel called onprem-vpc-tunnel0:

gcloud compute vpn-tunnels create onprem-vpc-tunnel0 \
  --peer-gcp-gateway vertex-networking-vpn-gw1 \
  --region us-central1\
  --ike-version 2 \
  --shared-secret [ZzTLxKL8fmRykwNDfCvEFIjmlYLhMucH] \
  --router onprem-vpc-router1 \
  --vpn-gateway onprem-vpn-gw1 \
  --interface 0

In the onprem-vpc network, create a VPN tunnel called onprem-vpc-tunnel1:

gcloud compute vpn-tunnels create onprem-vpc-tunnel1 \
  --peer-gcp-gateway vertex-networking-vpn-gw1 \
  --region us-central1\
  --ike-version 2 \
  --shared-secret [bcyPaboPl8fSkXRmvONGJzWTrc6tRqY5] \
  --router onprem-vpc-router1 \
  --vpn-gateway onprem-vpn-gw1 \
  --interface 1

In the Google Cloud console, go to the VPN page.

Go to VPN
In the list of VPN tunnels, verify that the four VPN tunnels have been created.

Establish BGP sessions

Cloud Router uses Border Gateway Protocol (BGP) to exchange routes between your VPC network (in this case, vertex-networking-vpc) and your on-premises network (represented by onprem-vpc). On Cloud Router, you configure an interface and a BGP peer for your on-premises router. The interface and BGP peer configuration together form a BGP session. In this section you create two BGP sessions for vertex-networking-vpc and two for onprem-vpc.

Once you've configured the interfaces and BGP peers between your routers, they will automatically start exchanging routes.

Establish BGP sessions for `vertex-networking-vpc`

In Cloud Shell, in the vertex-networking-vpc network, create a BGP interface for vertex-networking-vpc-tunnel0:

gcloud compute routers add-interface vertex-networking-vpc-router1 \
  --interface-name if-tunnel0-to-onprem \
  --ip-address 169.254.0.1 \
  --mask-length 30 \
  --vpn-tunnel vertex-networking-vpc-tunnel0 \
  --region us-central1

In the vertex-networking-vpc network, create a BGP peer for bgp-onprem-tunnel0:

gcloud compute routers add-bgp-peer vertex-networking-vpc-router1 \
  --peer-name bgp-onprem-tunnel0 \
  --interface if-tunnel0-to-onprem \
  --peer-ip-address 169.254.0.2 \
  --peer-asn 65002 \
  --region us-central1

In the vertex-networking-vpc network, create a BGP interface for vertex-networking-vpc-tunnel1:

gcloud compute routers add-interface vertex-networking-vpc-router1 \
  --interface-name if-tunnel1-to-onprem \
  --ip-address 169.254.1.1 \
  --mask-length 30 \
  --vpn-tunnel vertex-networking-vpc-tunnel1 \
  --region us-central1

In the vertex-networking-vpc network, create a BGP peer for bgp-onprem-tunnel1:

gcloud compute routers add-bgp-peer vertex-networking-vpc-router1 \
  --peer-name bgp-onprem-tunnel1 \
  --interface if-tunnel1-to-onprem \
  --peer-ip-address 169.254.1.2 \
  --peer-asn 65002 \
  --region us-central1

Establish BGP sessions for `onprem-vpc`

In the onprem-vpc network, create a BGP interface for onprem-vpc-tunnel0:

gcloud compute routers add-interface onprem-vpc-router1 \
  --interface-name if-tunnel0-to-vertex-networking-vpc \
  --ip-address 169.254.0.2 \
  --mask-length 30 \
  --vpn-tunnel onprem-vpc-tunnel0 \
  --region us-central1

In the onprem-vpc network, create a BGP peer for bgp-vertex-networking-vpc-tunnel0:

gcloud compute routers add-bgp-peer onprem-vpc-router1 \
  --peer-name bgp-vertex-networking-vpc-tunnel0 \
  --interface if-tunnel0-to-vertex-networking-vpc \
  --peer-ip-address 169.254.0.1 \
  --peer-asn 65001 \
  --region us-central1

In the onprem-vpc network, create a BGP interface for onprem-vpc-tunnel1:

gcloud compute routers add-interface   onprem-vpc-router1  \
  --interface-name if-tunnel1-to-vertex-networking-vpc \
  --ip-address 169.254.1.2 \
  --mask-length 30 \
  --vpn-tunnel onprem-vpc-tunnel1 \
  --region us-central1

In the onprem-vpc network, create a BGP peer for bgp-vertex-networking-vpc-tunnel1:

gcloud compute routers add-bgp-peer onprem-vpc-router1 \
  --peer-name bgp-vertex-networking-vpc-tunnel1 \
  --interface if-tunnel1-to-vertex-networking-vpc \
  --peer-ip-address 169.254.1.1 \
  --peer-asn 65001 \
  --region us-central1

Validate BGP session creation

In the Google Cloud console, go to the VPN page.

Go to VPN
In the list of VPN tunnels, verify that the value in the BGP session status column for each of the tunnels has changed from Configure BGP session to BGP established. You may need to refresh the Google Cloud console browser tab to see the new values.

Validate the `vertex-networking-vpc` learned routes

In the Google Cloud console, go to the VPC networks page.

Go to VPC networks
In the list of VPC networks, click vertex-networking-vpc.
Click the Routes tab.
Select us-central1 (Iowa) in the Region list and click View.
In the Destination IP range column, verify that the onprem-vpc-subnet1 subnet's IP range (172.16.10.0/29) appears twice.

Validate the `onprem-vpc` learned routes

Click the back arrow to return to the VPC networks page.
In the list of VPC networks, click onprem-vpc.
Click the Routes tab.
Select us-central1 (Iowa) in the Region list and click View.
In the Destination IP range column, verify that the workbench-subnet subnet IP range (10.0.1.0/28) appears twice.

Create the Private Service Connect consumer endpoint

In Cloud Shell, reserve a consumer endpoint IP address that will be used to access Google APIs:

gcloud compute addresses create psc-googleapi-ip \
  --global \
  --purpose=PRIVATE_SERVICE_CONNECT \
  --addresses=192.168.0.1 \
  --network=vertex-networking-vpc

Create a forwarding rule to connect the endpoint to Google APIs and services.

gcloud compute forwarding-rules create pscvertex \
 --global \
 --network=vertex-networking-vpc\
 --address=psc-googleapi-ip \
 --target-google-apis-bundle=all-apis

Create custom route advertisements for `vertex-networking-vpc`

In this section, you create a custom route advertisement for vertex-networking-vpc-router1 (the Cloud Router for vertex-networking-vpc) to advertise the PSC endpoint's IP address to the onprem-vpc network.

In the Google Cloud console, go to the Cloud Routers page.

Go to Cloud Routers
In the Cloud Router list, click vertex-networking-vpc-router1.
On the Router details page, click Edit.
In the Advertised routes section, for Routes, select Create custom routes.
Select the Advertise all subnets visible to the Cloud Router checkbox to continue advertising the subnets available to the Cloud Router. Enabling this option mimics the behavior of Cloud Router in default advertisement mode.
Click Add a custom route.
For Source, select Custom IP range.
For IP address range, enter the following IP address:
```
192.168.0.1
```

For Description, enter the following text:

Custom route to advertise Private Service Connect endpoint IP address

Click Done, and then click Save.

Validate that `onprem-vpc` has learned the advertised routes

In the Google Cloud console, go to the Routes page.

Go to Routes
On the Effective routes tab, do the following:
1. For Network, choose onprem-vpc.
2. For Region, choose us-central1 (Iowa).
3. Click View.
4. In the list of routes, verify that there are entries whose names begin with onprem-vpc-router1-bgp-vertex-networking-vpc-tunnel0 and onprem-vpc-router1-bgp-vfertex-networking-vpc-tunnel1, and that both of them have a Destination IP range of 192.168.0.1.
  
  If these entries don't appear right away, wait a few minutes, and then refresh the Google Cloud console browser tab.

Create a VM in `onprem-vpc` that uses a user-managed service account

In this section, you create a VM instance that simulates an on-premises client application that sends batch prediction requests. Following Compute Engine and IAM best practices, this VM uses a user-managed service account instead of the Compute Engine default service account.

Create a user-managed service account

In Cloud Shell, run the following commands, replacing PROJECT_ID with your project ID:
```
projectid=PROJECT_ID
gcloud config set project ${projectid}
```

Create a service account named onprem-user-managed-sa:

gcloud iam service-accounts create onprem-user-managed-sa \
  --display-name="onprem-user-managed-sa-onprem-client"

Assign the Vertex AI User (roles/aiplatform.user) role to the service account:

gcloud projects add-iam-policy-binding $projectid \
  --member="serviceAccount:onprem-user-managed-sa@$projectid.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

Assign the Storage Object Viewer (storage.objectViewer) role to the service account:

gcloud projects add-iam-policy-binding $projectid \
  --member="serviceAccount:onprem-user-managed-sa@$projectid.iam.gserviceaccount.com" \
  --role="roles/storage.objectViewer"

Create the `on-prem-client` VM instance

The VM instance that you create doesn't have an external IP address and doesn't permit direct access over the internet. To enable administrative access to the VM, you use Identity-Aware Proxy (IAP) TCP forwarding.

In Cloud Shell, create the on-prem-client VM instance:

gcloud compute instances create on-prem-client \
  --zone=us-central1-a \
  --image-family=debian-11 \
  --image-project=debian-cloud \
  --subnet=onprem-vpc-subnet1 \
  --scopes=https://www.googleapis.com/auth/cloud-platform \
  --no-address \
  --shielded-secure-boot \
  --service-account=onprem-user-managed-sa@$projectid.iam.gserviceaccount.com \
  --metadata startup-script="#! /bin/bash
    sudo apt-get update
    sudo apt-get install tcpdump dnsutils -y"

Create a firewall rule to allow IAP to connect to your VM instance:

gcloud compute firewall-rules create ssh-iap-on-prem-vpc \
  --network onprem-vpc \
  --allow tcp:22 \
  --source-ranges=35.235.240.0/20

Validate public access to Vertex AI API

In this section, you use the dig utility to perform a DNS lookup from the on-prem-client VM instance to the Vertex AI API (us-central1-aiplatform.googleapis.com). The dig output shows that the default access uses only public VIPs to access the Vertex AI API.

In the next section, you'll configure private access to the Vertex AI API.

In Cloud Shell, log into the on-prem-client VM instance using IAP:

gcloud compute ssh on-prem-client \
  --zone=us-central1-a \
  --tunnel-through-iap

In the on-prem-client VM instance, run the dig command:

dig us-central1-aiplatform.googleapis.com

You should see dig output similar to the following, where the IP addresses in the answer section are public IP addresses:

; <<>> DiG 9.16.44-Debian <<>> us-central1.aiplatfom.googleapis.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 42506
;; flags: qr rd ra; QUERY: 1, ANSWER: 16, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;us-central1.aiplatfom.googleapis.com. IN A

;; ANSWER SECTION:
us-central1.aiplatfom.googleapis.com. 300 IN A  173.194.192.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.250.152.95
us-central1.aiplatfom.googleapis.com. 300 IN A  172.217.219.95
us-central1.aiplatfom.googleapis.com. 300 IN A  209.85.146.95
us-central1.aiplatfom.googleapis.com. 300 IN A  209.85.147.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.250.125.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.250.136.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.250.148.95
us-central1.aiplatfom.googleapis.com. 300 IN A  209.85.200.95
us-central1.aiplatfom.googleapis.com. 300 IN A  209.85.234.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.251.171.95
us-central1.aiplatfom.googleapis.com. 300 IN A  108.177.112.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.250.128.95
us-central1.aiplatfom.googleapis.com. 300 IN A  142.251.6.95
us-central1.aiplatfom.googleapis.com. 300 IN A  172.217.212.95
us-central1.aiplatfom.googleapis.com. 300 IN A  74.125.124.95

;; Query time: 8 msec
;; SERVER: 169.254.169.254#53(169.254.169.254)
;; WHEN: Wed Sep 27 04:10:16 UTC 2023
;; MSG SIZE  rcvd: 321

Configure and validate private access to the Vertex AI API

In this section, you configure private access to the Vertex AI API so that when you send batch prediction requests, they are redirected to your PSC endpoint. The PSC endpoint in turn forwards these private requests to the Vertex AI batch prediction REST API.

Update the `/etc/hosts` file to point to the PSC endpoint

In this step, you add a line to the /etc/hosts file that causes requests sent to the public service endpoint (us-central1-aiplatform.googleapis.com) to be redirected to the PSC endpoint (192.168.0.1).

In the on-prem-client VM instance, use a text editor such as vim or nano to open the /etc/hosts file:
```
sudo vim /etc/hosts
```

Add the following line to the file:

192.168.0.1 us-central1-aiplatform.googleapis.com

This line assigns the PSC endpoint's IP address (192.168.0.1) to the fully qualified domain name for the Vertex AI Google API (us-central1-aiplatform.googleapis.com).

The edited file should look like this:

127.0.0.1       localhost
::1             localhost ip6-localhost ip6-loopback
ff02::1         ip6-allnodes
ff02::2         ip6-allrouters

192.168.0.1 us-central1-aiplatform.googleapis.com  # Added by you
172.16.10.6 on-prem-client.us-central1-a.c.vertex-genai-400103.internal on-prem-client  # Added by Google
169.254.169.254 metadata.google.internal  # Added by Google

Save the file as follows:
- If you're using vim, press the Esc key, and then type :wq to save the file and exit.
- If you're using nano, type Control+O and press Enter to save the file, and then type Control+X to exit.
Ping the Vertex AI endpoint as follows:
```
ping us-central1-aiplatform.googleapis.com
```
The ping command should return the following output. 192.168.0.1 is the PSC endpoint IP address:
```
PING us-central1-aiplatform.googleapis.com (192.168.0.1) 56(84) bytes of data.
```
Type Control+C to exit from ping.
Type exit to exit from the on-prem-client VM instance.

Create a user-managed service account for Vertex AI Workbench in the `vertex-networking-vpc`

In this section, to control access to your Vertex AI Workbench instance, you create a user-managed service account and then assign IAM roles to the service account. When you create the instance, you specify the service account.

In Cloud Shell, run the following commands, replacing PROJECT_ID with your project ID:
```
projectid=PROJECT_ID
gcloud config set project ${projectid}
```

Create a service account named workbench-sa:

gcloud iam service-accounts create workbench-sa \
  --display-name="workbench-sa"

Assign the Vertex AI User (roles/aiplatform.user) IAM role to the service account:

gcloud projects add-iam-policy-binding $projectid \
  --member="serviceAccount:workbench-sa@$projectid.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

Assign the BigQuery User (roles/bigquery.user) IAM role to the service account:

gcloud projects add-iam-policy-binding $projectid \
  --member="serviceAccount:workbench-sa@$projectid.iam.gserviceaccount.com" \
  --role="roles/bigquery.user"

Assign the Storage Admin (roles/storage.admin) IAM role to the service account:

gcloud projects add-iam-policy-binding $projectid \
  --member="serviceAccount:workbench-sa@$projectid.iam.gserviceaccount.com" \
  --role="roles/storage.admin"

Assign the Logs Viewer (roles/logging.viewer) IAM role to the service account:

gcloud projects add-iam-policy-binding $projectid \
  --member="serviceAccount:workbench-sa@$projectid.iam.gserviceaccount.com" \
  --role="roles/logging.viewer"

Create the Vertex AI Workbench instance

In Cloud Shell, create a Vertex AI Workbench instance, specifying the workbench-sa service account:

gcloud workbench instances create workbench-tutorial \
  --vm-image-project=deeplearning-platform-release \
  --vm-image-family=common-cpu-notebooks \
  --machine-type=n1-standard-4 \
  --location=us-central1-a \
  --subnet-region=us-central1 \
  --shielded-secure-boot=True \
  --subnet=workbench-subnet \
  --disable-public-ip \
  --service-account-email=workbench-sa@$projectid.iam.gserviceaccount.com

In the Google Cloud console, go to the Instances tab on the Vertex AI Workbench page.

Go to Vertex AI Workbench
Next to your Vertex AI Workbench instance's name (workbench-tutorial), click Open JupyterLab.

Your Vertex AI Workbench instance opens JupyterLab.
Select File > New > Notebook.
From the Select Kernel menu, select Python 3 (Local) and click Select.
When your new notebook opens, there is a default code cell where you can enter code. It looks like [ ]: followed by a text field. The text field is where you paste your code.

To install the Vertex AI SDK for Python, paste the following code into the cell and click Run the selected cells and advance:
```
!pip3 install --upgrade google-cloud-bigquery scikit-learn==1.2
```
In this step and each of the following ones, add a new code cell (if necessary) by clicking Insert a cell below, paste the code into the cell, and then click Run the selected cells and advance.

To use the newly installed packages in this Jupyter runtime, you need to restart the runtime:
```
# Restart kernel after installs so that your environment can access the new packages
import IPython

app = IPython.Application.instance()
app.kernel.do_shutdown(True)
```
Set the following environment variables in your JupyterLab notebook, replacing PROJECT_ID with your project ID.
```
# set project ID and location
PROJECT_ID = "PROJECT_ID"
REGION = "us-central1"
```

Create a Cloud Storage bucket for staging the training job:

BUCKET_NAME = f"{PROJECT_ID}-ml-staging"
BUCKET_URI = f"gs://{BUCKET_NAME}"
!gsutil mb -l {REGION} -p {PROJECT_ID} {BUCKET_URI}

Prepare the training data

In this section, you prepare data to be used to train a prediction model.

In your JupyterLab notebook, create a BigQuery client:

from google.cloud import bigquery
bq_client = bigquery.Client(project=PROJECT_ID)

Fetch data from the BigQuery ml_datasets public dataset:

DATA_SOURCE = "bigquery-public-data.ml_datasets.census_adult_income"
# Define the SQL query to fetch the dataset
query = f"""
SELECT * FROM `{DATA_SOURCE}` LIMIT 20000
"""
# Download the dataset to a dataframe
df = bq_client.query(query).to_dataframe()
df.head()

Use the sklearn library to split the data for training and testing:

from sklearn.model_selection import train_test_split
# Split the dataset
X_train, X_test = train_test_split(df, test_size=0.3, random_state=43)
# Print the shapes of train and test sets
print(X_train.shape, X_test.shape)

Export the train and test dataframes to CSV files in the staging bucket:

X_train.to_csv(f"{BUCKET_URI}/train.csv",index=False, quoting=1, quotechar='"')
X_test[[i for i in X_test.columns if i != "income_bracket"]].iloc[:20].to_csv(f"{BUCKET_URI}/test.csv",index=False,quoting=1, quotechar='"')

Prepare the training application

In this section includes you create and build the Python training application and save it to the staging bucket.

In your JupyterLab notebook, create a new folder for the training application files:
```
!mkdir -p training_package/trainer
```
You should now see a folder named training_package in the JupyterLab navigation menu.

Define the features, target, label, and steps for training and exporting the model to a file:

%%writefile training_package/trainer/task.py
from sklearn.ensemble import RandomForestClassifier
from sklearn.feature_selection import SelectKBest
from sklearn.pipeline import FeatureUnion, Pipeline
from sklearn.preprocessing import LabelBinarizer
import pandas as pd
import argparse
import joblib
import os

TARGET = "income_bracket"
# Define the feature columns that you use from the dataset
COLUMNS = (
  "age",
  "workclass",
  "functional_weight",
  "education",
  "education_num",
  "marital_status",
  "occupation",
  "relationship",
  "race",
  "sex",
  "capital_gain",
  "capital_loss",
  "hours_per_week",
  "native_country",
)

# Categorical columns are columns that have string values and
# need to be turned into a numerical value to be used for training
CATEGORICAL_COLUMNS = (
  "workclass",
  "education",
  "marital_status",
  "occupation",
  "relationship",
  "race",
  "sex",
  "native_country",
)

# load the arguments
parser = argparse.ArgumentParser()
parser.add_argument('--training-dir', dest='training_dir', default=os.getenv('AIP_MODEL_DIR'), type=str,help='get the staging directory')
args = parser.parse_args()

# Load the training data
X_train = pd.read_csv(os.path.join(args.training_dir,"train.csv"))
# Remove the column we are trying to predict ('income-level') from our features list
# Convert the Dataframe to a lists of lists
train_features = X_train.drop(TARGET, axis=1).to_numpy().tolist()
# Create our training labels list, convert the Dataframe to a lists of lists
train_labels = X_train[TARGET].to_numpy().tolist()

# Since the census data set has categorical features, we need to convert
# them to numerical values. We'll use a list of pipelines to convert each
# categorical column and then use FeatureUnion to combine them before calling
# the RandomForestClassifier.
categorical_pipelines = []

# Each categorical column needs to be extracted individually and converted to a numerical value.
# To do this, each categorical column will use a pipeline that extracts one feature column via
# SelectKBest(k=1) and a LabelBinarizer() to convert the categorical value to a numerical one.
# A scores array (created below) will select and extract the feature column. The scores array is
# created by iterating over the COLUMNS and checking if it is a CATEGORICAL_COLUMN.
for i, col in enumerate(COLUMNS):
   if col in CATEGORICAL_COLUMNS:
      # Create a scores array to get the individual categorical column.
      # Example:
      #  data = [39, 'State-gov', 77516, 'Bachelors', 13, 'Never-married', 'Adm-clerical',
      #         'Not-in-family', 'White', 'Male', 2174, 0, 40, 'United-States']
      #  scores = [0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
      #
      # Returns: [['Sate-gov']]
      scores = []
      # Build the scores array
      for j in range(len(COLUMNS)):
         if i == j:  # This column is the categorical column we want to extract.
            scores.append(1)  # Set to 1 to select this column
         else:  # Every other column should be ignored.
            scores.append(0)
      skb = SelectKBest(k=1)
      skb.scores_ = scores
      # Convert the categorical column to a numerical value
      lbn = LabelBinarizer()
      r = skb.transform(train_features)
      lbn.fit(r)
      # Create the pipeline to extract the categorical feature
      categorical_pipelines.append(
         (
            "categorical-{}".format(i),
            Pipeline([("SKB-{}".format(i), skb), ("LBN-{}".format(i), lbn)]),
         )
      )

# Create pipeline to extract the numerical features
skb = SelectKBest(k=6)
# From COLUMNS use the features that are numerical
skb.scores_ = [1, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0]
categorical_pipelines.append(("numerical", skb))

# Combine all the features using FeatureUnion
preprocess = FeatureUnion(categorical_pipelines)

# Create the classifier
classifier = RandomForestClassifier()

# Transform the features and fit them to the classifier
classifier.fit(preprocess.transform(train_features), train_labels)

# Create the overall model as a single pipeline
pipeline = Pipeline([("union", preprocess), ("classifier", classifier)])

# Save the model pipeline
joblib.dump(pipeline, os.path.join(args.training_dir,"model.joblib"))

Create an __init__.py file in each subdirectory to make it a package:

!touch training_package/__init__.py
!touch training_package/trainer/__init__.py

Create a Python package setup script:

%%writefile training_package/setup.py
from setuptools import find_packages
from setuptools import setup

setup(
   name='trainer',
   version='0.1',
   packages=find_packages(),
   include_package_data=True,
   description='Training application package for census income classification.'
)

Use the sdist command to create the source distribution of the training application:
```
!cd training_package && python setup.py sdist --formats=gztar
```

Copy the Python package to the staging bucket:

!gsutil cp training_package/dist/trainer-0.1.tar.gz $BUCKET_URI/

Verify that the staging bucket contains three files:

!gsutil ls $BUCKET_URI

The output should be:

gs://$BUCKET_NAME/test.csv
gs://$BUCKET_NAME/train.csv
gs://$BUCKET_NAME/trainer-0.1.tar.gz

Train the model

In this section, you train the model by creating and running a custom training job.

In your JupyterLab notebook, run the following command to create a custom training job:

!gcloud ai custom-jobs create --display-name=income-classification-training-job \
   --project=$PROJECT_ID \
   --worker-pool-spec=replica-count=1,machine-type='e2-highmem-2',executor-image-uri='us-docker.pkg.dev/vertex-ai/training/sklearn-cpu.1-0:latest',python-module=trainer.task \
   --python-package-uris=$BUCKET_URI/trainer-0.1.tar.gz \
   --args="--training-dir","/gcs/$BUCKET_NAME" \
   --region=$REGION

The output should look similar to the following. The first number in each custom job path is the project number (PROJECT_NUMBER). The second number is the custom job ID (CUSTOM_JOB_ID). Make a note of these numbers so that you can use them in the next step.

Using endpoint [https://us-central1-aiplatform.googleapis.com/]
CustomJob [projects/721032480027/locations/us-central1/customJobs/1100328496195960832] is submitted successfully.

Your job is still active. You may view the status of your job with the command

$ gcloud ai custom-jobs describe projects/721032480027/locations/us-central1/customJobs/1100328496195960832

or continue streaming the logs with the command

$ gcloud ai custom-jobs stream-logs projects/721032480027/locations/us-central1/customJobs/1100328496195960832

Run the custom training job and show progress by streaming logs from the job as it runs:
```
!gcloud ai custom-jobs stream-logs projects/PROJECT_NUMBER/locations/us-central1/customJobs/CUSTOM_JOB_ID
```
Replace the following values:
- PROJECT_NUMBER: the project number from the output of the previous command
- CUSTOM_JOB_ID: the custom job ID from the output of the previous command
Your custom training job is now running. It takes about 10 minutes to complete.

When the job has completed, you can import the model from the staging bucket to Vertex AI Model Registry.

Import the model

Your custom training job uploads the trained model to the staging bucket. When the job has completed, you can import the model from the bucket to Vertex AI Model Registry.

In your JupyterLab notebook, import the model by running the following command:

!gcloud ai models upload --container-image-uri="us-docker.pkg.dev/vertex-ai/prediction/sklearn-cpu.1-2:latest" \
   --display-name=income-classifier-model \
   --artifact-uri=$BUCKET_URI \
   --project=$PROJECT_ID \
   --region=$REGION

List the Vertex AI models in the project as follows:
```
!gcloud ai models list --region=us-central1
```
The output should look like the following. If two or more models are listed, the first one in the list is the one you most recently imported.

Make a note of the value in the MODEL_ID column. You need it to create the batch prediction request.
```
Using endpoint [https://us-central1-aiplatform.googleapis.com/]
MODEL_ID             DISPLAY_NAME
1871528219660779520  income-classifier-model
```
Alternatively, you can list the models in your project as follows:

In the Google Cloud console, in the Vertex AI section, go to the Vertex AI Model Registry page.

Go to the Vertex AI Model Registry page

To view the model IDs and other details for a model, click the model name, and then click the Version Details tab.

Get batch predictions from the model

Now you can request batch predictions from the model. The batch prediction requests are made from the on-prem-client VM instance.

Create the batch prediction request

In this step, you use sshto log into the on-prem-client VM instance. In the VM instance, you create a text file named request.json that contains the payload for a sample curl request that you send to your model to get batch predictions.

In Cloud Shell, run the following commands, replacing PROJECT_ID with your project ID:
```
projectid=PROJECT_ID
gcloud config set project ${projectid}
```

Log into the on-prem-client VM instance using ssh:

gcloud compute ssh on-prem-client \
  --project=$projectid \
  --zone=us-central1-a

In the on-prem-client VM instance, use a text editor such as vim or nano to create a new file named request.json that contains the following text:

{
  "displayName": "income-classification-batch-job",
  "model": "projects/PROJECT_ID/locations/us-central1/models/MODEL_ID",
  "inputConfig": {
    "instancesFormat": "csv",
    "gcsSource": {
      "uris": ["BUCKET_URI/test.csv"]
    }
  },
  "outputConfig": {
    "predictionsFormat": "jsonl",
    "gcsDestination": {
      "outputUriPrefix": "BUCKET_URI"
    }
  },
  "dedicatedResources": {
    "machineSpec": {
      "machineType": "n1-standard-4",
      "acceleratorCount": "0" 
    },
    "startingReplicaCount": 1,
    "maxReplicaCount": 2
  }
}

Replace the following values:

PROJECT_ID: your project ID
MODEL_ID: the model ID for your model
BUCKET_URI: the URI for the storage bucket where you staged your model

Run the following command to send the batch prediction request:

curl -X POST \
   -H "Authorization: Bearer $(gcloud auth print-access-token)" \
   -H "Content-Type: application/json; charset=utf-8" \
   -d @request.json \
   "https://us-central1-aiplatform.googleapis.com/v1/projects/PROJECT_ID/locations/us-central1/batchPredictionJobs"

Replace PROJECT_ID with your project ID.

You should see the following line in the response:

"state": "JOB_STATE_PENDING"

Your batch prediction job is now running asynchronously. It takes about 20 minutes to run.

In the Google Cloud console, in the Vertex AI section, go to the Batch predictions page.

Go to the Batch predictions page

While the batch prediction job is running, its status is Running. When it is finished, its status changes to Finished.
Click the name of your batch prediction job (income-classification-batch-job), and then click the Export location link in the details page to view the output files for your batch job in Cloud Storage.

Alternatively, you can click the View prediction output on Cloud Storage icon (between the Last updated column and the Actions menu).

Click the prediction.results-00000-of-00002 or prediction.results-00001-of-00002 file link, and then click the Authenticated URL link to open the file.

Your batch prediction job output should look similar to this example:

{"instance": ["27", " Private", "391468", " 11th", "7", " Divorced", " Craft-repair", " Own-child", " White", " Male", "0", "0", "40", " United-States"], "prediction": " <=50K"}
{"instance": ["47", " Self-emp-not-inc", "192755", " HS-grad", "9", " Married-civ-spouse", " Machine-op-inspct", " Wife", " White", " Female", "0", "0", "20", " United-States"], "prediction": " <=50K"}
{"instance": ["32", " Self-emp-not-inc", "84119", " HS-grad", "9", " Married-civ-spouse", " Craft-repair", " Husband", " White", " Male", "0", "0", "45", " United-States"], "prediction": " <=50K"}
{"instance": ["32", " Private", "236543", " 12th", "8", " Divorced", " Protective-serv", " Own-child", " White", " Male", "0", "0", "54", " Mexico"], "prediction": " <=50K"}
{"instance": ["60", " Private", "160625", " HS-grad", "9", " Married-civ-spouse", " Prof-specialty", " Husband", " White", " Male", "5013", "0", "40", " United-States"], "prediction": " <=50K"}
{"instance": ["34", " Local-gov", "22641", " HS-grad", "9", " Never-married", " Protective-serv", " Not-in-family", " Amer-Indian-Eskimo", " Male", "0", "0", "40", " United-States"], "prediction": " <=50K"}
{"instance": ["32", " Private", "178623", " HS-grad", "9", " Never-married", " Other-service", " Not-in-family", " Black", " Female", "0", "0", "40", " ?"], "prediction": " <=50K"}
{"instance": ["28", " Private", "54243", " HS-grad", "9", " Divorced", " Transport-moving", " Not-in-family", " White", " Male", "0", "0", "60", " United-States"], "prediction": " <=50K"}
{"instance": ["29", " Local-gov", "214385", " 11th", "7", " Divorced", " Other-service", " Unmarried", " Black", " Female", "0", "0", "20", " United-States"], "prediction": " <=50K"}
{"instance": ["49", " Self-emp-inc", "213140", " HS-grad", "9", " Married-civ-spouse", " Exec-managerial", " Husband", " White", " Male", "0", "1902", "60", " United-States"], "prediction": " >50K"}

Clean up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

You can delete the individual resources in the Google Cloud console as follows:

Delete the batch prediction job as follows:
1. In the Google Cloud console, in the Vertex AI section, go to the Batch predictions page.
  
  Go to the Batch predictions page
2. Next to the name of your batch prediction job (income-classification-batch-job), click the Actions menu, and choose Delete batch prediction job.
Delete the model as follows:
1. In the Google Cloud console, in the Vertex AI section, go to the Model Registry page.
  
  Go to the Model Registry page
2. Next to the name of your model (income-classifier-model), click the Actions menu, and choose Delete model.
Delete the Vertex AI Workbench instance as follows:
1. In the Google Cloud console, in the Vertex AI section, go to the Instances tab in the Workbench page.
  
  Go to Vertex AI Workbench
2. Select the workbench-tutorial Vertex AI Workbench instance and click Delete.
Delete the Compute Engine VM instance as follows:
1. In the Google Cloud console, go to the Compute Engine page.
  
  Go to Compute Engine
2. Select the on-prem-client VM instance, and click Delete.
Delete the VPN tunnels as follows:
1. In the Google Cloud console, go to the VPN page.
  
  Go to VPN
2. On the VPN page, click the Cloud VPN Tunnels tab.
3. In the list of VPN tunnels, select the four VPN tunnels you created in this tutorial and click Delete.
Delete the HA VPN gateways as follows:
1. On the VPN page, click the Cloud VPN Gateways tab.
  
  Go to Cloud VPN Gateways
2. In the list of VPN gateways, click onprem-vpn-gw1.
3. In the Cloud VPN gateway details page, click Delete VPN Gateway.
4. Click the back arrow if necessary to return to the list of VPN gateways, and then click vertex-networking-vpn-gw1.
5. In the Cloud VPN gateway details page, click Delete VPN Gateway.
Delete the Cloud Routers as follows:
1. Go to the Cloud Routers page.
  
  Go to Cloud Routers
2. In the list of Cloud Routers, select the four routers that you created in this tutorial.
3. To delete the routers, click Delete.
  
  This will also delete the two Cloud NAT gateways that are connected to the Cloud Routers.
Delete the pscvertex forwarding rule for the vertex-networking-vpc VPC network as follows:
1. Go to the Frontends tab of the Load balancing page.
  
  Go to Frontends
2. In the list of forwarding rules, click pscvertex.
3. In the Forwarding rule details page, click Delete.
Delete the VPC networks as follows:
1. Go to the VPC networks page.
  
  Go to VPC networks
2. In the list of VPC networks, click onprem-vpc.
3. In the VPC network details page, click Delete VPC Network.
  
  Deleting each network also deletes its subnetworks, routes, and firewall rules.
4. In the list of VPC networks, click vertex-networking-vpc.
5. In the VPC network details page, click Delete VPC Network.
Delete the storage bucket as follows:
1. In the Google Cloud console, go to the Cloud Storage page.
  
  Go to Cloud Storage
2. Select your storage bucket, and click Delete.
Delete the workbench-sa and onprem-user-managed-sa service accounts as follows:
1. Go to the Service accounts page.
  
  Go to Service accounts
2. Select the onprem-user-managed-sa and workbench-sa service accounts, and click Delete.

What's next

Learn about enterprise networking options for accessing Vertex AI endpoints and services
Learn how Private Service Connect works and why it offers significant performance benefits.
Learn how to use VPC Service Controls to create secure perimeters to allow or deny access to Vertex AI and other Google APIs on your online prediction endpoint.
Learn how and why to use a DNS forwarding zone instead of updating the /etc/hosts file in large scale and production environments.