Example: Private connectivity for a MySQL instance
This page explains through an example, how to use the Private Service Connect (PSC) to establish a connection between your MySQL backend system which is on a private network and the Integration Connectors runtime.
Considerations
When you create a PSC service attachment, consider the following key points:
- The PSC service attachment and the load balancer should be in different subnets within the same VPC. And specifically, the service attachment should be in a NAT subnet.
- Software running on your backend VMs must respond to both load balanced
traffic and health check probes sent to each forwarding
rule's IP address (the software must listen on
0.0.0.0:<port>
and not on a specific IP address assigned to a network interface). For more information, see Health check. - Configure the firewall rules to facilitate the traffic flow.
Ingress rules
- Traffic from the PSC service attachment's subnet should reach the ILB's subnet.
- Within the ILB's subnet, ILB should be able to send traffic to your backend system.
- The health check probe should be able to access your backend system. The
Google Cloud health check probes have a fixed IP range (
35.191.0.0/16, 130.211.0.0/22
). So these IPs can be allowed to send traffic to your backend server.
Egress rules
Egress traffic is enabled by default in a Google Cloud project, unless specific deny rules are configured.
- All your Google Cloud components such as the PSC service attachment and the load balancer should be in the same region.
- Your backend system should not be open to the public network, as this can be a security concern. However,
ensure that your backend system accepts traffic in the following scenarios:
- Pass-through load balancers (L4 TCP/UDP ILB): Requests from the PSC service attachment's NAT IPs should be able to reach your backend. These NAT IPs are auto-generated. Therefore, you must allow the entire NAT subnet's IP range in which your service attachment resides. For more information, see Private Service Connect subnets.
- Proxy-based/HTTP(s) load balancers (L4 proxy ILB, L7 ILB): All new requests originate from the load balancer. Therefore, your backend should accept requests from the proxy subnet of your VPC network. For more information, see Proxy-only subnets for Envoy-based load balancers.
Example
Consider you have a MySQL instance hosted on Google Cloud in a private VPC network and, you want to expose the MySQL instance to the Integration Connectors runtime.
The following illustration shows how the sample Google Cloud project will look after the PSC service attachment is configured.
![sample illustration](https://cloud.google.com/static/integration-connectors/images/ic-psc-example.png?authuser=4)
Before you begin
Before creating a PSC service attachment for the sample scenario, do the following tasks:
- Install the gcloud CLI.
- Enable the Compute Engine API for your Google Cloud project.
- To make your CLI commands less verbose, you can set the values
for your PROJECT_ID, REGION, and ZONE using the following commands:
gcloud config set project PROJECT_ID gcloud config set compute/region REGION gcloud config set compute/zone ZONE
- For the commands in this tutorial, replace BACKEND_SERVER_PORT with 3306 which is the default port on which the MySQL server runs.
- It's recommended to create a new VPC network and use that when trying this sample scenario. After testing the scenario, you can delete the VPC network and other resources.
- There should be at least one existing connection that you have created. The connection can be of any type. Having an existing connection lets you fetch the project ID of the service directory from the Integration Connectors runtime. This project ID is required for creating the PSC service attachment.
Create a PSC service attachment
To create a PSC service attachment for the sample scenario, do the following tasks:
- Create a VPC network and the required subnets.
- Create a VPC network.
gcloud compute networks create VPC_NETWORK --project=PROJECT_NAME --subnet-mode=custom --mtu=1460 --bgp-routing-mode=regional
- Add Subnet-1.
gcloud compute networks subnets create SUBNET_NAME_1 --network=VPC_NETWORK --range=SUBNET_RANGE_1 --purpose=PRIVATE_SERVICE_CONNECT
This command creates
Subnet-1
as a NAT subnet which will be exclusively used to host the PSC service attachment. You can't host any other service in this NAT subnet. - Add Subnet-2.
gcloud compute networks subnets create SUBNET_NAME_2 --network=VPC_NETWORK --range=SUBNET_RANGE_2
- Create a VPC network.
- Create a VM instance.
To create a VM instance in the newly created VPC, run the following command:
gcloud compute instances create \ --image-family debian-10 \ --image-project debian-cloud \ --network-interface=network-tier=PREMIUM,subnet=SUBNET_NAME_2,no-address \ mysql-test
This command creates a VM instance with the name
mysql-test
. - Configure Cloud NAT.
- Create a simple router.
gcloud compute routers create NAT_ROUTER_NAME \ --network=VPC_NETWORK
- Configure the network address translation.
gcloud compute routers nats create NAT_GATEWAY_NAME \ --router=NAT_ROUTER_NAME \ --auto-allocate-nat-external-ips \ --nat-all-subnet-ip-ranges
- Create a simple router.
- SSH to your VM instance.
- Create a firewall rule to allow SSH.
gcloud compute firewall-rules create VPC_NETWORK-allow-ssh --direction=INGRESS --priority=1000 --network=VPC_NETWORK --allow=tcp:22
- SSH to your VM instance.
gcloud compute ssh \ --tunnel-through-iap \ mysql-test
- Create a firewall rule to allow SSH.
- Install the MySQL server. For detailed instructions, see Install MySQL.
- Connect to the MySQL instance and create sample data.
- Connect to MySQL using the MySQL client.
sudo mysql -u root -p
- Create a new user and grant access to connect from any host address.
CREATE USER 'test-user'@'%' IDENTIFIED BY 'test-pass'; GRANT ALL PRIVILEGES ON * . * TO 'test-user'@'%'; FLUSH PRIVILEGES;
This command creates a user with username
test-user
and password astest-pass
. - Create a database along with the sample data.
CREATE DATABASE test-db; USE test-db; CREATE TABLE Singers (SingerId int, FirstName varchar(255), LastName varchar(255)); INSERT INTO Singers (SingerId, FirstName, LastName) VALUES (1, 'Marc', 'Richards');
- Exit from the MySQL client.
mysql> exit
- Exit from the VM instance.
exit
- Connect to MySQL using the MySQL client.
- Set up an unmanaged instance group.
- Create an unmanaged instance group.
gcloud compute instance-groups unmanaged create INSTANCE_GROUP_NAME
- Add the VM instance created in step 2 to the group.
gcloud compute instance-groups unmanaged add-instances INSTANCE_GROUP_NAME --instances=mysql-test
- Create an unmanaged instance group.
- Create a health check probe and allow the traffic from the probe.
- Create the health check probe.
gcloud compute health-checks create tcp HEALTH_CHECK_NAME --port BACKEND_SERVER_PORT --region=REGION
In this command, set BACKEND_SERVER_PORT to 3306 which is the default port on which the MySQL server runs.
- Create a firewall rule to allow traffic from the probe.
gcloud compute firewall-rules create VPC_NETWORK-allow-health-check --direction=INGRESS --priority=1000 --network=VPC_NETWORK --allow=tcp:BACKEND_SERVER_PORT --source-ranges=35.191.0.0/16,130.211.0.0/22
- Create the health check probe.
- Create an L4 internal load balancer and allow traffic from the load balancer.
- Create a backend service.
gcloud compute backend-services create BACKEND_SERVICE --load-balancing-scheme=internal --protocol=tcp --health-checks=HEALTH_CHECK_NAME --health-checks-region=REGION
- Add instance group to the backend service.
gcloud compute backend-services add-backend BACKEND_SERVICE --instance-group=INSTANCE_GROUP_NAME --instance-group-zone=ZONE
- Create a forwarding rule.
gcloud compute forwarding-rules create FORWARDING_RULE_NAME --load-balancing-scheme=internal --network=VPC_NETWORK --subnet=SUBNET_NAME_2 --ip-protocol=TCP --ports=BACKEND_SERVER_PORT --backend-service=BACKEND_SERVICE --backend-service-region=REGION
-
Create a firewall rule to allow internal traffic from load-balancer to the instance group.
gcloud compute firewall-rules create VPC_NETWORK-allow-internal --direction=INGRESS --priority=1000 --network=VPC_NETWORK --action=ALLOW --rules=all --source-ranges=SUBNET_RANGE_2
- Create a backend service.
- Create the PSC service attachment.
- Create a firewall rule to allow traffic from the PSC service attachment to the internal load balancer created in the previous step.
gcloud compute firewall-rules create VPC_NETWORK-allow-sa --direction=INGRESS --priority=1000 --network=VPC_NETWORK --allow=tcp:BACKEND_SERVER_PORT --source-ranges=SUBNET_RANGE_1
- Create service attachment with explicit approval.
gcloud compute service-attachments create SERVICE_ATTACHMENT_NAME --producer-forwarding-rule=FORWARDING_RULE_NAME --connection-preference=ACCEPT_MANUAL --consumer-accept-list=SERVICE_DIRECTORY_PROJECT_ID=LIMIT --nat-subnets=SUBNET_NAME_1
In this command, LIMIT is the connection limit for the project. The connection limit is the number of consumer Private Service Connect endpoints that can connect to this service. To understand how to get the SERVICE_DIRECTORY_PROJECT_ID, see Get the project ID of the service directory.
- Create a firewall rule to allow traffic from the PSC service attachment to the internal load balancer created in the previous step.
- Create an endpoint attachment.
You can think of the endpoint attachment as an interface to the PSC service attachment. You can't use the PSC service attachment directly for configuring private connectivity. The PSC service attachment can be accessed only through an endpoint attachment. And you can create the endpoint attachment either as an IP address or as a hostname. After creating the endpoint attachment, you can use it when you configure a connector for private connectivity. For more information, see Create an endpoint attachment.
- Verify the PSC setup. You can verify the PSC service attachment connectivity by creating
a MySQL connection to the
test-db
database that you created for this tutorial. For detailed steps on creating a MySQL connection, see Create a MySQL connection. When creating the connection, in theDestinations
section (see step 5 in Create a MySQL connection), select theDestination type
asHostname
, and then enter the appropriate endpoint IP address or hostname. If the connection creation is successful, the status of the newly created connection will beActive
in your Connections page in the Cloud console.
Get the project ID of the service directory
As a best practice, you can create the PSC service attachment such that it accepts requests only from the specified Google Cloud projects. However, to do this, you need the project ID of the service directory associated with your Google Cloud project. To get the project ID of the service directory, you can use the List Connections API as shown in the following example.
Syntax
curl -X GET \ -H "authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://connectors.googleapis.com/v1/projects/CONNECTORS_PROJECT_ID/locations/-/connections"
Replace the following:
- CONNECTORS_PROJECT_ID: The ID of your Google Cloud project where you created your connection.
Example
This example gets the project ID of the service directory for the connectors-test
Google Cloud project.
curl -X GET \ -H "authorization: Bearer $(gcloud auth print-access-token)" \ -H "Content-Type: application/json" \ "https://connectors.googleapis.com/v1/projects/connectors-test/locations/-/connections"
Running this command on the terminal displays an output similar to the following:
..... { "connections": [ { "name": "projects/connectors-test/locations/asia-northeast1/connections/big-query-iam-invalid-sa", "createTime": "2022-10-07T09:02:31.905048520Z", "updateTime": "2022-10-07T09:22:39.993778690Z", "connectorVersion": "projects/connectors-test/locations/global/providers/gcp/connectors/bigquery/versions/1", "status": { "state": "ACTIVE" }, "configVariables": [ { "key": "project_id", "stringValue": "connectors-test" }, { "key": "dataset_id", "stringValue": "testDataset" } ], "authConfig": {}, "serviceAccount": "564332356444-compute@developer.gserviceaccount.com", "serviceDirectory": "projects/abcdefghijk-tp/locations/asia-northeast1/namespaces/connectors/services/runtime", "nodeConfig": { "minNodeCount": 2, "maxNodeCount": 50 } }, ....
In the sample output, for the connectors-test
Google Cloud project, the project ID
of the service directory is abcdefghijk-tp
.
PSC service attachment for other backend systems
You can follow the steps in the previous example to set up a PSC service attachment for other backend systems. However, you'll have to modify step 5 and 6 to suite your desired backend system. After you've installed your backend system in the VM instance, add the backend system to the instance group and follow the remaining steps as is.