Manage private connectivity configurations

Overview

In this section, you learn how to use the Datastream API to create, retrieve information about, and delete private connectivity configurations.

Private connectivity configurations contain information that Datastream uses to communicate with a data source over a private network (internally within Google Cloud, or with external sources connected over VPN or Interconnect). This communication happens through a Virtual Private Cloud (VPC) peering connection.

There are two ways that you can use the Datastream API. You can make REST API calls or you can use the Google Cloud CLI (CLI).

To see high-level information about using gcloud to manage Datastream private connectivity configurations, click here.

To use a private connection, any streams and connection profiles that use a private connecivity configuration must be in the same region.

It's recommended that the source database run in a subnet in the same region as the private connectivity configuration. Otherwise, you'll have to manually add a firewall rule that enables communication between the regions.

Create a private connectivity configuration

Before creating a private connectivity configuration, make sure that you:

• Have a VPC network to peer to Datastream's private network. For more information on creating a VPC network, see Using VPC Network Peering.
• Have an available IP range on the VPC network. Datastream uses this IP range to create VPC peering with the provided VPC, so the range can't overlap with subnets within that VPC. As a result, Datastream can communicate with the source.

There are two parameters associated with creating a private connectivity configuration:

• vpcName: Specify the name and path of the VPC network.
• subnet: Provide an available IP range on the VPC network.

The following code shows a request to create a private connectivity configuration, and specifies the connectivity method as using VPC peering (vpcPeeringConfig).

REST

POST https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections?privateConnectionId=[private-connectivity configuration-id]
{
  "displayName": "[private-connectivity-configuration-display-name]",
  "vpcPeeringConfig": {
      "vpcName": "[name-and-path-of-VPC-network]",
      "subnet": "[subnet]"
  }
}

For example:

POST https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections?privateConnectionId=myPrivateConnection
{
  "displayName": "my private connectivity configuration",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }
}

As part of creating a private connectivity configuration, multiple Google Cloud resources are also being created to achieve the connectivity. Therefore, it may take a few minutes for the private connectivity configuration to be created.

After the private connectivity configuration is created, you can view information about it by calling the privateConnections/get method. Your output appears, similar to the following:

{
  "name": "projects/projectId/location/us-central1/privateConnections/myPrivateConnection",
  "createTime": "2019-12-22T16:17:37.159786963Z",
  "updateTime": "2019-12-22T16:17:37.159786963Z",
  "displayName": "my private connectivity configuration",
  "state": "CREATED",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }  
}

There are three states of a private connectivity configuration:

• CREATING: The private connectivity configuration is in the process of being created.
• CREATED: The private connectivity configuration is ready to be used by connection profiles.
• FAILED: A problem occurred while the private connectivity configuration was in the process of being created. If this happens, then a user-friendly error message appears, detailing why the error occurred and what to do to rectify the problem.

gcloud

For more information on using gcloud to create a private connectivity configuration, click here.

Get information about a private connectivity configuration

The following code shows a request to retrieve information about a private connectivity configuration that has been created.

REST

GET https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections/[private-connectivity-configuration-id]

For example:

GET https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection
     

gcloud

For more information on using gcloud to retrieve information about your private connectivity configuration, click here.

List private connectivity configurations

The following code shows a request to retrieve information about all of your private connectivity configurations.

REST

GET https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections

gcloud

For more information on using gcloud to retrieve information about all of your private connectivity configurations, click here.

Delete a private connectivity configuration

The following code shows a request to delete a private connectivity configuration. After the configuration is deleted, any connection profiles using the configuration won't work. In addition, any streams using these connection profiles won't be operable.

To resolve this issue, assign a different private connectivity configuration to each connection profile. For more information, see Update a connection profile.

REST

DELETE https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections/[private-connectivity-configuration-id] {"force": true} 

For example:

DELETE https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection {"force": true} 

You must add the {"force": true} piece of code to ensure that the private connectivity configuration is deleted properly.

gcloud

For more information on using gcloud to delete your private connectivity configuration, click here.