Manage private connectivity configurations

In this page, you learn how to use the Datastream API to create, retrieve information about, and delete private connectivity configurations.

Private connectivity configurations contain information that Datastream uses to communicate with a data source over a private network (internally within Google Cloud, or with external sources connected over VPN or Interconnect). This communication happens through a Virtual Private Cloud (VPC) peering connection.

There are two ways that you can use the Datastream API. You can make REST API calls or you can use the Google Cloud CLI (CLI).

To see high-level information about using gcloud to manage Datastream private connectivity configurations, click here.

Create a private connectivity configuration

Before creating a private connectivity configuration, make sure that you:

  • Have a VPC network to peer to Datastream's private network. For more information on creating a VPC network, see Using VPC Network Peering.
  • Have an available IP range on the VPC network. Datastream uses this IP range to create VPC peering with the provided VPC, so the range can't overlap with subnets within that VPC. As a result, Datastream can communicate with the source.

There are two parameters associated with creating a private connectivity configuration:

  • vpcName: Specify the name and path of the VPC network.
  • subnet: Provide an available IP range on the VPC network.

The following code shows a request to create a private connectivity configuration, and specifies the connectivity method as using VPC peering (vpcPeeringConfig).

REST

POST https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections?privateConnectionId=[private-connectivity configuration-id]
{
  "displayName": "[private-connectivity-configuration-display-name]",
  "vpcPeeringConfig": {
      "vpcName": "[name-and-path-of-VPC-network]",
      "subnet": "[subnet]"
  }
}

For example:

POST https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections?privateConnectionId=myPrivateConnection
{
  "displayName": "my private connectivity configuration",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }
}

After the private connectivity configuration is created, you can view information about it by calling the privateConnections/get method. Your output appears, similar to the following:

{
  "name": "projects/projectId/location/us-central1/privateConnections/myPrivateConnection",
  "createTime": "2019-12-22T16:17:37.159786963Z",
  "updateTime": "2019-12-22T16:17:37.159786963Z",
  "displayName": "my private connectivity configuration",
  "state": "CREATED",
  "vpcPeeringConfig": {
    "vpcName": "projects/my-project/global/networks/my-vpc",
    "subnet": "10.11.12.13/29"
  }  
}

gcloud

For more information on using gcloud to create a private connectivity configuration, click here.

Get information about a private connectivity configuration

The following code shows a request to retrieve information about a private connectivity configuration that has been created.

REST

GET https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections/[private-connectivity-configuration-id]

For example:

GET https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection

gcloud

For more information on using gcloud to retrieve information about your private connectivity configuration, click here.

List private connectivity configurations

The following code shows a request to retrieve information about all of your private connectivity configurations.

REST

GET https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections

gcloud

For more information on using gcloud to retrieve information about all of your private connectivity configurations, click here.

Delete a private connectivity configuration

The following code shows a request to delete a private connectivity configuration. After the configuration is deleted, any connection profiles using the configuration won't work. In addition, any streams using these connection profiles won't be operable.

To resolve this issue, assign a different private connectivity configuration to each connection profile. For more information, see Update a connection profile.

REST

DELETE https://datastream.googleapis.com/v1/projects/[project-id]/locations/
[location]/privateConnections/[private-connectivity-configuration-id] {"force": true}

For example:

DELETE https://datastream.googleapis.com/v1/projects/myProjectId/locations/
us-central1/privateConnections/myPrivateConnection {"force": true}

gcloud

For more information on using gcloud to delete your private connectivity configuration, click here.

What's next