Using APIs from an External Network

This tutorial demonstrates how to use APIs for Google Cloud Platform (GCP) services from an external network, such as your on-premises private network or another cloud provider's network. This approach allows your on-premises servers that are connected to your private network to access GCP services without using public IP addresses. This tutorial presents an example in which you use a private network on Amazon Virtual Private Cloud (Amazon VPC) to emulate an on-premises private network.

Architecture

The following diagram summarizes the overall architecture that you create in this tutorial.

  • You connect a private network in Amazon VPC to a virtual network in your GCP project through IPsec VPN. If you use an on-premises private network instead of Amazon VPC, you would use Cloud Interconnect to have a private network connection to your GCP project.

  • You use Private Google Access from GCP projects. Servers running outside GCP projects cannot reach GCP APIs, such as the Cloud Translation, by using an internal IP address, even when Private Google Access is enabled. Therefore, you use an HTTP or HTTPS proxy in your GCP project to transfer API requests from external servers to Google Cloud Platform APIs and services using internal IP addresses.

overall architecture

Objectives

  • Enable Private Google Access in order to allow Compute Engine VM instances to access Google Cloud Platform APIs without using public IP addresses.

  • Set up an HTTP or HTTPS proxy using a Compute Engine instance to allow servers in Amazon VPC to access Google Cloud Platform APIs without using public IP addresses.

Costs

This tutorial uses the following billable components of Google Cloud Platform:

You can use the pricing calculator to generate a cost estimate based on your projected usage. New GCP users might be eligible for a free trial.

Additionally, you might incur costs for Amazon Web Services (AWS) services, such as Amazon VPC, VPN, and Amazon Elastic Compute Cloud (Amazon EC2) instances.

Before you begin

Before you begin this tutorial, use the GCP Console to create a GCP project and enable billing. Don't use an existing project, because when you're done, you need to delete the project to avoid incurring further costs.

  1. Create a GCP project for the tutorial.

    GO TO THE MANAGE RESOURCES PAGE

  2. Make sure that billing is enabled for your Google Cloud Platform project.

    Learn how to enable billing

  3. Enable the Translation.

    ENABLE THE Translation

    The Compute Engine API is automatically enabled in new projects.

  4. In the GCP Console, open the Credentials page.

    GO TO CREDENTIALS

  5. For Create credentials, select API key, and then click Close.

  6. Take note of the API key string displayed in the console.

Connecting Amazon VPC to your GCP virtual network

In this tutorial, you use Amazon VPC to emulate your on-premises data center. To establish the Amazon VPC, follow the instructions in Using Cloud VPN with Amazon Web Services, in the sections "Policy Based IPsec VPN: Configuration - AWS" and "Policy Based IPsec VPN: Configuration - GCP UI."

  • Use the default VPC with the network address 172.31.0.0/16 CIDR.
  • Use the default subnet with the network address 172.31.0.0/16 CIDR.
  • Create a VPN connection with AWS using the static IP address aws-vpn as a Compute Engine VPN gateway.

You use IPsec VPN to connect Amazon VPC to the GCP virtual network that you create in GCP.

gcloud

  1. Open Cloud Shell:

    ACTIVATE Cloud Shell

  2. Create a GCP virtual network:

    gcloud compute networks create "aws-vpn" --subnet-mode "auto"
    gcloud compute addresses create "aws-vpn" --region "asia-east1"
    
  3. Follow the instructions in Using Cloud VPN with Amazon Web Services.

  4. To show the VPN configuration in the GCP Console, open the VPN page:

    OPEN VPN

    Your VPN configuration in the GCP Console looks similar to the following image. The green check mark indicates that the remote peer IP address for aws-vpn is set up correctly.

    VPN configuration on the GCP Console

Console

  1. In the GCP Console, go to the VPC Networks page.

    OPEN VPC NETWORKS

  2. Click Create VPC Network.

  3. Set the following options, leaving other options as default:

    • Name: aws-vpn
    • Subnets: Automatic
  4. Click Create.

  5. Go to the External IP addresses page:

    OPEN EXTERNAL IP ADDRESSES

  6. Click Reserve static address.

  7. Set the following options, leaving other options as default:

    • Name: aws-vpn
    • Region: asia-east1
  8. Click Reserve.

  9. Follow the instructions in Using Cloud VPN with Amazon Web Services.

  10. To show the VPN configuration in the GCP Console, open the VPN page:

    OPEN VPN

    Your VPN configuration in the GCP Console looks similar to the following image. The green check mark indicates that the remote peer IP address for aws-vpn is set up correctly.

    VPN configuration on the GCP Console

Configuring the Amazon VPC routing table

To enable the Amazon EC2 instances to access the GCP virtual network in your project, follow these steps:

  1. In the AWS Management Console, click the Route Table tab.

  2. Select Route Propagation and click Edit.

  3. Select Propagate in your VPC network private IP range, and then click Save.

Enabling Private Google Access

You enable Private Google Access on the subnet connected to Amazon VPC.

gcloud

  1. Run the following command in Cloud Shell:

    gcloud compute networks subnets update aws-vpn \
        --enable-private-ip-google-access --region asia-east1

Console

  1. In the GCP Console, go to the VPC Networks page:

    OPEN VPC NETWORKS

  2. Select the aws vpn subnet in the asia-east1 region.

  3. Click Edit.

  4. In the Private Google Access box, select On, and then click Save.

Configuring firewall rules

The next step is to add firewall rules that allow the proxy connection from Amazon VPC, and an SSH connection from all external networks. The SSH connection is used only for configuring the HTTP or HTTPS proxy instance. To configure the proxy without manually signing in to the instance, you can use a startup script, in which case you don't need to create the firewall rule for the SSH connection.

gcloud

  1. In Cloud Shell, add a firewall rule for the proxy:

    gcloud compute firewall-rules create "aws-vpn-allow-proxy" \
      --description "Allow Google private access from AWS VPC" \
      --network "aws-vpn" --allow tcp:8118 \
      --source-ranges "172.31.0.0/16" --target-tags "api-proxy"
    

    When the firewall rule is created, the following output appears:

    Creating firewall...done.
    NAME                 NETWORK  SRC_RANGES     RULES     SRC_TAGS  TARGET_TAGS
    aws-vpn-allow-proxy  aws-vpn  172.31.0.0/16  tcp:8118            api-proxy
    
  2. If you're not using a startup script, add a firewall rule for the SSH connection:

    gcloud compute firewall-rules create "aws-vpn-allow-ssh" \
      --description "Allow SSH from anywhere" \
      --network "aws-vpn" --allow tcp:22  --source-ranges "0.0.0.0/0"
    

    When the firewall rule is created, the following output appears:

    Creating firewall...done.
    NAME               NETWORK  SRC_RANGES  RULES   SRC_TAGS  TARGET_TAGS
    aws-vpn-allow-ssh  aws-vpn  0.0.0.0/0   tcp:22
    

Console

  1. In the GCP Console, go to the Firewall rules page:

    OPEN FIREWALL RULES

  2. Click Create Firewall rule.

  3. Set the following options, leaving other options as default:

    • Name: aws-vpn-allow-proxy
    • Description: Allow Google Private Access from Amazon VPC
    • Network: aws-vpn
    • Target tags: api-proxy
    • Source IP ranges: 172.31.0.0/16
    • Protocols and ports: tcp:8118
  4. Click Create.

  5. Click Create Firewall rule.

  6. Set the following options, leaving other options as default:

    • Name: aws-vpn-allow-ssh
    • Description: Allow SSH from anywhere
    • Network: aws-vpn
    • Target tags: api-proxy
    • Source IP ranges: 0.0.0.0/0
    • Protocols and ports: tcp:22
  7. Click Create.

Creating an HTTP or HTTPS proxy instance

Next, you need to create a Compute Engine instance, configure it as an HTTP or HTTPS proxy, and sign in.

gcloud

  1. Launch a Compute Engine instance:

    gcloud compute instances create "api-proxy" \
      --zone "asia-east1-a" --machine-type "n1-standard-1" \
      --subnet "aws-vpn" --private-network-ip "10.140.0.2" \
      --tags "api-proxy"
    

    Specify the private IP address of the instance that you want to use as a proxy address.

  2. Sign in to the instance:

    gcloud compute ssh "api-proxy" --zone "asia-east1-a"
  3. In the SSH session, install the privoxy service:

    sudo apt-get -y install privoxy
  4. Configure the privoxy service:

    sudo sed -i "s/listen-address  127.0.0.1:8118/listen-address  10.140.0.2:8118/" /etc/privoxy/config
    sudo systemctl restart privoxy
    
  5. Confirm that the service is configured to accept connections to IP address 10.140.0.2 through port 8118:

    sudo netstat -nlt

    The output looks similar to the following:

    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State
    tcp        0      0 10.140.0.2:8118         0.0.0.0:*               LISTEN
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
    tcp6       0      0 :::22                   :::*                    LISTEN
    

Console

  1. In the GCP Console, go to the VM Instances page:

    OPEN VM INSTANCES

  2. Click Create.

  3. In the Create an instance page, set the following options, leaving other options as default:

    • Name: api-proxy
    • Zone: asia-east1-a
  4. Open Management, security, disks, networking, sole tenancy and select Networking.

  5. Set the following options, leaving other options as default:

    • Network tags: api-proxy
    • Network interface > Network: aws-vpn
    • Network interface > Subnetwork: aws-vpn(10.140.0.0/20)
    • Network interface > Primary internal IP: Custom
    • Network interface > Custom ephemeral IP address: 10.140.0.2
  6. Click Done.

  7. Click Create.

  8. Select the api-proxy instance, and click SSH to open an SSH terminal.

  9. In the SSH session, install the privoxy service:

    sudo apt-get -y install privoxy
  10. Configure the privoxy service:

    sudo sed -i "s/listen-address  127.0.0.1:8118/listen-address  10.140.0.2:8118/" /etc/privoxy/config
    sudo systemctl restart privoxy
    
  11. Confirm that the service is configured to accept connections to IP address 10.140.0.2 through port 8118:

    sudo netstat -nlt

    The output looks similar to the following:

    Active Internet connections (only servers)
    Proto Recv-Q Send-Q Local Address           Foreign Address         State
    tcp        0      0 10.140.0.2:8118         0.0.0.0:*               LISTEN
    tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN
    tcp6       0      0 :::22                   :::*                    LISTEN
    
  12. Exit from the instance:

    exit
    

At this point, you don't need the SSH connection to the instance anymore. If your security standards require you to remove the firewall rule that allows the SSH connection, do the following operation:

gcloud

  1. Delete the firewall rule:

    gcloud compute firewall-rules delete "aws-vpn-allow-ssh"

Console

  1. In the GCP Console, go to the Firewall rules page:

    OPEN FIREWALL RULES

  2. Select aws-vpn-allow-ssh.

  3. Click Delete.

Removing an external IP address from the proxy instance

You must prevent the proxy instance from accidentally using a public IP address to reach GCP APIs. This section shows how to remove the external IP address from the proxy instance.

gcloud

  1. Determine the configuration name:

    gcloud compute instances describe api-proxy | grep -A5 accessConfigs

    In the following example output, the configuration name for the external IP address is external-nat.

    - accessConfigs:
    - kind: compute#accessConfig
        name: external-nat
        natIP: 104.199.157.31
        type: ONE_TO_ONE_NAT
    kind: compute#networkInterface
    
  2. Remove the external IP address, specifying the configuration name from the previous step.

    gcloud compute instances delete-access-config api-proxy --access-config-name "external-nat"

Console

  1. In the GCP Console, go to the VM Instances page.

    OPEN VM INSTANCES

  2. Click api-proxy.

  3. Click Edit.

  4. Click nic0: aws-vpn.

  5. Set the following options, leaving other options as default:

    • External IP: None
  6. Click Done.

  7. Click Save.

Using a Google API from Amazon VPC

In this section, you use the Cloud Translation to verify that you can use a Google Cloud Platform API from an Amazon EC2 instance that is running on Amazon VPC.

  1. Launch an Amazon EC2 instance in the Amazon VPC that you created.

  2. Connect to the Amazon EC2 instance by using the SSH terminal.

  3. Create an API request message file:

    echo '{"q": ["this is a test translation via proxy"], "target": "it"}' >/tmp/translation.json
  4. Post a request that specifies the proxy address. Replace [YOUR_API_KEY] with the API key string you created earlier.

    API_KEY=[YOUR_API_KEY]
    curl -H 'Content-Type: application/json' --dump-header - \
      --proxy http://10.140.0.2:8118 -d @/tmp/translation.json \
      -X POST https://translation.googleapis.com/language/translate/v2?key=${API_KEY}
    

    The Translation service replies as follows:

    HTTP/1.1 200 Connection established
    Proxy-Agent: Privoxy/3.0.21
    
    HTTP/1.1 200 OK
    Content-Type: application/json; charset=UTF-8
    Vary: X-Origin
    Vary: Referer
    Date: Thu, 06 Jul 2017 23:44:57 GMT
    Server: ESF
    Cache-Control: private
    X-XSS-Protection: 1; mode=block
    X-Frame-Options: SAMEORIGIN
    X-Content-Type-Options: nosniff
    Accept-Ranges: none
    Vary: Origin,Accept-Encoding
    Transfer-Encoding: chunked
    
    {
      "data": {
        "translations": [
          {
            "translatedText": "Questa è una traduzione del test tramite il proxy",
            "detectedSourceLanguage": "en"
          }
        ]
      }
    }
    

Cleaning up

When you have completed this tutorial, delete your project to avoid incurring further costs.

  1. In the GCP Console, go to the Projects page.

    Go to the Projects page

  2. In the project list, select the project you want to delete and click Delete delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

Was this page helpful? Let us know how we did:

Send feedback about...