Secure non-Google Cloud applications using the app connector

This page describes how to set up and use the BeyondCorp Enterprise app connector to secure non-Google Cloud applications.

Overview

You can use the BeyondCorp Enterprise app connector to provide identity and context-aware access to HTTPS applications running in non-Google Cloud environments. The BeyondCorp Enterprise app connector is a secure, authenticated interface between the BeyondCorp Enterprise enforcement plane and applications running in other clouds and on-premises environments.

A benefit to using the BeyondCorp Enterprise app connector is that you do not need to open up firewalls or set up site-to-site Cloud VPN connections.

Architecture

Following is a high-level architecture diagram that depicts the main components of the BeyondCorp Enterprise app connector.

BeyondCorp Enterprise app connector components

The BeyondCorp Enterprise app connector components are governed using three main API resources: app connectors, app connections, and app gateways.

App Connectors
A connector resource defines a unique app connector remote agent. App connector remote agents are deployed in remote enterprise environments, such as in other clouds and on-premises environments. Remote agents initiate and maintain tunnel sessions to Google Cloud, receive traffic from Google Cloud, and relay traffic to remote application endpoints in the same environment.
App Connections
A connection resource defines a logical connection from Google Cloud to a specific application endpoint that is identified using IP address:Port or FQDN:Port. A connection resource orchestrates a set of managed app connector gateways that are allocated for a given application endpoint. The gateways are managed in Google Cloud and terminate the tunnel sessions from the remote agents when no longer needed.
App Gateways

An app gateway is a Google managed service offering. App connector remote agents connect to one or more gateways that allow app connections and app connectors to relay traffic from end users to remote agents. Traffic for app connections that are hosted on the same app gateway is routed through a common virtual infrastructure.

When you connect an application using the app connector, app gateways are implicitly created and managed for you. You can also create additional app gateways if you want to organize app connections resources into groups. App gateways provide the flexibility to group or segregate app connections to allow app isolation.

Each app gateway can support a maximum of up to 1 Gbps in total throughput for up to 200,000 concurrent connections. We recommend that you create dedicated gateways for critical applications that require high performance. You can allocate a maximum of up to 10 apps per gateway.

To support up to 200,000 concurrent connections and up to 1 Gbps in total throughput, we recommend that you use a dedicated VM of 8 cores and 8 GB of memory to run the app connector remote agent. For example, you can dedicate 2 app connector remote agents in an HA cluster with 8 cores and 8 GB of memory each to support a high performance web application (app1.examplecompany.com) with a dedicated app gateway (app_gateway1). Create another cluster of 2 VMs each with 8 cores and 8 GB memory to support a combination of 10 low usage apps with another dedicated app gateway (app_gateway2).

The app connector, app connection, and app gateway resources are regional. You can only use them within a regional context. For example, you cannot assign an app connector in region A to an app connection or app gateway created in region B.

Before you begin

You can use the BeyondCorp Enterprise app connector only with HTTPS applications. Additionally, HTTPS load balancers do not use the Server Name Indication (SNI) extension for connections to the backend. For more details, see Encryption from the load balancer to the backends.

To use the BeyondCorp Enterprise app connector, you must have the following:

Each BeyondCorp Enterprise app connector agent requires a Linux VM that is running Docker. We recommend the following Linux distributions:

  • Debian 10.7 (buster)
  • Red Hat Enterprise Linux Server 7.8 (Maipo) x86_64
  • Ubuntu 16.04 or later

For the Linux VM hardware, we recommended a minimum of 2 CPUs with 2GB of RAM.

You can set up and enable the app connector by using gcloud and the API, or by using the Google Cloud console. To use gcloud and APIs, complete the following steps. To use the Google Cloud console, go to Set up the app connector using the Google Cloud console.

Setting up the Google Cloud project

To set up a Google Cloud project to use with the BeyondCorp Enterprise app connector, you must enable the BeyondCorp Enterprise API. Enable the API by completing the following steps:

gcloud CLI

Before you complete the following steps, ensure that you have the gcloud CLI SDK installed.

  1. Set the required environment variable by running the following command: 

    PROJECT_ID=my-project

    Replace my-project with the Google Cloud project ID.

  2. Enable the API by running the following command: 

    gcloud config set project $PROJECT_ID
gcloud services enable beyondcorp.googleapis.com

API

  1. Get an access token and set the required environment variables and aliases by running the following commands: 

    PROJECT_NUMBER=my-project-number
ACCESS_TOKEN=my-access-token

    Replace my-project with the Google Cloud project ID.

  2. Set up a convenient alias to use your access token: 

    alias gcurl="curl -H 'Authorization: Bearer ${ACCESS_TOKEN}' -H 'Content-Type: application/json'"

  3. Enable the API by running the following command: 

    gcurl https://serviceusage.googleapis.com/v1/projects/${PROJECT_NUMBER}/services/beyondcorp.googleapis.com:enable -d "{}"

Installing an app connector remote agent

You must deploy an app connector remote agent VM for every network environment that hosts an application that you want to protect with BeyondCorp Enterprise. You must have a dedicated VM, or any Bare Metal server with Docker installed, for each remote agent that you create.

To create a remote agent, complete the following steps:

  1. Create a VM instance in your application environment.
    1. Ensure that the remote agent VM network firewall allows all outbound traffic initiated at port 443 for the IAP-TCP IP range 35.235.240.0/20. See Verify the firewall configuration for other domains that the remote agent VM firewall should allow outbound traffic to.

  2. Install the remote agent:

    1. Verify that your DNS is configured correctly and that you can access the application by using curl.
    2. Install the Docker Engine.
      1. Optional: If you are using a proxy, verify that the Docker daemon is configured correctly.
    3. Optional: If you are using a proxy, verify that both the HTTP and HTTPS proxy environment variables are set and using the http:// schema.
    4. Run the following command on the remote agent VM to install the remote agent: 
      curl https://raw.githubusercontent.com/GoogleCloudPlatform/beyondcorp-applink/main/bash-scripts/install-beyondcorp-runtime -o ./install-beyondcorp-runtime && chmod +x ./install-beyondcorp-runtime && ./install-beyondcorp-runtime
    5. To add the bce-connctl alias to your shell, run the following command: 
       source ~/.bce_alias

    6. Set the required environment variables by running the following commands: 

      PROJECT_ID=my-project
REGION=us-central1
CONNECTOR_NAME=my-connector
      Replace the following:

      • my-project: the Google Cloud project ID.
      • us-central1: the region in which to deploy the connection and gateways.
      • my-connector: the name of the connector.

    7. Enroll the connector using one of the following credential types:

    Service account

    1. Enroll the connector by running the following command in the bce-connctl CLI: 

      bce-connctl init --project=$PROJECT_ID --region=$REGION --connector=$CONNECTOR_NAME

    2. Follow the on-screen instructions to copy and paste the enrollment script into your Google Cloud projects using Cloud Shell or the gcloud CLI and finish enrolling your connector. To run the enrollment script, you must have Owner permissions on the project.

      When the enrollment script is complete, the runtime syncs the connector configuration automatically.

    Federated identity

    1. Follow the workload identity federation guide to grant your external identity access to Google Cloud using service account impersonation.
    2. Download the client library configuration and place the file on the remote agent VM at /var/beyondcorp/credentials/credentials.json.
      1. If using an AWS EC2 instance as your remote agent VM, do one of the following:
        1. Make IMDSv2 optional on the EC2 instance.
        2. Add the following line to the credential_source object in the client library configuration file: 
          "imdsv2_session_token_url": "http://169.254.169.254/latest/api/token"
    3. On the remote agent VM, run the following commands in the bce-connctl CLI: 
      bce-connctl config set project $PROJECT_ID
bce-connctl config set region $REGION
bce-connctl config set connector $CONNECTOR_NAME
    4. Run the following command to restart the agent and have it pick up the credentials: 
      sudo systemctl restart beyondcorp
    5. On your Cloud Shell or gcloud CLI endpoint, set the required environment variable by running the following command: 
      SERVICE_ACCOUNT=my-service-account
      Replace the following:
      • my-service-account: the Google Cloud service account associated with the external identity.

    6. Grant the Google Cloud service account the BeyondCorp Connection Agent role by running the following command:

      gcloud projects add-iam-policy-binding $PROJECT_ID \
    --member="serviceAccount:$SERVICE_ACCOUNT" \
    --role="roles/beyondcorp.connectionAgent"

    7. Run the following command to create the Google Cloud app connector resource.

      gcloud alpha beyondcorp app connectors create $CONNECTOR_NAME \
    --project=$PROJECT_ID \
    --location=$REGION \
    --member="serviceAccount:$SERVICE_ACCOUNT" \
    --display-name=$CONNECTOR_NAME

      After the app connector resource is created, the remote agent runtime syncs the connector configuration automatically.

  3. Verify the remote agent installation:

    1. Run the following command to ensure the service is running: 
      sudo systemctl status beyondcorp
    2. The remote agent consists of three Docker containers: bce-control-runtime, bce-logagent, and bce-connector. Verify that all three containers are running by running the following command: 
      docker ps --filter name=bce
    3. Optional: You can check the Docker containers log files by running the following command: 
      docker logs -f CONTAINER_NAME
      Replace CONTAINER_NAME with the Docker container name.

Connecting a remote application to a VPC

gcloud CLI

Before you complete the following steps, ensure that you have the gcloud CLI SDK installed and you are logged in with an account that has the beyondcorp.admin role.

  1. Set the required environment variables by running the following commands:

    CONNECTOR_NAME=my-connector
CONNECTION_NAME=my-connection
PROJECT_ID=my-project
REGION=us-central1
APP_ENDPOINT=APP_HOST:APP_PORT

    Replace the following:

    • my-connector: the name of the connector, defined in an earlier step.
    • my-connection: a unique name for the connection.
    • my-project: the Google Cloud project ID.
    • us-central1: the region in which to deploy the connection and gateways.
    • APP_HOST: the on-premises IP address or an FQDN that hosts your application.
    • APP_PORT: the port number to connect to your application.

  2. To create a connection between your application and your VPC, run the following command:

    gcloud alpha beyondcorp app connections create $CONNECTION_NAME \
    --project=$PROJECT_ID \
    --location=$REGION \
    --application-endpoint=$APP_ENDPOINT \
    --type=tcp \
    --connectors=$CONNECTOR_NAME \
    --display-name=$CONNECTION_NAME

    Note that this step might take several minutes to complete.

  3. After you create the connection, run the following commands to retrieve the gateway URIs: 

    GATEWAY_URI="$(gcloud alpha beyondcorp app connections describe $CONNECTION_NAME --project=$PROJECT_ID --location=$REGION --format='get(gateway.uri)')"
GATEWAY_URI_L7="${GATEWAY_URI/bce-psc/bce-ps7}"

    The GATEWAY_URI_L7 is used to create a forwarding rule. The BeyondCorp Enterprise app connector uses Private Service Connect to link the consumer project to the managed resources in Google managed projects.

API

  1. Set the required environment variables by running the following commands:

    CONNECTOR_NAME=my-connector
CONNECTION_NAME=my-connection
PROJECT_ID=my-project
REGION=us-central1
APP_HOST=my-app-host
APP_PORT=my-app-port

    Replace the following:

    • my-connector: the name of the connector, defined in an earlier step.
    • my-connection: a unique name for the connection.
    • my-project: the Google Cloud project ID.
    • us-central1: the region in which to deploy the connection and gateways.
    • my-app-host: the on-premises IP address or an FQDN that hosts your application.
    • my-app-port: the port number to connect to your application.

  2. To create a connection between your application and your VPC, run the following command:

    gcurl https://beyondcorp.googleapis.com/v1/projects/${PROJECT_NUMBER}/locations/${REGION}/appConnections?app_connection_id=${CONNECTION_NAME} \
-d "{ \
    'application_endpoint': \
    { \
        'host': '${APP_HOST}', \
        'port': '${APP_PORT}' \
    }, \
    'type': 'TCP_PROXY', \
    'display_name': '${CONNECTION_NAME}' \
}"

    Note that this step might take several minutes to complete.

  3. After you create the connection, run the following commands to retrieve the gateway URIs: 

    GATEWAY_URI=$(gcurl https://beyondcorp.googleapis.com/v1/projects/${PROJECT_NUMBER}/locations/${REGION}/appConnections/${CONNECTION_NAME} | jq -r '.gateway.uri')
GATEWAY_URI_L7="${GATEWAY_URI/bce-psc/'bce-ps7'}"

    The GATEWAY_URI_L7 is used to create a forwarding rule. The BeyondCorp Enterprise app connector uses Private Service Connect to link the consumer project to the managed resources in Google managed projects.

Configuring an external Application Load Balancer

You can connect only HTTPS applications to an external Application Load Balancer. HTTP applications are not supported.

gcloud CLI

Before you complete the following steps, ensure that you have the gcloud CLI SDK installed and you are logged in with an account that has the beyondcorp.admin role.

  1. Set the required environment variables by running the following commands: 

    LB_PREFIX=web-app
PROJECT_ID=my-project
REGION=us-central1
DOMAIN_NAME=app.example.com
    Replace the following:

    • web-app: the prefix appended to the names of all the load balancer resources.
    • my-project: the Google Cloud project ID.
    • us-central1: the region in which the connection and gateways will be deployed.
    • app.example.com: the domain name of your application.

  2. Create a Network Endpoint Group (NEG) using the following command:

    gcloud beta compute network-endpoint-groups create $LB_PREFIX-neg \
    --region=$REGION \
    --network-endpoint-type=private-service-connect \
    --psc-target-service=$GATEWAY_URI_L7

    Optional flags:

    • Set the --network flag to specify the name of the network for the network endpoints in the NEG. If not specified, the default project network is used.
    • Set the --subnet flag to specify the name of the subnetwork for the network endpoints in the NEG. If not specified, the network endpoint can belong to any subnetwork in the region where the network endpoint group is created.

  3. Create an NEG based backend service and connect it to a Private Service Connect service attachment by running the following commands:

    gcloud beta compute backend-services create $LB_PREFIX-backend-service \
    --global \
    --protocol=HTTPS \
    --load-balancing-scheme=EXTERNAL_MANAGED

gcloud beta compute backend-services add-backend $LB_PREFIX-backend-service \
    --global \
    --network-endpoint-group=$LB_PREFIX-neg \
    --network-endpoint-group-region=$REGION

  4. Create a Google managed certificate for your domain by running the following commands:

    gcloud compute addresses create $LB_PREFIX-static-ip \
    --ip-version=IPV4 \
    --global
gcloud compute ssl-certificates create $LB_PREFIX-ssl-cert \
    --domains=$DOMAIN_NAME \
    --global

LB_IP="$(gcloud compute addresses describe $LB_PREFIX-static-ip --global --format='value(address)')"

  5. Create an external Application Load Balancer based on the backend from the previous step by running the following commands:

    gcloud compute url-maps create $LB_PREFIX-map-https \
    --default-service=$LB_PREFIX-backend-service

gcloud compute target-https-proxies create $LB_PREFIX-https-lb-proxy \
    --url-map=$LB_PREFIX-map-https \
    --ssl-certificates=$LB_PREFIX-ssl-cert

gcloud beta compute forwarding-rules create "$LB_PREFIX-https-lb-forwarding-rule" \
    --global \
    --target-https-proxy=$LB_PREFIX-https-lb-proxy \
    --address=$LB_IP \
    --ports=443 \
    --load-balancing-scheme=EXTERNAL_MANAGED

    After the external Application Load Balancer is created, your application is accessible over the internet by that IP address.

  6. After the external Application Load Balancer is created, you must associate its IP address with the domain name so that Google Cloud can provision an SSL certificate. Use your DNS provider instructions for how to associate the IP address with the DNS name. Run the following command to check the provisioning status:

    1. Show the IP address to set up in the DNS provider: 
      echo "Load Balancer ip address - $LB_IP"
    2. After you set the DNS, check whether or not the domain name resolves to the IP address by running the following command: 
      dig $DOMAIN_NAME
    3. Get the provisioning status by running the following command:
    gcloud compute ssl-certificates describe $LB_PREFIX-ssl-cert \
    --global \
    --format="get(name,managed.status, managed.domainStatus)"

    After the SSL Certificate is provisioned, you should be able to access your application by using the DNS name.

API

  1. Set the required environment variables by running the following commands: 

    LB_PREFIX=web-app
PROJECT_ID=my-project
REGION=us-central1
DOMAIN_NAME=app.example.com
    Replace the following:

    • web-app: the prefix appended to the names of all the load balancer resources.
    • my-project: the Google Cloud project ID.
    • us-central1: the region in which the connection and gateways will be deployed.
    • app.example.com: the domain name of your application.

  2. Create a Network Endpoint Group (NEG) using the following command:

    gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/regions/${REGION}/networkEndpointGroups \
-d "{ \
    'name': '${LB_PREFIX}-neg', \
    'network_endpoint_type': 'private-service-connect', \
    'psc_target_service': '${GATEWAY_URI_L7}' \
}"

    Optional fields:

    • Set the network field to specify the URL of the network for the network endpoints in the NEG. If not specified, the default project network is used.
    • Set the subnetwork field to specify the URL of the subnetwork for the network endpoints in the NEG. If not specified, the network endpoint can belong to any subnetwork in the region where the network endpoint group is created.

  3. Create an NEG based backend service and connect it to a Private Service Connect service attachment by running the following commands:

    gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/backendServices \
-d "{ \
    'name': '${LB_PREFIX}-backend-service', \
    'service_protocol': 'HTTPS', \
    'port_name': 'https', \
    'load_balancing_scheme': 'EXTERNAL_MANAGED' \
}"

gcurl -X PATCH https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/backendServices/${LB_PREFIX}-backend-service \
-d "{ \
    'backends': \
    [{ \
        'group': 'https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/regions/${REGION}/networkEndpointGroups/${LB_PREFIX}-neg' \
    }] \
}"

  4. Create a Google managed certificate for your domain by running the following commands:

    gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/addresses \
-d "{ \
    'name': '${LB_PREFIX}-static-ip', \
    'ip_version': 'IPV4' \
}"

gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/sslCertificates \
-d "{ \
    'name': '${LB_PREFIX}-ssl-cert', \
    'managed': \
    { \
        'domains': '${DOMAIN_NAME}' \
    }, \
    'type': 'MANAGED' \
}"

LB_IP=$(gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/addresses/${LB_PREFIX}-static-ip | jq -r '.address')

  5. Create an external Application Load Balancer based on the backend from the previous step by running the following commands:

    gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/urlMaps \
-d "{ \
    'name': '${LB_PREFIX}-map-https', \
    'default_service': 'https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/backendServices/${LB_PREFIX}-backend-service' \
}"

gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/targetHttpsProxies \
-d "{ \
    'name': '${LB_PREFIX}-https-lb-proxy', \
    'url_map': 'https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/urlMaps/${LB_PREFIX}-map-https', \
    'ssl_certificate': 'https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/sslCertificates/${LB_PREFIX}-ssl-cert' \
}"

gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/forwardingRules \
-d "{ \
    'name': '${LB_PREFIX}-https-lb-forwarding-rule', \
    target: 'https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/targetHttpsProxies/${LB_PREFIX}-https-lb-proxy', \
    'ip_address': '${LB_IP}', \
    'port_range': '443-443', \
    'load_balancing_scheme': 'EXTERNAL_MANAGED' \
}"

    After the external Application Load Balancer is created, your application is accessible over the internet by that IP address.

  6. After the external Application Load Balancer is created, you must associate its IP address with the domain name so that Google Cloud can provision an SSL certificate. Use your DNS provider instructions for how to associate the IP address with the DNS name. Run the following command to check the provisioning status:

    1. Show the IP address to set up in the DNS provider: 
      echo "Load Balancer ip address - $LB_IP"
    2. After you set the DNS, check whether or not the domain name resolves to the IP address by running the following command: 
      dig $DOMAIN_NAME
    3. Get the provisioning status by running the following command:
    echo $(gcurl https://www.googleapis.com/compute/v1/projects/${PROJECT_NUMBER}/global/sslCertificates/${LB_PREFIX}-ssl-cert \
| jq -r '.name + " " + .managed.status + " " + (.managed.domainStatus | keys[]) + "=" + .managed.domainStatus[]')

    After the SSL Certificate is provisioned, you should be able to access your application by using the DNS name.

Enabling Identity-Aware Proxy

Follow the Identity-Aware Proxy (IAP) documentation for setting up Identity-Aware Proxy. After you set up IAP, enable IAP for the backend service by completing the following steps:

  1. Go to the IAP page:
    IAP

  2. Under HTTPS RESOURCES, find $LB_PREFIX-backend-service. To enable IAP for this resource, click the toggle in the IAP column. In the Turn On IAP window that appears, select the checkbox to confirm the configuration requirements of this resource and click TURN ON to confirm that you want IAP to secure your resource.

  3. To allow users access to your application, you must grant them the IAP-secured Web App User role for the project by completing the following steps:

    1. Select the $LB_PREFIX-backend-service checkbox.
    2. On the Info panel, select ADD PRINCIPAL, and then enter the email addresses of the groups or individuals who should have access to your application.
    3. In the Select a role drop-down list, select Cloud IAP > IAP-secured Web App User, and then click Save.

Your on-premises application should now be published to the internet and protected by BeyondCorp Enterprise.

Set up the app connector using the Google Cloud console

Complete the procedures in this section to deploy the BeyondCorp Enterprise app connector by using the Google Cloud console.

Enter your application details

Add your application's details by completing the following steps:

  1. In the Google Cloud console, go to the IAP page.
    Go to IAP

  2. Click the CONNECT NEW APPLICATION button, and then select Connect via App Connector.

  3. For Application name, enter the name of the application that you want to secure.

  4. For External facing application URL, enter a publicly accessible URL for your users to access your application.

  5. Click Next.

Set up the connection to Google Cloud

Complete the following steps to establish connectivity between Google Cloud and your non-Google Cloud environment.

  1. Click the CREATE APP CONNECTOR button, and enter the following information:

    • App connector name: A name for the connector.
    • Region: The region in which to deploy the connection gateways and connector.

  2. Click the CREATE button.

  3. Follow the instructions on the Provision a virtual machine pane to deploy the remote agent on your VM.

  4. Click the TEST CONNECTION STATUS button to test the connection.

  5. Configure the connector you created in the previous step by entering the following information in Where can Google Cloud find your application?:

    • Internal hostname: The on-premises IP address or the FQDN that hosts your application.
    • Port: The port number to connect to your application.

Grant access to your application (optional)

  1. For New principals, enter one or more of the following:

    • Google Account email: user@gmail.com
    • Google Group: admins@googlegroups.com
    • Service account: server@example.gserviceaccount.com
    • Google Workspace domain: example.com
    • anybody: enter allUsers to grant all users access
    • all Google Accounts: allAuthenticatedUsers to grant access to any user signed in to a Google Account

  2. Select one or more access levels, and then click NEXT.

Publish your application

  1. To publish your application, click the SECURELY PUBLISH APPLICATION button.

  2. After you publish your application and you receive the Securely published application confirmation, you must create a DNS record to point your external facing application URL to the IP address of the proxy. Complete the following steps:

    • From the STEP 2 - Update DNS section, copy the IP address from the IP address field. Use the address to create a DNS record by following your DNS provider instructions for how to associate the IP address with the DNS name.

  3. After your create the DNS record, test it by clicking the TEST button. You should receive a DNS test passed. confirmation.

  4. Click DONE.

Enable IAP

  1. Go to the IAP page.
    Go to IAP

  2. From the list of resources, locate your application, and then select the button in the IAP column.

  3. In the Turn on IAP dialog that appears, select the I have read the configuration requirements and configured my Backend Service according to documentation. checkbox, and then click TURN ON.

Your on-premises application should now be published to the internet and protected by BeyondCorp Enterprise.

Create an app gateway (Optional)

App gateways are implicitly created and managed when you connect an application by using the app connector. You can also create your own app gateways to organize app connections resources into groups. Traffic for app connections hosted in different app gateways is routed through a separate virtual infrastructure.

To create and specify an app gateway, complete the following steps.

gcloud CLI

  1. Set the required environment variables by running the following command:

    CONNECTOR_NAME=CONNECTOR_NAME
CONNECTION_NAME=CONNECTION_NAME
GATEWAY_NAME=GATEWAY_NAME
PROJECT_ID=PROJECT_ID
REGION=REGION
APP_ENDPOINT=APP_HOST:APP_PORT

    Replace the following:

    • CONNECTOR_NAME: the name of the connector, defined in an earlier step.
    • CONNECTION_NAME: a unique name for the connection.
    • GATEWAY_NAME: the name of the app gateway.
    • PROJECT_ID: the Google Cloud project ID.
    • REGION: the region in which to deploy the app gateway, such as us-central1.
    • APP_HOST: the on-premises IP address or an FQDN that hosts your application.
    • APP_PORT: the port number to connect to your application.

  2. Create an app gateway by running the following command:

    gcloud beta beyondcorp app gateways create $GATEWAY_NAME \
--project=$PROJECT_ID --location=$REGION --display-name=$GATEWAY_NAME

  3. Specify an app gateway in a create connection call by running the following command:

    gcloud beta beyondcorp app connections create $CONNECTION_NAME \
--project=$PROJECT_ID \
--location=$REGION \
--application-endpoint=$APP_ENDPOINT \
--type=tcp \
--connectors=$CONNECTOR_NAME \
--display-name=$CONNECTION_NAME \
--app-gateway=$GATEWAY_NAME

API

  1. Set the required environment variables by running the following command:

    CONNECTOR_NAME=CONNECTOR_NAME
CONNECTION_NAME=CONNECTION_NAME
GATEWAY_NAME=GATEWAY_NAME
PROJECT_ID=PROJECT_ID
REGION=REGION
APP_ENDPOINT=APP_HOST:APP_PORT

    Replace the following:

    • CONNECTOR_NAME: the name of the connector, defined in an earlier step.
    • CONNECTION_NAME: a unique name for the connection.
    • GATEWAY_NAME: the name of the app gateway.
    • PROJECT_ID: the Google Cloud project ID.
    • REGION: the region in which to deploy the app gateway, such as us-central1.
    • APP_HOST: the on-premises IP address or an FQDN that hosts your application.
    • APP_PORT: the port number to connect to your application.

  2. Create an app gateway by running the following command:

    gcurl https://beyondcorp.googleapis.com/v1/projects/${PROJECT_ID}/locations/${REGION}/appGateways?app_gateway_id=${GATEWAY_NAME} \
-d "{ \
'type': 'TCP_PROXY', \
'display_name': '${CONNECTION_NAME}' \
}"

  3. Specify an app gateway in a create connection call by running the following command:

    gcurl https://beyondcorp.googleapis.com/v1/projects/${PROJECT_ID}/locations/${REGION}/appConnections?app_connection_id=${CONNECTION_NAME} \
-d "{ \
'application_endpoint': \
{ \
'host': '${APP_HOST}', \
'port': '${APP_PORT}' \
}, \
'gateway': { 'app_gateway' : 'projects/${PROJECT_ID}/locations/${REGION}/appGateways/${GATEWAY_NAME}'}, \
'connectors':['projects/${PROJECT_ID}/locations/${REGION}/appConnectors/${CONNECTOR_NAME}'], \
'type': 'TCP_PROXY', \
'display_name': '${CONNECTION_NAME}' \

Next steps

Troubleshooting

Can't find previously created resources

When using gcloud CLI version 392.0.0 or later, use the new app connector default resources AppConnector and AppConnection. For example, gcloud alpha beyondcorp app connectors and gcloud alpha beyondcorp app connections both refer to the new resources.

To access legacy resources created using a gcloud CLI version prior to version 392.0.0, use the keyword legacy. For example, gcloud alpha beyondcorp app legacy connectors and gcloud alpha beyondcorp app legacy connections both refer to the legacy resources.

Configurations that use legacy resources continue to work for now, but will be deprecated in the future.

  • Follow the procedures in this guide to configure new resources.
  • If your configuration uses legacy resources, use the keyword legacy to locate those resources and remove them. Recreate the resources by following the procedures in this document.
  • If you have have an incomplete configuration that uses legacy resources, use the keyword legacy to locate those resources and remove them. Recreate the resources by following the procedures in this document.

TLS/SSL errors

ERR_SSL_VERSION_OR_CIPHER_MISMATCH

The browser shows the error ERR_SSL_VERSION_OR_CIPHER_MISMATCH or similar TLS/SSL errors and does not redirect to the login page.

  • Verify the status of the certificates on the Google Cloud load balancer details page.

    NOTE: Provisioning a Google-managed certificate might take up to 60 minutes.

TLS error

The browser shows the error upstream connect error or disconnect/reset before headers. retried and the latest reset reason: connection failure, transport failure reason: TLS error or similar TLS/SSL errors after redirecting to the login page.

  • Verify that the application endpoint used in the Connection is HTTPS.

  • Verify that the app endpoint is accessible from the remote agent VM using curl:

    curl https://$APP_ENDPOINT

    You might need to use the -k flag if your application certificate is self-signed.

Verify the firewall configuration

Ensure that firewalls between the remote agent and the internet allow outbound connections to the following domains:

Connection Type Domain Port
TCP raw.githubusercontent.com 443
TCP gcr.io 443
TCP *.googleapis.com 443
TCP tunnel.cloudproxy.app 443
TCP *.tunnel.cloudproxy.app 443
TCP accounts.google.com 443
TCP oauth2.googleapis.com 443

Change the connectors associated with a connection

gcloud

  1. Set the required environment variables by running the following command:

    CONNECTOR_NAME=my-connector
CONNECTION_NAME=my-connection
REGION=us-central1
PROJECT_ID=my-project

    Replace the following:

    • my-connector: the name of the connector. This can also be a comma-separated list like connector1,connector2.
    • my-connection: the name of the connection to update.
    • us-central1: the region in which the connection is deployed.
    • my-project: the Google Cloud project ID.

  2. To change the connectors associated with a connection, run the following command:

    gcloud alpha beyondcorp app connections update $CONNECTION_NAME \
    --project=$PROJECT_ID \
    --location=$REGION \
    --connectors=$CONNECTOR_NAME

API

  1. Set the required environment variables by running the following command:

    CONNECTOR_NAME=my-connector
CONNECTION_NAME=my-connection
REGION=us-central1
PROJECT_ID=my-project

    Replace the following:

    • my-connector: the name of the connector.
    • my-connection: the name of the connection to be updated.
    • us-central1: the region in which the connection is deployed.
    • my-project: the Google Cloud project ID.

  2. To change the connectors associated with a connection, run the following command:

    gcurl -X PATCH \
https://beyondcorp.googleapis.com/v1/projects/${PROJECT_ID}/locations/${REGION}/appConnections/${CONNECTION_NAME}?update_mask=connectors \
-d "{ \
    'connectors': ['projects/${PROJECT_ID}/locations/${REGION}/appConnectors/${CONNECTOR_NAME}'], \
}"

    You can verify the update by running the following command and checking the connectors field of the output:

    gcurl https://beyondcorp.googleapis.com/v1/projects/${PROJECT_ID}/locations/${REGION}/appConnections/${CONNECTION_NAME}

Proxy

Support for proxies was introduced in early 2024 and requires changes to the BeyondCorp systemd file. If you installed the remote agent earlier than this time and want to use a proxy, re-install the remote agent.

  • Verify that the Docker daemon is configured correctly to work with proxies.

  • Verify that both the HTTP and HTTPS proxy environment variables are set, and that both are using the http:// schema. The https:// schema and other schemas are not supported.

  • Verify that the proxy environment variables are exported to subprocesses by running the env command.

  • Only HTTP_PROXY, HTTPS_PROXY, NO_PROXY, and their lowercase equivalents are supported.

  • If you need to update the proxy settings after installation, update the environment variables in /var/beyondcorp/env/PROXY if using the default beyondcorp user. If using a custom user, update /home/$USER/.docker/config.json according to the instructions for configuring the Docker client. To apply the proxy changes, restart the remote agent using the following command: 

    sudo systemctl restart beyondcorp

Diagnostics tools

run-post-install-checks

run-post-install-checks is a script installed in /var/beyondcorp/scripts/ that verifies whether your remote agent is properly installed, enrolled, and healthy. After you successfully create an app connection and assign it to a remote agent (connector), go to the remote agent VM, and run run-post-install-checks with the following command:

sudo /var/beyondcorp/scripts/run-post-install-checks

The script prints a success message if everything is in order.

run-diagnostics

run-diagnostics is a script installed in /var/beyondcorp/scripts/ that diagnoses common issues on the remote agent VM and prints a report that you can share with the BeyondCorp Enterprise support team. To run this diagnostics script, run the following command:

sudo /var/beyondcorp/scripts/run-diagnostics

The diagnostics report is written to the console as well as to ./diagnostics-report.txt

Remote Agent CLI

bce-connctl is the remote agent admin command-line interface (CLI) for interacting with the remote agent locally. This CLI supports various commands, such as for enrolling the remote agent, checking status, and setting config values.

Init command

You can use the Init command to initialize the remote agent and generate a script to enroll the agent.

Example: 

bce-connctl init --project=$PROJECT_ID --region=$REGION --connector=$CONNECTOR_NAME

Status commands

You can use the Status commands to manage the status of the remote agent.

  • List: Use this command to list the overall status of the remote agent, as well as the status of any connections supported by this agent. Example: bce-connctl status list

Config commands

You can use the Config commands to manage configuration values for your remote agent.

  • List: Use this command to list the configuration values of the remote agent. Example: bce-connctl config list
  • Set: Use this command to set a configuration value of the remote agent. Example: bce-connctl config set <key> <value>

Enrollment commands

You can use the Enrollment commands to manage the enrollment of the remote agent.

  • Describe: Use this command to get the enrollment status of the remote agent. Example: bce-connctl enrollment describe

    When the enrollment script from bce-connctl init completes successfully, and the public key uploads successfully, the status is ENROLLED.

Help

You can append the -h or --help flag to any bce-connctl command to print usage information.

bce-connctl init -h