gRPC direct connectivity

Direct connectivity is a connection solution that allows high-performance, authenticated, direct gRPC network connections between a Google Cloud client library and Cloud Storage, resulting in lower latency and connection overhead. When you use gRPC to connect to Google Cloud using direct connectivity, requests initiated through supported Google Cloud client libraries are routed directly to Cloud Storage, bypassing Google Front Ends (GFEs).

Direct connectivity is only available for requests made from Compute Engine virtual machines (VMs).

Requirements for direct connectivity

Direct connectivity is enabled by default when you use supported Cloud Storage client libraries to connect to Cloud Storage, but becomes available only if the following conditions are all met:

  • The Compute Engine VMs interacting with Cloud Storage must have an attached service account, even if the service account has no permissions. The service account is used to represent the Compute Engine VM in the Application Layer Transport Security handshake process.

  • The Compute Engine VMs interacting with a Cloud Storage bucket must be co-located with the bucket. For example, if the bucket is in us-central1, the VM can be located in us-central1-a. If the bucket is in a multi-region or dual-region, the VM must be located in a region that makes up the multi-region or dual-region. For example, if a bucket is located in the us multi-region, the VM can be located in us-east4-c.

    For more information about bucket locations, see Locations.

  • Your routes and firewall rules allow IPv4 traffic to reach 34.126.0.0/18 and IPv6 traffic to reach 2001:4860:8040::/42. In addition, traffic must be allowed to reach the endpoints storage.googleapis.com:443 and directpath-pa.googleapis.com:443.

    To learn how to check whether a connection can be made to these endpoints, see Check network configurations. For information about setting up routes, see Configure routes.

  • Cloud Storage client libraries have specific credential requirements in order to authenticate applications to Cloud Storage and have direct connectivity be available:

    • The Java client library requires either user account credentials or service account credentials.

    • The C++ client library requires service account credentials.

    • The Go client library requires service account credentials. Versions before 1.52.0 require the default service account credentials of the VM accessing your bucket.

    To view the name of the account that your VM uses to authenticate to Cloud Storage, use the gcloud auth list command:

    gcloud auth list --filter=status:ACTIVE --format="value(account)"

    If you're using the Go client library and need to check if the Google account being used to authenticate matches your VM's default service account, use the gcloud compute instances describe command:

    gcloud compute instances describe INSTANCE_NAME --format='yaml(serviceAccounts)'

    Replace INSTANCE_NAME with the name of your instance.

    For more information about authentication, see Overview of Google identity management.

Check connectivity by using the Google Cloud CLI

The Google Cloud CLI can test the diagnostics of direct connectivity on your bucket by issuing a GET call to the bucket using gcloud storage buckets describe gs://example-bucket. This section describes how to perform a diagnostic test for direct connectivity by using the Google Cloud CLI.

Before you begin

  1. Make sure you have the storage.buckets.get IAM permission on the bucket, which can be granted using the Storage Legacy Bucket Reader (roles/storage.legacyBucketReader) role.

  2. The gcloud alpha storage diagnose command is only available in the Google Cloud CLI version 531.0.0 or later. To use this command, we recommend upgrading to the latest Google Cloud CLI version.

Run the diagnosis

To check whether direct connectivity is available, run gcloud alpha storage diagnose with the argument --test-type=DIRECT_CONNECTIVITY:

gcloud alpha storage diagnose --test-type=DIRECT_CONNECTIVITY gs://BUCKET_NAME

Replace the following:

  • BUCKET_NAME: the name of your bucket. For example, my-bucket.

If a connection can be made, you receive a response similar to the following:

Using my-bucket bucket for the diagnostic tests.
Bucket location : US
Bucket storage class : STANDARD
Running diagnostic: Direct Connectivity Diagnostic...
WARNING: This diagnostic is experimental. The output may change, and checks may be added or removed at any time. Please do not rely on the diagnostic being present.
Finished running diagnostic: Direct Connectivity Diagnostic
Generating diagnostic report...
NAME
Direct Connectivity Diagnostic
    ┌────────────────────────────┬─────────────────────────────────────────────────────────────────────────┬──────────┐
    │            NAME            │                           PAYLOAD_DESCRIPTION                           │  RESULT  │
    ├────────────────────────────┼─────────────────────────────────────────────────────────────────────────┼──────────┤
    │ Direct Connectivity Call   │ Able to get bucket metadata using Direct Connectivity network path.     │ Success. │
    └────────────────────────────┴─────────────────────────────────────────────────────────────────────────┴──────────┘

Check network configurations

If you encounter an Unable to connect to Traffic Director error while checking for direct connectivity using the Google Cloud CLI, use the following instructions to ensure that your network configurations allow traffic to pass from your VMs to the required endpoints.

Direct connectivity requires a service hosted at directpath-pa.googleapis.com:443 and at storage.googleapis.com:443. To check whether a connection can be made to the service, make a curl call to directpath-pa.googleapis.com:443 or storage.googleapis.com:443. For example:

curl directpath-pa.googleapis.com:443

If a connection can be made, you receive an empty response without errors:

curl: (52) Empty reply from server

If a connection cannot be made, you receive an error similar to the following:

curl: (56) Recv failure: Connection reset by peer

Limitations

Direct connectivity doesn't support Private Service Connect.