Connectivity Tests overview

Connectivity Tests is a static configuration analyzer and diagnostics tool that enables you to check connectivity in the following ways:

  • Between source and destination endpoints in your Virtual Private Cloud (VPC) network
  • Between your VPC network and Google-managed services, such as Google Kubernetes Engine (GKE) cluster masters and Cloud SQL instances (alpha)
  • From your VPC network to and from the internet
  • From your VPC network to and from your on-premises network

Connectivity Tests simulates the expected forwarding path of a test packet through your VPC network, Cloud VPN tunnels, or Cloud Interconnect attachments to your on-premises network. Connectivity Tests can also simulate the expected inbound forwarding path to resources in your VPC network.

The API for Connectivity Tests is the Network Management API. For more information, see the API documentation.

Why use Connectivity Tests?

Connectivity Tests can help you troubleshoot the following network connectivity issues due to misconfiguration:

  • Inconsistent configurations that are unintended
  • Obsolete configurations due to network configuration changes or migrations
  • Configuration errors for a variety of network services and functions

When testing Google-managed services (alpha), Connectivity Tests can also help you determine whether there is an issue in your VPC network or in the Google-owned VPC network used for the service resources.

How Connectivity Tests analyzes configurations

Connectivity Tests performs a static reachability analysis to evaluate configurations in your Google Cloud network. This type of analysis does not evaluate the actual data plane.

To perform this analysis, Connectivity Tests uses an abstract state machine to model how a VPC network should process packets.

Google Cloud processes a packet in several logical steps. Due to the variety of VPC network services and features that Connectivity Tests supports, a test packet traversing a VPC network configuration can take many possible paths.

The following diagram shows a model for how Connectivity Tests simulates trace traffic between two virtual machine (VM) instances—the VM box on the left and another on the right.

Depending on your Google Cloud network and resource configurations, this traffic might go through a Cloud VPN tunnel, a Google Cloud load balancer, or a peered VPC network before reaching the destination VM instance.

The network abstract state machine.
The network abstract state machine

The bounded number of steps between discrete states until a packet has been delivered or dropped is modeled in Connectivity Tests as a finite state machine. This finite state machine can be in exactly one of many finite states at any one time and might have multiple successor states.

For example, when Connectivity Tests matches several routes according to route precedence, Google Cloud can choose a route among several routes based on an unspecified hashing function in the data plane.

In the previous case, the Connectivity Tests trace returns all of the possible routes but can't determine the method Google Cloud used to return the routes. This is because that method is internal to Google Cloud and is subject to change.

Google-managed services (alpha)

Google-managed services, such as Cloud SQL and GKE, allocate resources for customers in projects and VPC networks that Google owns and manages. Customers do not have permission to access these resources.

Connectivity Tests can still run a test and provide an overall reachability result for Google-managed services, but it does not provide details for the tested resources in the project owned by Google.

The following diagram shows a model for how Connectivity Tests simulates trace traffic from a VM instance in a customer VPC network to a Cloud SQL instance in the Google-owned VPC network. In this example, the networks are connected through VPC Network Peering.

Similar to a standard test between two VMs, the logical steps include checking the egress firewall and matching the route. When you run a test, Connectivity Tests provides details about these steps. However, for the final logical step of analyzing the configuration in the Google-owned VPC network, Connectivity Tests provides only an overall reachability result. Connectivity Tests does not provide details for the resources in the Google-owned project because you do not have permission to view them.

For more information, see the test examples in the Testing Google-managed services (alpha) section in the Common use cases document.

The network abstract state machine for Google-managed services.
The network abstract state machine for Google-managed services

Supported configurations

Connectivity Tests supports testing the network configurations as described in the following sections.

Traffic flows

  • VM instances to and from the internet
  • VM instance to VM instance
  • From Google Cloud to and from on-premises networks

VPC networking features

You can test connectivity between resources that use the following features:

Google Cloud hybrid networking solutions

The following hybrid networking solutions are supported:

Cloud NAT

Cloud NAT is supported.

Cloud Load Balancing

  • Connectivity to the following Google Cloud load balancers is supported: external HTTP(S) load balancers, network load balancers, internal HTTP(S) load balancers, internal TCP/UDP load balancers, TCP proxy load balancers, and SSL proxy load balancers.
  • Connectivity to load balancer IP addresses is supported.
  • Connectivity of Cloud Load Balancing health checks to backends is supported. Backends can be instance groups or network endpoint groups (NEGs).

For Cloud Load Balancing features that are unsupported, see the Unsupported configurations section.

Google Kubernetes Engine (GKE)

  • Connectivity to and between GKE nodes and the GKE master is supported.
    • When testing the private IP address of a GKE cluster master, Connectivity Tests determines whether the packet can be delivered to the master. This includes analyzing the configuration within the Google-owned VPC network. This is an alpha feature.
    • When testing the public IP address of a GKE cluster master, Connectivity Tests determines whether the packet can be sent to the Google-owned VPC network where the master runs. This does not include analyzing the configuration within the Google-owned VPC network.
  • Connectivity to the GKE service through Cloud Load Balancing is supported.
  • Connectivity to a GKE pod in a VPC-native cluster is supported.

For GKE features that are unsupported, see the Unsupported configurations section.

Other Google Cloud products and services

The following additional Google Cloud products or services are supported:

  • Cloud SQL instances are supported.
    • When testing the private IP address of a Cloud SQL instance, Connectivity Tests determines whether the packet can be delivered to the instance. This includes analyzing the configuration within the Google-owned VPC network. This is an alpha feature.
    • When testing the public IP address of a Cloud SQL instance, Connectivity Tests determines whether the packet can be sent to the Google-owned VPC network where the instance runs. This does not include analyzing the configuration within the Google-owned VPC network.

Unsupported configurations

Connectivity Tests does not support testing the following network configurations:

  • Google Cloud Armor policies are not considered or used when tracing connectivity to an external HTTP(S) load balancer IP address.
  • GKE network policy. Connectivity Tests does not incorporate network policies when tracing connectivity to IP addresses within GKE clusters.
  • An external HTTP(S) load balancer that uses Cloud Storage buckets is not supported.
  • Cloud SQL instances that use public IP addresses are not supported.

Considerations

Evaluate the following considerations when deciding whether to use Connectivity Tests.

Connectivity Tests tests configurations, not the data plane

The static analysis that Connectivity Tests performs is based entirely on configuration information for Google Cloud resources and might not represent the actual condition or status of the data plane for a VPC network.

While Connectivity Tests does acquire some dynamic configuration information, such as the Cloud VPN tunnel state and dynamic routes on Cloud Router, it does not access or maintain the health state of Google internal production infrastructure and data plane components. It also does not perform active probing of the data plane or measure performance data such as packet drop rate and latency.

A status of Packet could be delivered for a Connectivity Test does not guarantee that traffic can pass through the data plane. The purpose of the test is to validate configuration issues that can cause traffic drop.

For some network issues, Connectivity Tests might tell you that the trace packet can be delivered but network connectivity is still blocked. In such cases, you can narrow down the possible causes in the network configuration to fix the issue in the network data plane itself (for example, issues such as a physical or other failure).

Connectivity Tests has no knowledge of networks outside of Google Cloud

Outside networks are defined as the following:

  • On-premises networks that reside in your data center or other facility where you operate your hardware devices and software applications.
  • Other cloud providers where you run resources.
  • A host on the internet that sends traffic to your VPC network.

Connectivity Tests doesn't perform firewall connection tracking

Connection tracking for VPC firewalls stores information about new and established connections and enables allowing or restricting subsequent traffic based on that information.

Connectivity Tests doesn't support firewall connection tracking because the firewall connection table is located in the data plane for a VM instance and is inaccessible. However, Connectivity Tests can simulate connection tracking by allowing a return connection that would normally be denied by an ingress firewall rule, as long as Connectivity Tests initiates the outbound connection.

Connectivity Tests can't test VM instances configured to modify forwarding behavior

Connectivity Tests can't test VM instances that have been configured to act in the data plane as routers, firewalls, NAT gateways, VPNs, and so on. This type of configuration makes it difficult to assess the environment running on the VM instance.

Connectivity Tests result times can vary

Getting Connectivity Tests results can take from 30 seconds to up to 10 minutes. The time a test takes is based on the size of your VPC network configuration and the number of Google Cloud resources that you use.

The following table shows response times that you can expect for all users running a test against a sample configuration in a query. This configuration contains VM instances, a Cloud VPN tunnel, and Google Cloud load balancers.

Response times per query
Project size Number of Google Cloud resources Response latency
Small project Fewer than 50 60 seconds for 95% of queries from all users
Medium project Greater than 50 but fewer than 5000 120 seconds for 95% of queries from all users
Large project Greater than 5000 600 seconds for 95% of queries from all users

What is reachability?

Reachability is the ability to reach a Google Cloud resource or determine whether a path exists from one Google Cloud resource to another. This term comes from graph theory as applied to computer networks. The entire reachability graph contains all of the network endpoints and the reachability between each pair of network endpoints.

Reachability analysis is a more generic term that describes a body of analysis that can be conducted to determine network reachability. One of the use cases from a reachability analysis is a connectivity test. Connectivity, in this case, refers to the state of network connections.

For every step along the network forwarding path, a reachability analysis tests and provides results for the underlying network configuration (for example, the routes and firewall rules that are applied to the simulated test packet).

How Connectivity Tests works

This section describes how Connectivity Tests and its components work.

Connectivity Tests performs a static reachability analysis that evaluates the Google Cloud resources in your testing path against an ideal configuration model, rather than against the live data plane.

As a network administrator, you have full control of all configurations that could impact the analysis result, except for the configuration of VPC networks that host Google-managed services such as Cloud SQL instances. For example, your firewall rule configurations control the types of traffic that you allow between VM instances.

When you run a Connectivity Test, you input a specific set of parameters and receive formatted results in the form of a network trace, or query. A Connectivity Test generates more than one trace if a test has multiple possible paths in the network (for example, when the destination endpoint is a Google Cloud load balancer with multiple backends).

  • A match means that Connectivity Tests finds a Google Cloud configuration that allows the simulated packet to continue through the testing path.
  • No match means that Connectivity Tests can't find a match. Thus, the configuration doesn't exist.
  • A denied match means that Connectivity Tests finds a Google Cloud configuration where the simulated test packet must be dropped.

Connectivity Tests components

A Connectivity Test is the top-level component that contains all other testing sub-components such as the following:

  • Source and destination endpoints
  • Reachability details for the test and its traces, including an overall reachability result
  • One or more traces that each contain one or more steps
  • A state for each step

Each test has a unique name and each step has a state and Info metadata associated with it. For example, if a step checks a route, RouteInfo metadata is included in that step.

The following diagram shows a test from one Compute Engine VM instance to another. For descriptions of the test components, see the sections that follow.

State machine for a VM-to-VM trace.
State machine for a VM-to-VM trace

Source and destination endpoints

Connectivity Tests supports a 5-tuple packet header without the source port. This is because the source port is not used for validating resources in Google Cloud network configurations. Thus, you don't need to provide it when running tests.

The packet header contains the following components:

  • A network protocol
  • A source endpoint, consisting of one of the following:
    • A source IP address
    • A VM instance name
    • A cluster name for a GKE master (alpha)
    • A Cloud SQL instance name (alpha)
  • A destination endpoint, consisting of one of the following and a port number:
    • A destination IP address
    • A VM instance name
    • A cluster name for a GKE master (alpha)
    • A Cloud SQL instance name (alpha)

You can also specify a Google Cloud or non-Google Cloud network type or a combination of a network type and IP address or VM instance name to uniquely identify a network location.

The following destination protocols are supported for any supported Google Cloud resource:

  • TCP
  • UDP
  • ICMP
  • ESP
  • AH
  • SCTP
  • IPIP

Destination ports for the TCP or UDP protocols are supported. If you don't specify a port, the default setting is port 80.

Traces, steps, and states

A Connectivity Test contains one or more traces. Each trace represents one unique, simulated packet forwarding path in a test.

  • Each trace contains multiple ordered steps.
  • Each step contains a state related to the Google Cloud configuration that Connectivity Tests checks for that step.
  • States are categorized into non-final and final states.

Non-final states

Non-final states represent a configuration check for each Google Cloud resource in the testing path, such as a VM instance, endpoint, VPC firewall, route, or Google Cloud load balancer.

There are four non-final states:

  • Initial
  • Config checking
  • Forwarding
  • Transition

For more information, see Test states.

Final state

Each trace must end with a final state, which is the last step in the trace.

There are four possible final states:

  • Drop
  • Abort
  • Forward
  • Deliver

Each state has a reason associated with it. For more information, see the details for each final state.

Overall reachability result

Connectivity Tests also provides an overall reachability result that can take one of four values: Reachable, Unreachable, Ambiguous, or Undetermined.

Knowing the overall reachability result can be helpful for setting up monitoring or automation.

For more information, see Overall reachability result.

Spoof checking

Connectivity Tests performs a spoof check when a simulated packet to or from a VM instance uses an IP address not owned by that instance. IP addresses owned by a VM include all VM internal IP addresses and secondary IP addresses.

If the address is an address that appears to originate from external traffic, also called a foreign address, then the IP address fails the spoof check.

Metadata

Each state can have metadata associated with it in the form of an Info field. For example, InstanceInfo contains details for a VM instance, including the name and IP address.

Connectivity Tests provides metadata for the test itself and metadata for each step in a test.

What's next