Common use cases for Connectivity Tests

Connectivity Tests evaluates whether traffic can travel successfully between endpoints within your network. It assesses reachability by analyzing your network configuration and, in certain use cases, by sending probe packets.

This page describes common use cases for Connectivity Tests that fall into five categories:

  • Testing within Virtual Private Cloud (VPC) networks, including networks that use Shared VPC and VPC Network Peering
  • Testing Google-managed services (preview)
  • Testing from a VPC network to a non-VPC network, such as an on-premises data center
  • Testing from a non-VPC network to a VPC network
  • Testing Google Cloud load balancers

Detecting invalid or inconsistent configurations describes how to handle those scenarios.

The use case for testing from one virtual machine (VM) instance to another describes a full end-to-end testing scenario, including the test results provided in the console.

For instructions about how to run tests, see Running Connectivity Tests.

Legend for trace diagrams

Trace diagrams on this page use the symbols described in the following legend.

Symbol Name Meaning
Legend for packet trace diagram: gray diamond.
Gray diamond
Checkpoint A decision point where Connectivity Tests checks a configuration and decides if a trace packet should be forwarded, delivered, or dropped.
Legend for packet trace diagram: blue rectangle.
Blue rectangle
Hop A step in the forwarding path for a trace packet, representing a Google Cloud resource that forwards a packet to the next hop in a VPC network (for example, to a Cloud Load Balancing proxy or to a Cloud VPN tunnel).
Legend for packet trace diagram: orange hexagon.
Orange hexagon
Endpoint The source or destination of a trace packet.

Testing within VPC networks

A common use case for Connectivity Tests is testing between two Compute Engine virtual machine (VM) instances in the same or peered VPC networks. For this type of test, Connectivity Tests evaluates reachability by using both configuration analysis and dynamic verification.

To analyze the configuration, Connectivity Tests identifies and evaluates a trace path.

The following diagram shows the typical trace path between two VM instances. The Match routes object can represent routes that direct traffic in a single VPC network or between two peered VPC networks.

Source VM to destination VM trace.
Source VM to destination VM trace

The following steps describe the checkpoints that correspond to each point in the trace diagram. At each checkpoint, the check might fail. The query results show the reason for each failure. For a complete list of test states and messages, see Configuration analysis states reference.

  1. Connectivity Tests verifies that the source VM can send egress packets with the specified source IP address, or can otherwise default to the primary internal IP address. If not, that VM must have IP forwarding enabled.
  2. Connectivity Tests performs a spoof check when a simulated packet to or from a VM instance uses an IP address not owned by that instance. IP addresses owned by a VM include all VM internal IP addresses and secondary IP addresses.

    If the address is an address that appears to originate from external traffic, also called a foreign address, then the IP address fails the spoof check.

  3. To determine whether trace packets can be sent from the source, Connectivity Tests verifies the appropriate egress firewall rules. As part of this process, Connectivity Tests begins by evaluating any hierarchical firewall policy rules that exist. For details about how hierarchical firewall policy rules and VPC firewall rules affect connectivity, see Hierarchical firewall policy examples.
  4. Connectivity Tests finds (matches) a route for the destination IP address, according to the routing order. If no other routes are available to the destination VM instance, Connectivity Tests uses the default static route with the next hop as the internet gateway. All VPC networks use this default route unless you have removed it.
  5. Connectivity Tests verifies that the network's ingress firewall rules allow the packet to arrive at the destination VM. Again, Connectivity Tests begins by evaluating any hierarchical firewall policy rules that exist.
  6. If needed, Connectivity Tests runs a spoof check on the packet that arrives at the second VM.
  7. Connectivity Tests verifies that the destination VM can receive a packet with the specified destination IP address. If this address is a foreign IP address, the destination VM must have IP forwarding enabled. A foreign IP address is an address that does not belong to the VM.

Console

The following screenshot from the Google Cloud Console shows the results of a VM-to-VM test.

The configuration analysis shows a result of Packet could be delivered (in the API response, this label corresponds to a final state of Deliver).

This result shows that the configuration analysis has validated network connectivity for every Google Cloud resource in the path from the source VM to the destination VM. In this case, the route included two VPC firewall rules: an implied VPC firewall rule (named default) and one that was created for this VPC network.

Additionally, Connectivity Tests dynamically verified that the destination VM is reachable by using active probing. The Last packet transmission result field shows the details of this result.

Cloud Console screenshot for VM-to-VM trace.
Cloud Console screenshot for VM-to-VM trace

You can expand each of the cards in the trace path to view more detail.

The following example shows an expanded card for an ingress firewall rule. This card includes information about the VPC network, the configured action for the firewall rule (allow), and the rule priority.

Ingress firewall rule card expanded.
Ingress firewall rule card expanded

When a trace contains a VPC network route with the next hop as a peered VPC network, the start of the trace is not a VM instance, but a VPC network. This type of trace validates firewall rules and routes at the network level because the IP address that you test comes from a network range instead of a VM instance.

Peered networks can exist in the same or in different projects. The following trace example shows peered networks in different projects.

VM-to-VM trace through an accessible peered VPC network in a different project.
VM-to-VM trace through an accessible peered VPC network in a different project

Test failures for VPC networks

The following table lists common failures for tests within VPC networks.

Failure type Description Trace results
Blocked by a firewall rule Traffic leaving a source or destination endpoint is blocked by a hierarchical firewall policy rule or a VPC firewall rule.
  • If connectivity is blocked by a hierarchical firewall policy rule, the trace includes the name of the policy. Be aware that the person running the test might not have permission to view the policy details. For more details about this situation, see Troubleshooting.
  • If connectivity is blocked by a VPC firewall rule, the trace lists the name of the relevant ingress or egress firewall rule.
No matching route A route to the destination endpoint could not be found.
  • If the source and destination VM instances are in different VPC networks and those networks aren't peered, the analysis determines that Packet could be dropped
  • If the VMs are in the same network, but a matching route is not found, traffic is sent on the default static route, with a next hop to internet gateway. In this case, traffic never arrives at the destination VM, and the analysis determines that Packet could be dropped.
  • If there is no route to an internet gateway, the analysis determines that Packet could be dropped.
Instance not running The destination VM instance exists, but is not in a running state. The analysis determines that Packet could be dropped.
Invalid next hop The configured next hop for a VM instance no longer exists, and the route to that instance is invalid. The analysis determines that Packet could be dropped.

Console

The following screenshot illustrates a trace that failed because connectivity was blocked by an ingress hierarchical firewall policy rule.

Cloud Console screenshot for a trace that's blocked by a hierarchical firewall policy rule.
Cloud Console screenshot for a trace that's blocked by a hierarchical firewall policy rule

Test failures for Shared VPC networks

In Shared VPC networks, not having permissions to the host project or the service project can cause the test failures listed in the following table.

Failure type Behavior Trace results
Permissions only to the host project You can't run the trace because you don't have permissions to the service project where the destination IP address is located. The configuration analysis shows a result of Configuration analysis aborted (in the API response, this label corresponds to a final state of Abort).
Permissions only to the service project

You can't run the trace or select the host project network in the Cloud Console because you don't have permission.

Because the host project owns network configurations, a trace against resources in the service project cannot proceed without access to VPC firewall rules, network routes, or IP addresses in the host project.

The overall reachability result is Undetermined because Connectivity Tests can't determine if the packet can be delivered to the destination.

Test failures for VPC Network Peering networks

With VPC Network Peering, not having permission to the peered network's Google Cloud project from the primary network can cause the test failures listed in the following table.

Failure type Behavior Trace results
You have no permissions to the project configuration in the peered VPC network. Connectivity Tests can only trace the configurations in the primary network's project. The configuration analysis shows a result of Packet could be forwarded. This result indicates that a packet would leave the network and be sent to a network that you do not have access to. In this case, the packet has been forwarded to a peered network gateway. In the API response, this state corresponds to a Forward.

The following trace path shows this failure for peered VPC networks.

VM-to-VM trace through an inaccessible peered VPC network in a different project.
VM-to-VM trace through an inaccessible peered VPC network in a different project

Testing Google-managed services

You can get a Connectivity Tests configuration analysis on routes to and from Google-managed services, such as Google Kubernetes Engine (GKE) cluster masters or Cloud SQL instances. Even though you do not have permission to access the Google-owned project where these resources are located, Connectivity Tests can still analyze the configuration of the project's VPC network and provide an overall reachability result. Connectivity Tests does not provide configuration details for analysis within the Google-owned project.

By default, Connectivity Tests attempts to run a test using the private IP address of the Google-managed service endpoint and the VPC Network Peering connection between your network and the Google-owned network. If the endpoint does not have a private IP address, Connectivity Tests uses the public IP address. Connectivity Tests analyzes whether the packet can reach the endpoint, which includes analyzing the configuration within the Google-owned VPC network for the endpoint. If Connectivity Tests detects configuration issues within your project, its analysis stops before reaching the Google-owned network.

The following diagram shows the trace path when testing connectivity to Google-managed services, using a GKE node-to-master test as an example:

Source GKE node to destination GKE master trace.
Source GKE node to destination GKE master trace

Cloud SQL to a VM

This section describes an example test from a Cloud SQL instance to a VM.

Sample inputs

The following table shows example input values for a test from a Cloud SQL instance. To view output for a successful test, continue to the next section.

Input parameter Input value
Protocol TCP
Source Cloud SQL instance instance-1
Source project my-project
Destination VM vm-1
Destination port 80

A successful test

This section provides an example result of a successful test from a Cloud SQL instance.

Console

The following screenshot from the Cloud Console shows a trace from a Cloud SQL instance that indicates an overall result of Reachable.

This result shows that Connectivity Tests has validated network connectivity from the source Cloud SQL instance to the destination VM. This includes analyzing the configuration inside the Google-owned project where the Cloud SQL instance runs. The trace does not provide details for the resources in the Google-owned project because you do not have permission to view them.

Cloud Console screenshot for Cloud SQL to VM trace.
Cloud Console screenshot for Cloud SQL instance to VM trace

GKE node to GKE cluster master

This section describes an example test from a Google Kubernetes Engine (GKE) node to a GKE cluster master.

Sample test inputs

The following table shows example input values for a test to a GKE master. To view output for a successful test, continue to the next section.

Input parameter Input value
Protocol TCP
Source GKE node gke-cluster-1-node-1
Source project my-project
Destination GKE cluster master projects/myproject/locations/us-central1/clusters/cluster-1
Destination port 443

A successful test

This section provides an example result of a successful test to a GKE master.

Console

The following screenshot from the Cloud Console shows a trace to a destination GKE master that indicates an overall result of Reachable.

This result shows that Connectivity Tests has validated network connectivity from the source GKE node to the destination GKE master. This includes analyzing the configuration of the resources inside the Google-owned project where the master runs. The trace does not provide details for the resources in the Google-owned project because you do not have permission to view them.

Cloud Console screenshot for GKE node to master trace.
Cloud Console screenshot for GKE node-to-master trace

GKE node to GKE master through public IP address

The following table shows example input values for a test to a GKE master through a public IP address. To view output for a successful test, continue to the next section.

Sample test inputs

Input parameter Input value
Protocol TCP
Source GKE node gke-cluster-1-node-1
Source project my-project
Destination IP address 104.1.1.1
Destination port 443

A successful test

This section describes an example of a successful test to a GKE master through a public IP address.

Console

The following screenshot from the Cloud Console shows a trace to the public IP address of a GKE master that indicates an overall result of Reachable.

This result shows that Connectivity Tests has validated network connectivity from the source GKE node to the network in which the GKE master runs, but not to the GKE master. When testing through a public IP address, Connectivity Tests does not analyze the configuration of the resources inside the Google-owned project where the master runs.

Cloud Console screenshot for GKE node-to-master trace through a public IP address.
Cloud Console screenshot for GKE node-to-master trace through a public IP address

Test failures for Google-managed services

When testing Google-managed services, the test can fail with an error message that the packet was dropped inside the service (for example DROPPED_INSIDE_GKE_SERVICE or DROPPED_INSIDE_CLOUD_SQL_SERVICE). This message might indicate a configuration issue in the Google-owned project that hosts the service in the following cases:

  • You tested connectivity between a GKE master and a GKE node in the same cluster (in either direction).
  • You tested connectivity from your VPC network to a Cloud SQL instance connected to your network, where both the source and destination are in the same region.

If you receive the previously mentioned error message for one of the cases listed earlier, contact Support. Otherwise, the error message might indicate invalid input. Confirm that you are testing to or from an endpoint that exists and that you expect to reach the managed resource in the Google-owned project. For example, if you are testing from a GKE node to a GKE master, confirm that the node exists and is expected to have connectivity to the master.

Sample test inputs: VM to a Cloud SQL instance in a different region

The following table shows example input values for a test from a VM to a Cloud SQL instance, where the VM is in a different region than the Cloud SQL instance. To view output for the failed test, continue to the next section.

Input parameter Input value
Protocol TCP
Source VM instance-1
This VM is in a different region than the Cloud SQL instance.
Source project my-project
Destination Cloud SQL instance vm-1
Destination port 5432

A failed test

This section describes an example of a failed test to a Cloud SQL instance.

Console

The following screenshot from the Cloud Console shows a trace to a destination Cloud SQL instance that indicates an overall result of Unreachable.

Cloud Console screenshot for failed VM to Cloud SQL trace.
Cloud Console screenshot for failed VM-to-Cloud SQL trace

Testing from a VPC network to a non-Google Cloud network

You can use the Connectivity Tests configuration analysis to test connectivity from your VPC network to a non-Google Cloud network over Cloud VPN or Cloud Interconnect. Typically, a non-Google Cloud network is your on-premises network or another cloud provider's network, also known as a peer network.

The configuration analysis evaluates the network path only up to the external IP address of the router or VPN gateway in a peer network.

The following example shows a trace from VM1 in a VPC network, over a Classic VPN tunnel using static routing, to VM2 in an on-premises network.

Packet trace through a Cloud VPN tunnel using static routes.
Packet trace through a Cloud VPN tunnel using static routes

If there is a matching static or dynamic route for the destination IP address in a peer network, the configuration analysis matches and verifies the route according to route precedence.

There is a default static route for all destinations with the next hop as the internet gateway. Connectivity Tests can match this default route unless you have removed or modified it.

If the default static route does not exist and there are no other valid routes to the destination, the trace returns a final state of Drop.

Trace path to a non-Google Cloud network using static routing.
Trace path to a non-Google Cloud network using static routing


Trace path to a non-Google Cloud network using dynamic routing.
Trace path to a non-Google Cloud network using dynamic routing

Testing from a non-Google Cloud network to a VPC network

The configuration analysis verifies that your VPC network can receive an inbound packet from your on-premises network after that packet could arrive at your VPC network. The analysis also verifies that the VPC network configuration is likely to allow delivery of this packet to the intended destination. The configuration analysis shows that Packet could be forwarded (in the API response, the final state is Forward). The destination is considered reachable.

When your VPC network peers with your on-premises network through Cloud Router, the VPC network receives one or more dynamic routes from your peered on-premises network. At the same time, your VPC network advertises its own routes to your peered on-premises network.

Because Connectivity Tests doesn't have access to your on-premises network configuration, it can't verify the configuration of correct routes and firewall rules on your on-premises router. Thus, traffic from your on-premises network to your VPC network is always considered valid by the Connectivity Tests configuration analysis.

However, Connectivity Tests can evaluate whether the VPC configuration would allow delivery of a packet to a destination in Google Cloud. To assess reachability, it evaluates the following Google Cloud resources:

  • The VPC network's ingress firewall rules.
  • The advertised route for IP addresses in your VPC network that Cloud Router advertises to your on-premises (peer) network.

Sample test inputs

The following table shows example input values for a test Google Cloud load balancer described in the previous section. Continue to the next section to view output for a successful test.

Input parameter Input value
Protocol TCP
Sample on-premises IP address 10.0.29.2
This is an IP address used in Google Cloud checkbox Clear this checkbox when specifying an on-premises IP source address.

A successful test from an on-premises network

This section describes an example of a successful test from an on-premises network to a VPC network.

Console

The following test result evaluates connectivity through Cloud VPN from the on-premises IP address to a VM instance. It also evaluates the Border Gateway Protocol (BGP) session, routes, and VPC firewall rules.

Example output for a successful test from on-premises to Google Cloud.
Example output for a successful test from on-premises to Google Cloud

Testing Google Cloud load balancers

Connectivity Tests configuration analysis supports tracing simulated packets to all types of Google Cloud load balancers.

The trace path for an external HTTP(S) load balancer also applies to a TCP proxy load balancer and an SSL proxy load balancer. For more information, see the Cloud Load Balancing overview.

In the following example, Connectivity Tests traces a simulated packet from an external host to a VIP for an external HTTP(S) load balancer. The TCP connection from the external host terminates at the proxy for the external HTTP(S) load balancer. The external HTTP(S) load balancer then initiates a new TCP connection to a VM acting as a load balancer backend.

A typical trace path to an external HTTP(S) load balancer with diagram legend.
A typical trace path to an external HTTP(S) load balancer

In the following trace path, the Connectivity Tests configuration analysis provides three traces, one for each possible path to the three load balancer backends. Connectivity Tests does this because it validates only configurations, not the live data plane.

Packet trace to an external HTTP(S) load balancer.
Packet trace to an external HTTP(S) load balancer(see diagram legend)

Sample test inputs

The following table shows example input values for a test to the external HTTP(S) load balancer described in the previous section. To view output for a successful test, continue to the next section.

Input parameter Input value
Destination protocol TCP
Destination port 80
Sample external source IP address (not shown) 62.35.1.1
External IP address of the load balancer 203.0.113.99
Destination project myproject

A successful test to a load balancer

This section describes an example of a successful test to the external HTTP(S) load balancer described previously.

In the actual data plane, the load balancing algorithm chooses a VM instance for each backend connection. Because there are three load balancer backends in this example, the Trace selection menu on the Results screen enables you to select the trace that you want to view.

Console

The following successful test result validates that all of the following Google Cloud resources for the external HTTP(S) load balancer are configured correctly.

  • The forwarding rule
  • The load balancer backends, including the ability for the load balancer to successfully send health checks to those backends
  • The proxy connection
  • VPC firewall rules

This result shows that a simulated packet from an external IP address could successfully reach the backend VM instances.

For a detailed example of a trace to all three backends, see Detecting invalid or inconsistent configurations.

Example output for a successful test to an external HTTP(S) load balancer.
Example output for a successful test to an external HTTP(S) load balancer

If you don't have permissions to review the Google Cloud resources in the network path for the external HTTP(S) load balancer, you still see results in the Cloud Console, including successful results. However, the card for each resource tested reads "No permission to view resource in PROJECT_NAME."

A test showing a missing firewall rule for a health check

A load balancer trace verifies many of the same Google Cloud resource configurations described previously. However, if the following load balancer resources are misconfigured, the analysis shows Packet could be dropped (a final state of the trace isDrop).

Console

The following test result shows that the VPC network's ingress firewall rules do not allow a health check to the load balancer backends, making the backends unavailable to the load balancer.

Example output for a missing firewall rule.
Example output for a missing firewall rule

In addition to invalid VPC firewall rules, the following problems are common configuration issues that Connectivity Tests detects for Google Cloud load balancers.

To correct these issues, use the solutions described in the following table.

Configuration issue Solution
The input parameters don't match the protocol or port that you defined in the forwarding rule for the load balancer. Before you run a test, change the input parameter to match the protocol or port that you defined in the forwarding rule.
The forwarding rule for the load balancer does not have backends configured. Before you run a test, configure the backends for the load balancer.
The load balancer has invalid or inconsistent configurations. Before you run a test, correct the invalid or inconsistent configurations.
Traffic can't reach an internal TCP/UDP load balancer that has a mismatched region because the internal TCP/UDP load balancer is a regional service. Before you run a test, configure load balancer components so that they are located in the same region.

What's next