Test states reference for Connectivity Tests

A Connectivity Test goes through a series of test states as it checks the configuration of each Google Cloud resource in a network path. Use this reference to interpret these states.

For more information about Connectivity Tests, see the overview.

Test states

A Connectivity Test provides data for the following test states in the order listed:

  • Initial state
  • Config checking state
  • Forwarding state
  • Transition state
  • Special state
  • Final state
  • Overall reachability result

Some of these states appear in every trace, while others only appear when testing the configuration of a specific Google Cloud resource or when performing a certain task.

The final state and the overall reachability result provide the most important test output.

In addition, test output can include metadata for Google Cloud resources that are associated with one or more of the states; for example, information about a virtual machine (VM) instance's name and IP address.

How Connectivity Tests verifies configurations

Connectivity Tests simulates a test packet through a network path by verifying configurations for Google Cloud resources in that path. Some examples of invalid configurations are a Cloud Load Balancing forwarding rule that has no backends, or a network route that doesn't exist.

For the config checking state, verified means that Connectivity Tests confirms that a configuration for the Google Cloud resource tested is valid and that the configuration allows the simulated test packet to continue through the network path being tested.

For ingress and egress firewall rules, verified means that Connectivity Tests confirms that the firewall rule is valid and that the simulated test packet can pass through the firewall.

If Connectivity Tests determines that a configuration is invalid, the packet has a final state of Drop.

Overall reachability result

Connectivity Tests provides an overall summary of reachability status, also known as a result. Results can have one of four values: Reachable, Unreachable, Ambiguous, and Undetermined.

Values table

The following table describes the value for each type of overall reachability result.

Overall reachability result Description
Reachable There are two possible scenarios. In both scenarios, Connectivity Tests does not find any configuration issues. Thus, both scenarios are considered Reachable.
  • In the first scenario, the packet originating from the source is expected to reach its destination. The final state of one or more traces is Deliver.
  • In the second scenario, the analysis is partially complete based on configurations where the user has permission. The final state of one or more traces is Forward.
Unreachable The packet originating from the source is expected to be dropped before reaching its destination. The final state of all the traces is Drop.
Ambiguous

This result is returned if the source and destination endpoints do not uniquely identify the test location in the network, and the overall reachability result contains multiple traces with mixed Reachable and Unreachable states.

In this case, the final states among multiple traces return different final states. The Ambiguous result doesn't apply to tests that contain only one trace.

Undetermined

Reachability could not be determined. The final state for one trace is Forward or Abort. For multiple traces, the final state is a combination of either Forward or Abort.

The reachability from the source to the destination cannot be determined for one of the following reasons:

  • The analysis is aborted due to a permission error. The user does not have read permission to the projects listed in the test.
  • The analysis is aborted due to internal errors.

Multiple traces

Each Connectivity Test can contain multiple traces, and the final state of these traces might not be the same. For example, a packet to the VIP for a Google Cloud load balancer might have n traces if there are n backend VM instances configured for the load balancer. These n traces might not have the same final states.

Because a Connectivity Test can produce multiple possible traces, the following is true:

  • If there is only one trace result, the overall reachability result is the same as the final state of the trace.
  • If there are multiple trace results, the overall reachability result is calculated based on the distribution of the final states contained in all of the traces.

Result metadata

In addition to the overall reachability result for traces, every test result contains the following metadata:

  • The time the test state for the test was verified
  • Error details of a test failure or cancellation
  • Trace details for each trace

Error details of a test failure or cancellation are shown as codes and messages displayed in the overall reachability result. For example, a test with a final state of Abort might show an error message such as Failed to pull initial config. An internal error occurred.

Final state

There are four final states: Drop, Abort, Forward, and Deliver. Each of the following sections has a table that contains messages and descriptions for each state.

Drop

Connectivity Tests dropped the simulated test packet because the test target was unreachable for the following reasons.

Message Description
UNKNOWN_EXTERNAL_ADDRESS The destination external address could not be resolved to a known target.
FOREIGN_IP_DISALLOWED The VM instance can only send or receive a packet with a foreign IP address if ip_forward is enabled. In other words, the foreign IP address failed a spoof check.
FIREWALL_RULE

Dropped due to a firewall rule unless allowed due to connection tracking.

Connectivity Tests might deny a test packet through a Virtual Private Cloud (VPC) firewall because the packet matches a blocking firewall rule. However, the actual data plane might allow the packet through due to connection tracking on the firewall. Connection tracking allows packets for an existing connection to return through the firewall.

NO_ROUTE Dropped due to no routes.
ROUTE_BLACKHOLE Dropped because the next hop of the matched route doesn't exist.
ROUTE_WRONG_NETWORK The packet was sent to the wrong (unintended) network, as shown in Invalid or inconsistent configurations encountered.
PRIVATE_TRAFFIC_TO_INTERNET A packet with an internal destination address was sent to an internet gateway.
PRIVATE_GOOGLE_ACCESS_DISALLOWED A VM instance with only an internal IP address tried to access a Google API or Google service, but Private Google Access was not enabled.
NO_EXTERNAL_ADDRESS A VM instance with only an internal IP address tried to access external hosts through a route whose next hop is the default internet gateway. Expected when Cloud NAT is not enabled in the subnet or when there's no other default route that uses a different type of next hop (such as a proxy VM).
UNKNOWN_INTERNAL_ADDRESS A destination internal address could not be resolved to a known target.
FORWARDING_RULE_MISMATCH A forwarding rule's protocol and ports did not match the packet header, or the packet does not originate from or is not directed to the same region as the regional load balancer.
FORWARDING_RULE_NO_INSTANCES A forwarding rule did not have backends configured.
FIREWALL_BLOCKING_LOAD_BALANCER_BACKEND_HEALTH_CHECK

Firewalls blocked the health check probes to the backends and caused the backends to be unavailable for traffic from the load balancer.

As part of its testing sequence for Cloud Load Balancing, Connectivity Tests verifies that existing VPC firewall rules have been configured to allow a health check probing packet to be sent to the Cloud Load Balancing backends. This configuration check results in a healthCheckFirewallState. For details, see Health check firewall rules.

INSTANCE_NOT_RUNNING A packet was sent from or to a VM instance that was not in a running state.
TRAFFIC_TYPE_BLOCKED The type of traffic was blocked and the user could not configure a firewall rule to enable it. For details, see Always blocked traffic.
GKE_MASTER_UNAUTHORIZED_ACCESS Access to the Google Kubernetes Engine master's endpoint was not authorized. For details, see Access to the cluster endpoints.
DROPPED_INSIDE_GKE_SERVICE (alpha) Packet was dropped inside Google Kubernetes Engine service.
CLOUD_SQL_INSTANCE_UNAUTHORIZED_ACCESS (alpha) Access to the Cloud SQL instance endpoint is not authorized. For details, see Authorizing with authorized networks.
DROPPED_INSIDE_CLOUD_SQL_SERVICE (alpha) Packet was dropped inside the Cloud SQL service.

Abort

Connectivity Tests aborted the test due to a lack of basic information, such as lack of access to the network configuration.

This state usually happens when Connectivity Tests does not have the correct permissions to obtain the configuration from the host project for a service project, as shown in the following table.

Message Description
UNKNOWN_NETWORK

Aborted due to an unknown network. The analysis cannot proceed because, in a Shared VPC network, the user running the test does not have access to the host project's network configurations, including firewall rules and routes.

Running a Connectivity Test requires that the user running the test can read configurations for resources such as firewall rules and routes in the host project. This happens because network resources are allocated in the host project, but the actual resources exist in the service project.

UNKNOWN_IP

The analysis aborted because the IP addresses required for analysis were unknown. This is due to incorrect user input or, based on the provided input parameters, Connectivity Tests could not determine a valid endpoint.

In a Shared VPC network, the user running the test did not have access to the host project's network configurations. This access is required for testing against IP addresses in the service project.

UNKNOWN_PROJECT The analysis aborted because no project information could be derived from the input to the Connectivity Test. This is due to incorrect user input or, based on the provided input parameters, Connectivity Tests could not determine a valid project.
PERMISSION_DENIED The analysis was aborted because the user lacked the permission to access all or part of the network configurations required to run the test.
NO_SOURCE_LOCATION The analysis aborted because no valid source endpoint could be derived from the test input. This is due to incorrect user input or, based on the provided input parameters, Connectivity Tests could not determine a valid source endpoint.
INVALID_ARGUMENT The analysis aborted because the source and/or destination endpoint specified in the test input were invalid. The possible reasons for this message include the following:
  • A malformed IP address
  • A non-existent VM instance or network URI
  • An IP address that is not in the range of the specified network URI
  • A VM instance that doesn't own the network interface in the specified network
NO_EXTERNAL_IP The analysis aborted because traffic was sent from a public IP address to a VM instance that did not have an external IP address.
UNINTENDED_DESTINATION The analysis aborted because none of the traces were able to match the destination information specified in the test input.
TRACE_TOO_LONG The analysis aborted because the number of steps in the trace exceeded a certain limit. This issue might be caused by a routing loop.

Forward

A Connectivity Test stopped at a specific endpoint but could go no further:

  • The analysis is partially complete based on configurations where the user has permission.
  • The test packet was forwarded to a network with an unknown configuration.
  • The test target has not been dropped according to the known configuration, and the test packet has been forwarded to a network where Connectivity Tests has no visibility.
Message Forwarded
PEERING_VPC To a peer VPC network
VPN_GATEWAY To a Cloud VPN gateway
INTERCONNECT To a Cloud Interconnect connection
GKE_MASTER To a GKE cluster master
IMPORTED_CUSTOM_ROUTE_NEXT_HOP To the next hop of a custom route imported from a peered VPC network
CLOUD_SQL_INSTANCE (alpha) To a Cloud SQL instance

Deliver

A Connectivity Test was able to reach the target and deliver the simulated test packet.

Because Connectivity Tests performs a static reachability analysis, a final state of Deliver does not guarantee that traffic can pass through the data plane. The purpose of the analysis is to validate configuration issues that might cause traffic to drop.

Message Target
INSTANCE A Compute Engine VM instance
INTERNET The internet
GOOGLE_API A Google API
GKE_MASTER (alpha) A GKE cluster master
CLOUD_SQL_INSTANCE (alpha) A Cloud SQL instance

Metadata

Connectivity Tests shows the following metadata for the final state.

Metadata name Description
AbortInfo Cause of an Abort final state and the resource URI that caused that state.
DropInfo Cause of a Drop final state and the resource URI that caused that state.
ForwardInfo The target type and target resource URI that a test packet was finally forwarded to (Forward final state).

Other states

Initial state

During the initial state, Connectivity Tests simulates starting from a network endpoint.

Message Description
START_FROM_INSTANCE The packet originated from a Compute Engine instance. InstanceInfo metadata was populated by Connectivity Tests.
START_FROM_INTERNET The packet originated from the internet. EndpointInfo metadata was populated by Connectivity Tests.
START_FROM_PRIVATE_NETWORK The packet originated from a VPC network or an on-premises network with an internal source IP address. If the source was a VPC network visible to the user, the NetworkInfo metadata was populated with network details by Connectivity Tests.

Config checking state

During the config checking state, Connectivity Tests checks the configuration of Google Cloud resources in the simulated network path, verifies that the resource configuration is valid, and verifies that the configuration allows the simulated test packet to continue on through the network path.

If needed, Connectivity Tests performs a spoof check.

Message Description
APPLY_INGRESS_FIREWALL_RULE Verified ingress firewall rule.
APPLY_EGRESS_FIREWALL_RULE Verified egress firewall rule.
APPLY_ROUTE Verified route.
APPLY_FORWARDING_RULE Matched forwarding rule.
SPOOFING_APPROVED The packet was sent or received under a foreign IP address, but allowed. For details, see spoof checking.

Forwarding state

During the forwarding state, Connectivity Tests simulates a packet arriving at an intermediate Google Cloud resource in the testing path (for example, a packet arriving at a Cloud VPN gateway or a Google Cloud load balancer).

Message Description
ARRIVE_AT_INSTANCE Arrived at a Compute Engine VM instance.
ARRIVE_AT_INTERNAL_LOAD_BALANCER Arrived at a Google Cloud load balancer that uses a private IP address as a VIP.
ARRIVE_AT_EXTERNAL_LOAD_BALANCER Arrived at the public IP address of a Google Cloud load balancer.
ARRIVE_AT_VPN_GATEWAY Arrived at a Cloud VPN gateway.
ARRIVE_AT_VPN_TUNNEL Arrived at a Cloud VPN tunnel.

Transition state

During the transition state, Connectivity Tests verifies simulated configurations where a packet is changed (for example, where Cloud NAT translates a packet header, or when a Google Cloud load balancing proxy terminates and reinitiates an inbound TCP session to VM instances).

Message Description
NAT The packet header was translated.
PROXY_CONNECTION The original connection was terminated, and a new proxied connection was initiated.

Special state

In this state, a test viewer does not have permission to view one or more Google Cloud resources. For more information, see Test permissions.

Metadata name Description
VIEWER_PERMISSION_MISSING The viewer of the test result does not have permission to see the configuration for the Google Cloud resource in this step.

Resource metadata

Connectivity Tests shows the following metadata for the Google Cloud resource configurations that it checks.

Metadata name Description
EndpointInfo Endpoints used for the test. Connectivity Tests obtains EndpointInfo from source and destination endpoints and validates the information by using the model for the data plane.
FirewallInfo Metadata associated with a VPC firewall rule.
ForwardingRuleInfo Metadata associated with a VPC forwarding rule.
InstanceInfo Metadata associated with a Compute Engine VM instance.
LoadBalancerInfo Metadata associated with a Google Cloud load balancer.
NetworkInfo Metadata associated with a VPC network.
RouteInfo Metadata associated with a VPC network route.

What's next