Manage destination overlap

This page describes how producer network administrators can manage destination overlap in a Virtual Private Cloud (VPC) network that uses a Private Service Connect interface.

Google Cloud ensures that the IP address ranges of subnets that are assigned to network interfaces on the same virtual machine (VM) instance can't have overlapping IP address ranges. However, subnets in the consumer and producer VPC networks can overlap, as shown in figure 1. When using a Private Service Connect interface with overlapping destination IP address ranges, additional configuration is necessary to ensure that traffic reaches the correct destination in the intended network.

The commands described on this page show how to temporarily update routing for a VM that uses the Debian operating system. To permanently update the VM, or to configure a VM that has a different operating system, consult the operating system's public documentation.

Figure 1. Subnet-a in a producer VPC network overlaps with subnet-c in a consumer VPC network because both subnets use the same IP address range. A producer VM needs to be able to reach 10.0.1.5 in both networks.

You can manage overlapping destination IP address ranges in the following ways, which are described in detail on this page:

Use network namespaces to create separate routing tables for each application.
Apply policy-based routing to your interface's VM to direct traffic based on target ports.

The following approaches can also be used to manage destination overlap, but they are not described on this page:

Use a socket library and bind() to control routing.
Use a completely non-overlapping IP address space for the producer network.
If the overlapping IP addresses on the producer side are only for first-party API endpoints, you can Configure Private Google Access for on-premises hosts.
Use eBPF to customize advanced routing rules based on criteria other than IP address. This approach is recommended for cases where the previous options are not feasible.

Manage destination address overlap by using network namespaces

You can manage destination address overlap by using network namespaces. This approach works well when some applications on a producer VM only need to access consumer workloads, and other applications on the producer VM only need to access producer workloads.

To manage destination address overlap by using network namespaces, do the following:

Connect to the VM that has your Private Service Connect interface.
Run the following command:
```
ip address
```
In the list of network interfaces, find and note the guest OS name for your Private Service Connect interface—for example, ens5. This name is associated with your Private Service Connect interface's IP address.
Create a network namespace for consumer-bound traffic:
```
sudo ip netns add consumer-ns
```
Move the Private Service Connect interface to the consumer network namespace. Run the following commands individually:
```
sudo ip link set OS_INTERFACE_NAME netns consumer-ns
```
```
sudo ip netns exec consumer-ns ip link set OS_INTERFACE_NAME up
```
Replace OS_INTERFACE_NAME with the guest OS name for your Private Service Connect interface that you found in step 2—for example, ens5.
Restore the Private Service Connect interface's IP address:
```
sudo ip netns exec consumer-ns ip addr add INTERFACE_IP/32 dev OS_INTERFACE_NAME
```
Replace INTERFACE_IP with the IP address of your Private Service Connect interface.
Validate the changes to your Private Service Connect interface:
```
sudo ip netns exec consumer-ns ip a
```
Ensure that the guest OS name of your Private Service Connect interface is listed in the command's output. Ensure that the interface has the correct IP address.
Add a route to the gateway IP address:
```
sudo ip netns exec consumer-ns ip route add GATEWAY_IP dev OS_INTERFACE_NAME scope link
```
Replace GATEWAY_IP with the IP address of the default gateway for your Private Service Connect interface's subnet.

Add a default route for your Private Service Connect interface:

sudo ip netns exec consumer-ns ip route add default via GATEWAY_IP dev OS_INTERFACE_NAME

Validate the routing table for the consumer-ns namespace:
```
sudo ip netns exec consumer-ns ip route
```
Ensure that the routing table has an entry of the following form:
```
default via GATEWAY_IP dev OS_INTERFACE_NAME
```
To verify that your interface can reach VMs in each part of the overlapping IP address range, do the following:
1. Ensure that firewall rules are configured to allow ingress ICMP traffic to your target VMs.
2. Send an ICMP ping from your interface's VM to a consumer VM that's in the overlapping IP address range. Use your consumer namespace:
```
sudo ip netns exec consumer-ns ping CONSUMER_IP_ADDRESS
```
  Replace CONSUMER_IP_ADDRESS with the IP address of a consumer VM from the overlapping IP address range.
3. Send an ICMP ping from your interface's VM to a producer VM that's in the overlapping IP address range. Use the default namespace:
```
ping PRODUCER_IP_ADDRESS
```
  Replace PRODUCER_IP_ADDRESS with the IP address of a producer VM from the overlapping IP address range.

Manage destination address overlap with policy-based routing

You can manage destination address overlap by configuring policy-based routing on the operating system of your interface's VM. This approach works well when the same application needs to access workloads that are in both the consumer and producer VPC networks, but you must repeat the procedure for each different port that you want to reach in the overlapping IP range.

When you set up policy-based routing to manage destination overlap, you choose target ports to use for consumer applications. Traffic that's bound for one of these ports flows through your Private Service Connect interface to the consumer subnet, while other traffic flows through the default interface to the producer subnet.

Connect to your Private Service Connect interface's VM.
If the iproute2 command is not available, install it.
Ensure that you can write to the following file: /etc/iproute2/rt_tables
Create a route table. Add a default route for your Private Service Connect interface:
```
echo "200 pscnet" >> /etc/iproute2/rt_tables \
sudo ip route add default dev OS_INTERFACE_NAME table pscnet
```
Replace OS_INTERFACE_NAME with the guest OS name of your Private Service Connect interface—for example, ens5.
Add a route to the overlapping consumer subnet range through your default gateway:
```
sudo ip route add CONSUMER_SUBNET_RANGE via GATEWAY_IP dev OS_INTERFACE_NAME table pscnet
```
Replace the following:
- CONSUMER_SUBNET_RANGE: the IP address range of your consumer subnet.
- GATEWAY_IP: the IP address of the default gateway for your Private Service Connect interface's subnet.
Update the route table so that egress packets from this VM use your interface's IP address as their source IP address:
```
sudo ip route add GATEWAY_IP src INTERFACE_IP dev OS_INTERFACE_NAME table pscnet
```
Replace INTERFACE_IP with the IP address of your Private Service Connect interface.
Add an IP rule that applies to all traffic that's destined for the target port of your consumer application:
```
sudo ip rule add dport CONSUMER_PORT table pscnet
```
Replace CONSUMER_PORT with the port that you have configured for traffic to your consumer VM.
To verify that a packet is routed to the correct VM based on its destination port, do the following:
1. Create test VMs in the producer and consumer networks that both use the same IP address from the overlapping range.
2. Set up an HTTP server on each test VM. Configure the consumer test VM to listen on the port that you've set up for the consumer application. Configure the producer test VM to listen on a port that's different from the one you've set up for the consumer application.
3. Ensure that firewall rules are configured to allow HTTP traffic to your test VMs.
4. Using the port that you've configured for your consumer application, make a GET request to the test IP address, and then verify that you've reached the correct instance:
```
curl TEST_IP_ADDRESS:CONSUMER_PORT
```
  Replace the following:
  - TEST_IP_ADDRESS: the IP address of your test VMs.
  - CONSUMER_PORT: the port of your consumer application.
5. Using the port that you've configured for your production test VM, make a GET request to the test IP address, and then verify that you've reached the correct instance:
```
curl IP_ADDRESS:PRODUCER_PORT
```
  Replace the following:
  - IP_ADDRESS: the IP address of your test VMs.
  - PRODUCER_PORT: the port that you've configured for your production test VM.