This page describes how to to stretch a Layer 2 network from your on-premises environment to your private cloud by using a Layer 2 VPN (L2VPN).
L2VPN based stretching of Layer 2 networks can work with or without NSX-based networks in your on-premises VMware environment. If you don't have NSX-based networks for workloads on-premises, use a standalone NSX Edge Services Gateway.
Deployment scenario
To stretch your on-premises network using L2VPN, you must configure an L2VPN server (destination NSX-T Tier0 router) and an L2VPN client (source standalone client).
In this deployment scenario, you connect your private cloud to your on-premises environment by using a site-to-site VPN tunnel. The VPN tunnel allows on-premises management and vMotion subnets to communicate with the private cloud management and vMotion subnets. This arrangement enables Cross vCenter vMotion (xVC-vMotion). You deploy an NSX-T Tier0 router as an L2VPN server in the private cloud.
You deploy an NSX standalone Edge in your on-premises environment as an L2VPN client and then pair it with the L2VPN server. Create a GRE tunnel endpoint on each side and configure it to stretch the on-premises Layer 2 network to your private cloud.
For more information about migration using L2VPN, see the VMware documentation on Virtual private networks.
Prerequisites
Verify that the following prerequisites are in place before deploying and configuring the solution:
- A site-to-site VPN connects your on-premises environment and your private cloud data center.
- The on-premises vSphere version is 6.7U1+ or 6.5P03+.
- The on-premises vSphere license is at the Enterprise Plus level (for vSphere Distributed Switch).
- You have identified the workload Layer 2 network to be stretched to your private cloud.
- You have identified a Layer 2 network in your on-premises environment for deploying your L2VPN client appliance.
- You have created a private cloud.
- The version of the standalone NSX-T Edge appliance is compatible with the NSX-T Manager version (NSX-T 2.3.0) used in your private cloud environment.
- A trunk port group is already created in the on-premises vCenter.
- A public IP address is reserved to use for the NSX-T standalone client uplink IP address, and a 1:1 NAT is in place for translation between the two addresses.
- DNS forwarding is set on the on-premises DNS servers so that the domain points to the private cloud DNS servers.
- RTT latency is less than or equal to 150 ms, which is required for vMotion to work across the two sites.
Limitations and considerations
The following table lists supported vSphere versions and network adaptor types.
| vSphere version | Source vSwitch type | Virtual NIC driver | Target vSwitch type | Supported? |
|---|---|---|---|---|
| All | DVS | All | DVS | Yes |
| vSphere 6.7UI or higher, 6.5P03 or higher | DVS | VMXNET3 | N-VDS | Yes |
| vSphere 6.7UI or higher, 6.5P03 or higher | DVS | E1000 | N-VDS | Not supported per VMware |
| vSphere 6.7UI or 6.5P03, NSX-V or versions below NSX-T2.2, 6.5P03 or higher | All | All | N-VDS | Not supported per VMware |
As of the VMware NSX-T 2.3 release:
- The logical switch on the private cloud side that is stretched to on-premises over L2VPN can't be routed at the same time. The stretched logical switch can't be connected to a logical router.
- L2VPN and route-based IPSEC VPNs can only be configured using API calls.
For more information, see the VMware documentation on Virtual private networks.
Sample L2VPN deployment
The following tables provide example specifications for L2VPN deployment.
On-premises network where the standalone ESG (L2VPN client) is deployed
| Item | Value |
|---|---|
| Network name | MGMT_NET_VLAN469 |
| VLAN | 469 |
| CIDR | 10.250.0.0/24 |
| Standalone Edge appliance IP address | 10.250.0.111 |
| Standalone Edge appliance NAT IP address | 192.227.85.167 |
On-premises network to be stretched
| Item | Value |
|---|---|
| VLAN | 472 |
| CIDR | 10.250.3.0/24 |
Private cloud IP schema for NSX-T Tier0 Router (L2VPN server)
| Item | Value |
|---|---|
| Loopback interface | 192.168.254.254/32 |
| Tunnel interface | 5.5.5.1/29 |
| Logical switch (stretched) | Stretch_LS |
| Loopback interface (NAT IP address) | 104.40.21.81 |
Private cloud network to be mapped to the stretched network
| Item | Value |
|---|---|
| VLAN | 712 |
| CIDR | 10.200.15.0/24 |
Get the logical router ID needed for L2VPN
The following steps show how to fetch the logical-router ID of a Tier0 DR logical router instance for the IPsec and L2VPN services. Use the logical-router ID later, when implementing the L2VPN.
- Log in to NSX-T Manager and select Networking > Routers > Provider-LR > Overview. For High Availability Mode, select Active-Standby. This action opens a pop-up window that shows the Edge VM on which the Tier0 router is currently active.
- Select Fabric > Nodes > Edges. Make a note of the management IP address of the active Edge VM (Edge VM1) identified in the previous step.
- Open an SSH session to the management IP address of the Edge VM. Run the
get-logical-routercommand. - If you don't see an entry
DR-Provider-LR, complete the following steps:- Create two overlay-backed logical switches. One logical switch is stretched to on-premises where the migrated workloads reside. For more information, see Creating a logical switch.
- Attach the second logical switch to the Tier1 router with a link local IP address or any non-overlapping subnet from on-premises or your private cloud. For more information, see Adding a downlink port on a Tier-1 logical router.
- Run the
get logical-routercommand again on the SSH session of the Edge VM. The output shows that the UUID of theDR-Provider-LRlogical router is displayed. Make a note of the UUID, which is required when you configure the L2VPN.
Get the logical-switch ID needed for L2VPN
- Log in to NSX-T Manager.
- Select Networking > Switching > Switches > Overview.
- Make a note of the UUID of the stretch logical switch, which is required when you configure the L2VPN.
Routing and security considerations for L2VPN
To establish an IPsec route-based VPN between the NSX-T Tier0 router and the NSX standalone Edge client, the loopback interface of the NSX-T Tier0 router must be able to communicate with the public IP address of the NSX standalone client on-premises over UDP 500/4500.
Allow UDP 500/4500 for IPsec
- Allocate a public IP address for the NSX-T Tier0 loopback interface in the Google Cloud VMware Engine portal.
- Create a firewall table with stateful rules that allow UDP 500/ 4500 inbound traffic, and attach the firewall table to the NSX-T HostTransport subnet.
Advertise the loopback interface IP to the underlay network
- Create a null route for the loopback interface network.
- Log in to NSX-T Manager and select Networking > Routing > Routers > Provider-LR > Routing > Static routes.
- Click Add.
- For Network, enter the loopback interface IP address.
- For Next Hops, click Add and specify Null for the next hop. Keep the default of 1 for Admin distance.
- Create an IP prefix list.
- Log in to NSX-T Manager and select Networking > Routing > Routers > Provider-LR > Routing > IP prefix lists.
- Click Add.
- Enter a name to identify the list.
- For Prefixes, click Add twice.
- In the first line, enter
0.0.0.0/0for Network and Deny for Action. - In the second line, select Any for Network and Permit for Action.
- Attach the IP prefix list to both BGP neighbors (TOR). Attaching the IP prefix list to the BGP neighbor prevents the default route from being advertised in BGP to the TOR switches. However, any other route that includes the null route advertises the loopback interface IP address to the TOR switches.
- Log in to NSX-T Manager and select
Networking > Routing >
Routers > Provider-LR >
Routing > BGP > Neighbors.
Select the first neighbor.
- Click Edit > Address families.
- For the IPv4 family, edit the Out filter column and select the IP prefix list that you created. Click Save. Repeat this step for the second neighbor.
- Redistribute the null static route into BGP. To advertise the loopback
interface route to the underlay, you must redistribute the null static route
into BGP.
- Log in to NSX-T Manager and select Networking > Routing > Routers > Provider-LR > Routing > Route redistribution > Neighbors.
- Select Provider-LR-route_redistribution and click Edit.
- Select the Static checkbox and click Save
Configure a route-based VPN on the NSX-T Tier0 router
Use the following template to provide all the details for configuring a route based VPN on the NSX-T Tier0 router. The UUIDs in each POST call are required in subsequent POST calls.
The IP addresses for the loopback and tunnel interfaces for L2VPN must be unique
and must not overlap with the on-premises or private cloud networks. The
loopback interface network must always be /32.
Loopback interface ip : 192.168.254.254/32 Tunnel interface subnet : 5.5.5.0/29 Logical-router ID : UUID of Tier0 DR logical router obtained in section "Steps to fetch Logical-Router ID needed for L2VPN" Logical-switch ID(Stretch) : UUID of Stretch Logical Switch obtained earlier IPSec Service ID : IKE profile ID : DPD profile ID : Tunnel Profile ID : Local-endpoint ID : Peer end-point ID : IPSec VPN session ID (route-based) : L2VPN service ID : L2VPN session ID : Logical-Port ID : Peer Code :
For all of the following API calls, replace the IP address with your NSX-T
Manager IP address. You can run all these API calls by using curl commands.
Enable the IPSec VPN service on the logical router
POST https://192.168.110.201/api/v1/vpn/ipsec/services/
{
"resource_type": "IPSecVPNService",
"description": "Manage VPN service",
"display_name": "IPSec VPN service",
"logical_router_id": "Logical-router ID",
"ike_log_level": "INFO",
"enabled": true
}
Create profiles: IKE
POST https://192.168.110.201/api/v1/vpn/ipsec/ike-profiles
{
"resource_type": "IPSecVPNIKEProfile",
"description": "IKEProfile for siteA",
"display_name": "IKEProfile siteA",
"encryption_algorithms": ["AES_128"],
"ike_version": "IKE_V2",
"digest_algorithms": ["SHA2_256"],
"sa_life_time":21600,
"dh_groups": ["GROUP14"]
}
Create profiles: DPD
POST https://192.168.110.201/api/v1/vpn/ipsec/dpd-profiles
{
"resource_type": "IPSecVPNDPDProfile",
"display_name": "nsx-default-dpd-profile",
"enabled": true
}
Create profiles: Tunnel
POST https://192.168.110.201/api/v1/vpn/ipsec/tunnel-profiles
{
"resource_type": "IPSecVPNTunnelProfile",
"display_name": "nsx-default-tunnel-profile",
"enable_perfect_forward_secrecy": true,
"encryption_algorithms": ["AES_GCM_128"],
"digest_algorithms": [],
"sa_life_time":3600,
"dh_groups": ["GROUP14"],
"encapsulation_mode": "TUNNEL_MODE",
"transform_protocol": "ESP",
"df_policy": "COPY"
}
Create a local endpoint
{
"resource_type": "IPSecVPNLocalEndpoint",
"description": "Local endpoint",
"display_name": "Local endpoint",
"local_id": "Public IP of Loopback interface",
"ipsec_vpn_service_id": {
"target_id": "IPSec VPN service ID"},
"local_address": "IP of Loopback interface",
"trust_ca_ids": [],
"trust_crl_ids": []
}
Create a peer endpoint
POST https://192.168.110.201/api/v1/vpn/ipsec/peer-endpoints
{
"resource_type": "IPSecVPNPeerEndpoint",
"description": "Peer endpoint for site B",
"display_name": "Peer endpoint for site B",
"connection_initiation_mode": "INITIATOR",
"authentication_mode": "PSK",
"ipsec_tunnel_profile_id": "IPSec Tunnel profile ID",
"dpd_profile_id": "DPD profile ID",
"psk":"nsx",
"ike_profile_id": "IKE profile ID",
"peer_address": "Public IP of Standalone client",
"peer_id": "Public IP of Standalone client"
}
Create a route-based VPN session
POST : https://192.168.110.201/api/v1/vpn/ipsec/sessions
{
"resource_type": "RouteBasedIPSecVPNSession",
"peer_endpoint_id": "Peer Endpoint ID",
"ipsec_vpn_service_id": "IPSec VPN service ID",
"local_endpoint_id": "Local Endpoint ID",
"enabled": true,
"tunnel_ports": [
{
"ip_subnets": [
{
"ip_addresses": [
"5.5.5.1"
],
"prefix_length": 24
}
]
}
]
}
Configure L2VPN on NSX-T Tier0 router
Provide the following information after every POST call. The IDs are required in subsequent POST calls.
L2VPN Service ID: L2VPN Session ID: Logical Port ID:
Create the L2VPN service
The output of the following GET command will be empty because the configuration is not yet complete.
GET : https://192.168.110.201/api/v1/vpn/l2vpn/services
For the following POST command, the logical router ID is the UUID of the Tier0 DR logical router obtained earlier.
POST : https://192.168.110.201/api/v1/vpn/l2vpn/services
{
"logical_router_id": "Logical Router ID",
"enable_full_mesh" : true
}
Create the L2VPN session
For the following POST command, the L2VPN service ID is the ID that you just obtained, and the IPsec VPN session ID is the ID obtained in the previous section.
POST: https://192.168.110.201/api/v1/vpn/l2vpn/sessions
{
"l2vpn_service_id" : "L2VPN service ID",
"transport_tunnels" : [
{
"target_id" : "IPSec VPN session ID"
}
]
}
These calls create a GRE tunnel endpoint. To check the status, run the following command.
edge-2> get tunnel-port Tunnel : 44648dae-8566-5bc9-a065-b1c4e5c3e03f IFUID : 328 LOCAL : 169.254.64.1 REMOTE : 169.254.64.2 ENCAP : GRE Tunnel : cf950ca1-5cf8-5438-9b1a-d2c8c8e7229b IFUID : 318 LOCAL : 192.168.140.155 REMOTE : 192.168.140.152 ENCAP : GENEVE Tunnel : 63639321-87c5-529e-8a61-92c1939799b2 IFUID : 304 LOCAL : 192.168.140.155 REMOTE : 192.168.140.156 ENCAP : GENEVE
Create a logical port with the tunnel ID specified
POST https://192.168.110.201/api/v1/logical-ports/
{
"resource_type": "LogicalPort",
"display_name": "Extend logicalSwitch, port for service",
"logical_switch_id": "Logical switch ID",
"admin_state" : "UP",
"attachment": {
"attachment_type":"L2VPN_SESSION",
"id":"L2VPN session ID",
"context" : {
"resource_type" : "L2VpnAttachmentContext",
"tunnel_id" : 10
}
}
}
Obtain the peer code for L2VPN on the NSX-T side
Obtain the peer code of the NSX-T endpoint. The peer code is required when configuring the remote endpoint. The L2VPN can be obtained from the previous section. For more information, see the NSX-T 2.3 API.
GET https://192.168.110.201/api/v1/vpn/l2vpn/sessions/session-id/peer-codes
Deploy the NSX-T standalone client (on-premises)
Before deploying, verify that your on-premises firewall rules allow inbound and outbound UDP 500/4500 traffic from or to the public IP address that was reserved earlier for the NSX-T T0 router loopback interface.
- Download the standalone edge client OVF and extract the files from the downloaded bundle into a folder.
- Go to the folder with all the extracted files.
- Select all the vmdks files and click Next.
NSX-l2t-client-large.mfandNSX-l2t-client-large.ovffor large appliance size.NSX-l2t-client-Xlarge.mfandNSX-l2t-client-Xlarge.ovffor extra large size appliance size.
- Enter a name for the NSX-T standalone client and click Next.
- Click Next as needed to reach the datastore settings.
- Select the appropriate datastore for the NSX-T standalone client and click Next.
- Select the correct port groups for the Trunk (Trunk PG), Public (Uplink PG), and HA (Uplink PG) interfaces for the NSX-T standalone client. Click Next.
- Provide the following information in the Customize template screen and
click Next.
- Expand L2T:
- Peer address. Enter the IP address reserved on the Google Cloud VMware Engine portal for the NSX-T Tier0 loopback interface.
- Peer code. Paste the peer code obtained from the L2VPN server deployment.
- Sub interfaces VLAN (tunnel ID). Enter the VLAN ID to be
stretched. In parentheses
(), enter the tunnel ID that was previously configured.
- Expand uplink interface:
- DNS IP address. Enter the on-premises DNS IP address.
- Default gateway. Enter the gateway of the VLAN to act as the default gateway for this client.
- IP address. Enter the uplink IP address of the standalone client.
- Prefix length. Enter the prefix length of the uplink VLAN/subnet.
- CLI admin/enable/root user password. Set the password for the admin/enable/root account.
- Expand L2T:
- Review the settings and click Finish.
Configure an on-premises sink port
If one of the VPN sites doesn't have NSX deployed, you can configure an L2VPN by deploying a standalone NSX edge at that site. For an edge services gateway appliance to function as an L2VPN client, deploy a standalone edge using an OVF file on a host that isn't managed by NSX.
If a standalone edge trunk vNIC is connected to a vSphere distributed switch, either promiscuous mode or a sink port is required for L2VPN function. Using promiscuous mode can cause duplicate pings and duplicate responses. For this reason, use sink port mode in the L2VPN standalone NSX edge configuration. For more information, see Configuring a sink port.
IPsec VPN and L2VPN verification
Use the following commands to verify IPsec and L2VPN sessions from NSX-T standalone edge.
nsx-l2t-edge> show service ipsec ----------------------------------------------------------------------- vShield Edge IPSec Service Status: IPSec Server is running. AESNI is enabled. Total Sites: 1, 1 UP, 0 Down Total Tunnels: 1, 1 UP, 0 Down ---------------------------------- Site: 10.250.0.111_0.0.0.0/0-104.40.21.81_0.0.0.0/0 Channel: PeerIp: 104.40.21.81 LocalIP: 10.250.0.111 Version: IKEv2 Status: UP Tunnel: PeerSubnet: 0.0.0.0/0 LocalSubnet: 0.0.0.0/0 Status: UP ---------------------------------- nsx-l2t-edge> show service l2vpn L2 VPN is running ---------------------------------------- L2 VPN type: Client/Spoke SITENAME IPSECSTATUS VTI GRE 1ecb00fb-a538-4740-b788-c9049e8cb6c6 UP vti-100 l2t-1
Use the following commands to verify IPsec and L2VPN sessions from the NSX-T Tier0 router.
edge-2> get ipsecvpn session
Total Number of Sessions: 1
IKE Session ID : 3
UUID : 1ecb00fb-a538-4740-b788-c9049e8cb6c6
Type : Route
Local IP : 192.168.254.254 Peer IP : 192.227.85.167
Local ID : 104.40.21.81 Peer ID : 192.227.85.167
Session Status : Up
Policy Rules
VTI UUID : 4bf96e3b-e50b-49cc-a16e-43a6390e3d53
ToRule ID : 560874406 FromRule ID : 2708358054
Local Subnet : 0.0.0.0/0 Peer Subnet : 0.0.0.0/0
Tunnel Status : Up
edge-2> get l2vpn session
Session : f7147137-5dd0-47fe-9e53-fdc2b687b160
Tunnel : b026095b-98c8-5932-96ff-dda922ffe316
IPSEC Session : 1ecb00fb-a538-4740-b788-c9049e8cb6c6
Status : UP
Use the following commands to verify the sink port on the ESXi host where the NSX-T standalone client VM resides in the on-premises environment.
[root@esxi02:~] esxcfg-vswitch -l |grep NSX
53 1 NSXT-client-large.eth2
225 1 NSXT-client-large.eth1
52 1 NSXT-client-large.eth0
[root@esxi02:~] net-dvs -l | grep "port\ [0-9]\|SINK\|com.vmware.common.alias"
com.vmware.common.alias = csmlab2DS , propType = CONFIG
port 24:
port 25:
port 26:
port 27:
port 13:
port 19:
port 169:
port 54:
port 110:
port 6:
port 107:
port 4:
port 199:
port 168:
port 201:
port 0:
port 49:
port 53:
port 225:
com.vmware.etherswitch.port.extraEthFRP = SINK
port 52:
What's next
- Set up VPN gateways on the Google Cloud VMware Engine network.
- Learn more about VMware on your private cloud.