Using NFS volume as vSphere datastore hosted by Google Cloud NetApp Volumes

You can use Google Cloud NetApp Volumes NFS storage for Standard, Premium, and Extreme service levels as an external datastore for ESXi hosts in Google Cloud VMware Engine. To do so, you must create NetApp Volumes in select regions and then mount them as external datastores to their existing ESXi hosts in VMware Engine.

This solution has the following functionality for datastores:

  • Availability in all regions where both VMware Engine and NetApp Volumes are available
  • Support for NFSv3 protocol
  • Data protection through crash-consistent snapshots and cross-region replication

The following limitations apply:

  • Copy offload (VAAI) and NFSv4.1 are not supported.
  • SnapMirror functionality to and from on-premises for disaster recovery is not supported.

Networking

NetApp Volumes and VMware Engine services are connected through private service access (PSA). Network charges resulting from storage access within a region don't apply. It is not recommended to access a datastore volume from a different region. You can create NetApp Volumes in Google Cloud console and enable deletion protection before mounting the volumes as datastores to their ESXi hosts.

Here's a diagram that shows NetApp Volumes being used with VMware Engine and Compute Engine:

Architecture diagram of Cloud Volumes Service in relation to VMware Engine and Compute Engine

Before you begin

The steps in this document assume that you have done the following:

  • Earmarked a /26 CIDR for the VMware Engine service network to be used for external NFS storage.

Service subnets

When you create a private cloud, VMware Engine creates additional service subnets (for example, service-1, service-2, service-3). Service subnets are targeted for appliance or service deployment scenarios, such as storage, backup and disaster recovery, or media streaming, providing high scale, linear throughput and packet processing for even the largest-scaled private clouds. VM communication across a service subnet travels from the VMware ESXi host directly into the Google Cloud networking infrastructure, empowering high-speed communication.

NSX-T gateway and distributed firewall rules don't apply to any service subnet.

Configuring service subnets

Service subnets don't have a CIDR allocation on initial creation. Instead, you must specify a non-overlapping CIDR range and prefix for service subnets using the VMware Engine console or API.

The first usable address becomes the gateway address. To allocate a CIDR range and prefix edit one of the service subnets.

Service subnets can be updated if CIDR requirements change. However, modification of an existing service subnet CIDR can cause network availability disruption for VMs attached to that service subnet.

You should add the reserved CIDR allocations for the service subnets you defined in the VMware Engine portal to the NetApp Volumes Export Rules in the Allowed Clients section while creating the volume.

Failing to do so returns the following error or similar in vmkernel.log:

2022-09-23T04:58:14.266Z cpu23:2103354 opID=be2a0887)NFS: 161: Command: (mount)
Server: (10.245.17.21) IP: (10.245.17.21) Path: (/vol-g-shared-vmware-002) Label:
(NFS) Options: (None)
...
2022-09-23T04:58:14.270Z cpu23:2103354 opID=be2a0887)NFS: 194: NFS mount
10.245.17.21:/vol-g-shared-vmware-002 failed: The mount request was denied by the
NFS server. Check that the export exists and that the client is permitted to
mount it.

Get VPC network details

When creating a peering connection between VMware Engine and NetApp Volumes, you need some details about the VPC network used by NetApp Volumes. To get these details, do the following:

  1. In the Google Cloud console, go to the VPC Network peerings page.

    Go to VPC Network peerings

  2. Select the peering connection created in NetApp Volumes for your project. The connection is named sn-netapp-prod.

    You might see multiple peering connections with the same name if you have more than one peered VPC network. The person who set up the VPC Network Peering connections can help you determine which connection to use for VMware Engine.

  3. Copy the Peered VPC network and Peered project ID fields, which begin with netapp and end with -tp, respectively.

Create a peering connection for legacy networks

To establish a connection between VMware Engine and NetApp Volumes service, make a one-time peering between tenant host projects. If your VMware Engine project and private clouds were created before Nov 12, 2023, you are using an earlier version of the VMware Engine network. For environments using an earlier version of VMware Engine network, do the following:

  1. In the Google Cloud console, go to the Private connections page.

    Go to Private connections

  2. Click Create.

  3. In Private connection name, provide a name for your peering, for example, peering-2-netapp-volumes.

  4. In VMware Engine network, specify the VMware Engine network you want to peer, for example, us-central1-default.

  5. For Private connection type, select Third party service.

  6. In Peered project ID, enter the peered project ID of the NetApp Volumes that contains your volume.

  7. In Peered VPC network, enter the name of the peered VPC network that volume is in.

  8. Click Create.

Expect the VPC peering status of your new private connection to stay in the Inactive state for up to 72 hours while VMware Engine services and validates the peering request.

Create a peering connection

If your VMware Engine project and private clouds were created after Nov 12, 2023, do the following. For more information on how to create VPC peerings for such environments, see Peer a VPC network.

  1. In the Google Cloud console, go to the VPC Network peerings page.

    Go to VPC Network peerings

  2. Click Create.

  3. In the Name field, provide a name for your networking peering. For example, peering-2-netapp-volumes.

  4. In the VMware Engine network section, keep the default "In current project" selected. Specify the VMware Engine network you want to peer, for example ven1.

  5. For Peering, select Google Cloud NetApp Volumes.

  6. In the Service tenant project ID field, enter the peered project ID of the Google Cloud project containing your volume.

  7. In the Service tenant VPC name field, enter the name of the peered VPC network your volume is in.

  8. In the Route exchange section, keep the default settings.

  9. Click Create.

Creating NetApp Volumes volumes

When creating a NetApp Volumes volume for use as a VMware Engine datastore, ensure that the volume export rule allows the following:

  • Access from the VMware Engine ESXi hosts (or all hosts) under Allowed Clients. This is usually the service subnet range you created earlier.
  • Read & write access
  • Root access

Marking the volumes as non-deletable

Before you can mount a NetApp Volumes volume as an external NFS datastore to ESXi hosts in VMware Engine, these volumes must be blocked from accidental deletion by users to avoid disruption to the vSphere environment.

You can also use a Google Cloud CLI command to set the restricted actions delete option on your volume.

VMware Engine mounts these volumes only if they are configured to be blocked from deletion.

After creating the volume, you can perform various volume management functions by using the NetApp Volumes UI/API/CLI. For more information, see Volumes overview.

Mounting the volume as a datastore

Contact VMware Engine Support for assistance in mounting the volume as a datastore. Provide the name of the private cloud, the volume mount path, and the service subnet.

Once the datastores are mounted and available, you can use vCenter UI to provision VMs against the external datastores, view metrics and view logs related to io operations performed against external datastores.

For more information about NetApp Volumes, see What is Google Cloud NetApp Volumes.