Adding Local SSDs

Compute Engine offers always-encrypted local solid-state drive (SSD) block storage for virtual machine instances. Each local SSD is 375 GB in size, but you can attach up to eight local SSD devices for 3 TB of total local SSD storage space per instance. Optionally, you can format and mount multiple local SSD devices into a single logical volume.

Unlike persistent disks, local SSDs are physically attached to the server that hosts your virtual machine instance. This tight coupling offers superior performance, very high input/output operations per second (IOPS), and very low latency compared to persistent disks. See Persistent Disk and Local SSD Performance for details.

Warning: The performance gains from Local SSDs require certain trade-offs in availability, durability, and flexibility. Because of these trade-offs, local SSD storage is not automatically replicated and all data on the local SSD may be lost if the instance terminates for any reason. See Local SSD data persistence for details.

Local SSDs are suitable only for temporary storage such as caches, processing space, or low value data. If you store important data in a Local SSD device, you must also store that same data in a durable storage option.

You cannot stop and restart an instance that has a local SSD. If you do shut down an instance with a Local SSD through the guest operating system, you will not be able to restart the instance and the data on the Local SSD will be lost.

If Local SSDs do not meet your redundancy or flexibility requirements, you can use local SSDs in combination with other instance storage options.

Before you begin

• If you want to use the command-line examples in this guide:
  1. Install or update to the latest version of the gcloud command-line tool.
  2. Set a default region and zone.
• If you want to use the API examples in this guide, set up API access.
• Read about the difference between persistent disks and local SSDs.

Understanding Local SSD data persistence

Before you create an instance with a local SSD, you must understand what events preserve your Local SSD data and what events can cause your Local SSD data to be unrecoverable.

Data on Local SSDs persists only through the following events:

• If you reboot the guest operating system.
• If you configure your instance for live migration and the instance goes through a host maintenance event.
• If the host system experiences a host error, Compute Engine attempts to preserve local SSD data, but it is only able to do so if the underlying drive recovers within 60 minutes.

Data on Local SSDs does not persist through the following events:

• If you terminate your instance manually.
• If you shut down the guest operating system and force the instance to stop.
• If you configure the instance to be preemptible and the instance goes through the preemption process.
• If you configure the instance to terminate on host maintenance events and the instance goes through a host maintenance event.
• If the host system experiences a host error, and the underlying drive does not recover within 60 minutes, Compute Engine does not attempt to preserve the data on your Local SSD.
• If you misconfigure the local SSD so that it becomes unreachable.
• If you disable project billing. The instance will stop and your data will be lost.

Create an instance with a local SSD

Because local SSDs are located on the physical machine where your virtual machine instance is running, they can be created only during the instance creation process. Local SSDs cannot be used as boot devices.

After creating a local SSD, you must format and mount the device before you can use it.

Selecting the NVMe or SCSI interfaces

You can connect local SSDs to your instances using either an NVMe interface or a SCSI interface. Most public images include NVMe and SCSI drivers. However, some images include optimized drivers that allow your instance to achieve the best performance over NVMe or are capable of using multi-queue SCSI to achieve the best performance over the SCSI interface.

• NVMe-optimized public images
  • Ubuntu 14.04 LTS image ubuntu-1404-trusty-v20170110 and newer, or image family ubuntu-1404-lts
  • Ubuntu 16.04 LTS imageubuntu-1604-xenial-v20170113 and newer, or image family ubuntu-1604-lts
  • Ubuntu 17.10 image family ubuntu-1710
  • Ubuntu 18.04 LTS image family ubuntu-1804-lts
• Multiqueue-SCSI capable public images:
  • Debian 9 Stretch images, or image family debian-9.
  • Ubuntu 14.04 LTS image ubuntu-1404-trusty-v20170807 and newer, or image family ubuntu-1404-lts
  • Ubuntu 16.04 LTS image ubuntu-1604-xenial-v20170803 and newer, or image family ubuntu-1604-lts
  • Ubuntu 17.10 image family ubuntu-1710
  • Ubuntu 18.04 LTS image family ubuntu-1804-lts
  • All Windows Server images
  • All SQL Server images

Optionally, you can enable multi-queue SCSI for custom images that you import to your project. Read enabling multi-queue SCSI to learn more.

Creating a local SSD

You can create an instance with a local SSD using the Google Cloud Platform Console, the gcloud tool, or the API.

Console

1. In the GCP Console, go to the VM Instances page.

   Go to the VM Instances page

2. Click Create instance.
3. On the Create a new instance page, fill in the properties for your instance.
4. Expand Management, security, disks, networking, sole tenancy.
5. Click Disks and under Additional disks , click Add new disk.
6. Under the Name field, enter a name for the new disk.
7. Under Type, choose Local SSD scratch disk (maximum 8).
8. Click Done.
9. Click Create to create the instance.
10. After creating a local SSD, you must format and mount the device before you can use it.

gcloud

To create a virtual machine instance with an attached local SSD, follow the instructions to create an instance, using the --local-ssd flag to create and attach local SSD devices. To create multiple local SSD devices, add additional --local-ssd flags. Optionally, you can also set values for the interface and the device name for each --local-ssd flag.

Similarly, you can create an instance with two local SSDs that each use a SCSI interface as follows:

gcloud compute instances create example-instance \
    --machine-type n1-standard-8 \
    --local-ssd interface=[INTERFACE_TYPE] \
    --local-ssd interface=[INTERFACE_TYPE] \
    --image-project [IMAGE_PROJECT] \
    --image-family [IMAGE_FAMILY]

where:

• [INTERFACE_TYPE] is the Local SSD interface that you want to use for this device. Specify nvme if your boot disk image has optimized NVMe drivers, or scsi for other images.
• [INSTANCE_NAME] is the name for the new instance.
• [IMAGE_FAMILY] is one of the available image families.
• [IMAGE_PROJECT] is the image project that the image family belongs to.

If necessary, you can attach local SSD devices a single instance using a combination of nvme and scsi for different devices. Performance for the nvme device depends on the boot disk image for your instance.

After creating a local SSD, you must format and mount the device before you can use it.

API

In the API, you can create a local SSD device when creating a virtual machine instance by using the initializeParams property. You must also provide the following properties:

• diskType: Set to local SSD
• autoDelete: Set to true
• type: Set to SCRATCH

The following properties cannot be set for local SSD instances:

• diskName
• sourceImage property
• diskSizeGb

Here is a sample request payload that creates an instance with a boot disk and a local SSD device:

{
   "machineType":"zones/us-central1-f/machineTypes/n1-standard-8",
   "name":"local-ssd-instance",
   "disks":[
      {
         "type":"PERSISTENT",
         "initializeParams":{
            "sourceImage":"projects/ubuntu-os-cloud/global/images/family/ubuntu-1604-lts"
         },
         "boot":true
      },
      {
         "type":"SCRATCH",
         "initializeParams":{
            "diskType":"zones/us-central1-f/diskTypes/local-ssd"
         },
         "autoDelete":true,
         "interface": "NVME"
      }
   ],
   "networkInterfaces":[
      {
         "network":"global/networks/default"
      }
   ]
 }

After creating a local SSD, you must format and mount the device before you can use it.

For more information on creating an instance in the API, see the API reference.

Format and mount a local SSD device

Format and mount individual local SSD devices

The easiest way to connect local SSDs to your instance is to format and mount each device with a single partition. Alternatively, you can combine multiple devices into a single logical volume.

Linux instances

Format and mount the new local SSD on your Linux instance. You can use any partition format and configuration that you need. For this example, create a single ext4 partition.

1. Go to the VM instances page.

   Go to the VM instances page

2. Click the SSH button next to the instance that has the new attached local SSD. The browser opens a terminal connection to the instance.

3. In the terminal, use the lsblk command to identify the local SSD that you want to mount. Local SSDs in SCSI mode have standard IDs like sdb. Local SSDs in NVMe mode have IDs like nvme0n1.

   $ lsblk
   

   NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
   sda       8:0    0   10G  0 disk
   └─sda1    8:1    0   10G  0 part /
   nvme0n1 259:0    0  375G  0 disk
   

4. Format the local SSD with an ext4 file system. This command deletes all existing data from the local SSD.

   $ sudo mkfs.ext4 -F /dev/[SSD_ID]
   

   where: [SSD_ID] is the ID of the Local SSD that you want to format.

5. Use the mkdir command to create a directory where you can mount the device.

   $ sudo mkdir -p /mnt/disks/[MNT_DIR]
   

   where: [MNT_DIR] is the directory where you want to mount your local SSD.

6. Mount the local SSD to the instance. Optionally, you can disable write cache flushing to improve write performance while risking reduced durability for up to two seconds of cached data writes.

   $ sudo mount /dev/[SSD_ID] /mnt/disks/[MNT_DIR]
   

   where:

   • [SSD_ID] is the ID of the Local SSD that you want to mount.
   • [MNT_DIR] is the directory where you want to mount your local SSD.

7. Configure read and write access to the device. For this example, grant write access to the device for all users.

   $ sudo chmod a+w /mnt/disks/[MNT_DIR]
   

   where: [MNT_DIR] is the directory where you mounted your local SSD.

Optionally, you can add the local SSD to the /etc/fstab file so that the device automatically mounts again when the instance restarts. This entry does not preserve data on your local SSD if the instance stops or terminates. See Understanding Local SSD data persistence for complete details on Local SSD data persistence.

When you specify the entry /etc/fstab file, be sure to include the nofail option so that the instance can continue to boot even if the Local SSD is not present. For example, if you take a snapshot of the boot disk and create a new instance without any local SSDs attached, the instance can continue through the startup process and not pause indefinitely. For instances running Ubuntu 12.04 or Ubuntu 14.04, specify the nobootwait option instead of nofail.

1. Create the /etc/fstab entry. Use the blkid command to find the UUID for the file system on the device and edit the /etc/fstab file to include that UUID with the mount options. You can complete this step with a single command. For example:

   $ echo UUID=`sudo blkid -s UUID -o value /dev/disk/by-id/google-local-ssd-0` /mnt/disks/[MNT_DIR] ext4 discard,defaults,[NOFAIL_OPTION] 0 2 | sudo tee -a /etc/fstab
   

   where:

   • [MNT_DIR] is the directory where you mounted your local SSD.
   • [NOFAIL_OPTION] is the option that specifies what the system should do if it is unable to mount this local SSD. For most distributions, specify nofail. For instances running Ubuntu 12.04 or Ubuntu 14.04, specify the nobootwait option.

2. Use the cat command to verify that your /etc/fstab entries are correct:

   $ cat /etc/fstab
   

If you create a snapshot from the boot disk of this instance and use it to create a separate instance that does not have local SSDs, edit the /etc/fstab file and remove the entry for this local SSD. Even with the nofail or nobootwait options in place, keep the /etc/fstab file in sync with the devices that are attached to your instance and remove these entries before you create your boot disk snapshot.

Windows instances

Use the Windows Disk Management tool to format and mount a local SSD on a Windows instance.

1. Connect to the instance through RDP. For this example, go to the VM instances page and click the RDP button next the instance that has the local SSDs attached. After you enter your username and password, a window opens with the desktop interface for your server.

2. Right click the Windows Start button and select Disk Management.

   

3. If you have not initialized the local SSD before, the tool prompts you to select a partitioning scheme for the new devices. Select GPT and click OK.

   

4. After the local SSD initializes, right click the unallocated disk space and select New Simple Volume.

   

5. Follow the instructions in the New Simple Volume Wizard to configure the new volume. You can use any partition format that you like, but for this example select NTFS. Also, check Perform a quick format to speed up the formatting process.

   

6. After you complete the wizard and the volume finishes formatting, check the new local SSD to ensure it has a Healthy status.

   

That's it! You can now write files to the local SSD.

Format and mount multiple local SSD devices into a single logical volume

Unlike persistent SSDs, local SSDs have a fixed 375 GB capacity for each device that you attach to the instance. If you want to combine multiple local SSD devices into a single logical volume, you must define volume management across these devices yourself.

Note: Due to the underlying hardware configuration, using parity or mirroring RAID levels across local SSD devices does not provide any actual reliability or redundancy benefit. Use RAID 0 to get the best performance and the greatest capacity from your local SSD arrays.

Linux instances

Use mdadm to create a RAID 0 array. This example formats the array with a single ext4 file system, but you can apply any file system that you prefer.

Note: Although you can create an /etc/fstab entry to automatically mount the local SSD during an instance reboot, it does not allow data on the local SSD to persist through termination or preemption.

1. Go to the VM instances page.

   Go to the VM instances page

2. Click the SSH button next to the instance that has the new attached local SSD. The browser opens a terminal connection to the instance.

3. In the terminal, install the mdadm tool. The install process for mdadm includes a user prompt that will halt scripts, so run this process manually.

   Debian and Ubuntu:

   $ sudo apt-get update && sudo apt-get install mdadm --no-install-recommends
   

   CentOS and RHEL:

   $ sudo yum install mdadm -y
   

   SLES and openSUSE:

   $ sudo zypper install -y mdadm
   

4. Use the lsblk command to identify all of the local SSDs that you want to mount together. For this example, the instance has eight local SSD devices in NVMe mode:

   $ lsblk
   
   NAME    MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
   sda       8:0    0   10G  0 disk
   └─sda1    8:1    0   10G  0 part /
   nvme0n1 259:0    0  375G  0 disk
   nvme0n2 259:1    0  375G  0 disk
   nvme0n3 259:2    0  375G  0 disk
   nvme0n4 259:3    0  375G  0 disk
   nvme0n5 259:4    0  375G  0 disk
   nvme0n6 259:5    0  375G  0 disk
   nvme0n7 259:6    0  375G  0 disk
   nvme0n8 259:7    0  375G  0 disk
   
   

   Local SSDs in SCSI mode have standard IDs like sdb. Local SSDs in NVMe mode have IDs like nvme0n1.

5. Use mdadm to combine multiple local SSD devices into a single array named /dev/md0. This example merges eight Local SSD devices in NVMe mode. For Local SSD devices in SCSI mode, specify the IDs that you obtained from the lsblk command:

   $ sudo mdadm --create /dev/md0 --level=0 --raid-devices=8 \
   /dev/nvme0n1 /dev/nvme0n2 /dev/nvme0n3 /dev/nvme0n4 \
   /dev/nvme0n5 /dev/nvme0n6 /dev/nvme0n7 /dev/nvme0n8
   
   mdadm: Defaulting to version 1.2 metadata
   mdadm: array /dev/md0 started.
   
   

6. Format the full /dev/md0 array with an ext4 file system. This command deletes all existing data from the local SSDs.

   $ sudo mkfs.ext4 -F /dev/md0
   

7. Create a directory to where you can mount /dev/md0. For this example, create the /mnt/disks/ssd-array directory:

   $ sudo mkdir -p /mnt/disks/[MNT_DIR]
   

   where: [MNT_DIR] is the directory where you want to mount your local SSD array.

8. Mount the /dev/md0 array to the /mnt/disks/ssd-array directory. Optionally, you can disable write cache flushing to improve write performance while risking reduced durability for up to two seconds of cached data writes.

   $ sudo mount /dev/md0 /mnt/disks/[MNT_DIR]
   

   where: [MNT_DIR] is the directory where you want to mount your local SSD array.

9. Configure read and write access to the device. For this example, grant write access to the device for all users.

   $ sudo chmod a+w /mnt/disks/[MNT_DIR]
   

   where: [MNT_DIR] is the directory where you mounted your local SSD array.

Optionally, you can add the local SSD to the /etc/fstab file so that the device automatically mounts again when the instance restarts. This entry does not preserve data on your local SSD if the instance terminates. See Understanding Local SSD data persistence for complete details on local SSD data persistence.

When you specify the entry /etc/fstab file, be sure to include the nofail option so that the instance can continue to boot even if the Local SSD is not present. For example, if you take a snapshot of the boot disk and create a new instance without any local SSDs attached, the instance can continue through the startup process and not pause indefinitely. For instances running Ubuntu 12.04 or Ubuntu 14.04, specify the nobootwait option instead of nofail.

1. Create the /etc/fstab entry. Use the blkid command to find the UUID for the file system on the device and edit the /etc/fstab file to include that UUID with the mount options. You can complete this step with a single command. For example:

   $ echo UUID=`sudo blkid -s UUID -o value /dev/md0` /mnt/disks/[MNT_DIR] ext4 discard,defaults,[NOFAIL_OPTION] 0 2 | sudo tee -a /etc/fstab
   

   where:

   • [MNT_DIR] is the directory where you mounted your local SSD array.
   • [NOFAIL_OPTION] is the option that specifies what the system should do if it is unable to mount this local SSD. For most distributions, specify nofail. For instances running Ubuntu 12.04 or Ubuntu 14.04, specify the nobootwait option.

2. Use the cat command to verify that your /etc/fstab entries are correct:

   $ cat /etc/fstab
   

If you create a snapshot from the boot disk of this instance and use it to create a separate instance that does not have local SSDs, edit the /etc/fstab file and remove the entry for this local SSD array. Even with the nofail or nobootwait options in place, keep the /etc/fstab file in sync with the devices that are attached to your instance and remove these entries before you create your boot disk snapshot.

Windows instances

Use the Windows Disk Management tool to format and mount an array of local SSDs on a Windows instance.

1. Connect to the instance through RDP. For this example, go to the VM instances page and click the RDP button next the instance that has the local SSDs attached. After you enter your username and password, a window opens with the desktop interface for your server.

2. Right click the Windows Start button and select Disk Management.

   

3. If you have not initialized the local SSDs before, the tool prompts you to select a partitioning scheme for the new devices. Select GPT and click OK.

   

4. After the local SSD initializes, right click the unallocated disk space and select New Striped Volume.

   

5. Select the local SSD devices that you want to include in the striped array. For this example, select all of the devices to combine them into a single local SSD device.

   

6. Follow the instructions in the New Striped Volume Wizard to configure the new volume. You can use any partition format that you like, but for this example select NTFS. Also, check Perform a quick format to speed up the formatting process.

   

7. After you complete the wizard and the volume finishes formatting, check the new local SSD to ensure it has a Healthy status.

   

That's it! You can now write files to the local SSD.