Known Issues

This page describes known issues that you might run into while using Compute Engine.

Known issues for Linux VM instances

The following are known issues with Linux images:

GPG error: EXPKEYSIG 3746C208A7317B0F when updating packages

On Debian and Ubuntu based systems, including your local workstations, you might encounter an error similar to the following example:

W: An error occurred during the signature verification.
The repository is not updated and the previous index files will be used.
GPG error: http://packages.cloud.google.com/apt cloud-sdk-stretch InRelease:
The following signatures were invalid: EXPKEYSIG 3746C208A7317B0F
Google Cloud Packages Automatic Signing Key <[e-mail]>

This error prevents you from obtaining the latest updates for several Google Cloud tools, including the following items:

• Compute Engine Linux Guest Environment
• Cloud SDK with the gcloud command-line tool
• Stackdriver Logging Agent

To resolve this error, obtain the latest valid apt-key.gpg key file from https://packages.cloud.google.com:

curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -

Alternatively, on Compute Engine VM instances running Debian or Ubuntu images, you can get the latest keys if you recreate your instances using the following image versions:

• Image project debian-cloud:
  • debian-9-stretch-v20180401 or image family debian-9
  • debian-8-jessie-v20180401 or image family debian-8
• Image project ubuntu-os-cloud:
  • ubuntu-1710-artful-v20180315 or image family ubuntu-1710
  • ubuntu-1604-xenial-v20180323 or image family ubuntu-1604-lts
  • ubuntu-1404-trusty-v20180308 or image family ubuntu-1404-lts

Red Hat Enterprise Linux 7 and CentOS 7 Read-only Root Filesystem Issue

VM instances running public images rhel-7-v20170719 and earlier, or centos-7-v20170719 and earlier can boot with the root filesystem mounted in read-only mode. Applications, daemons, or scripts that require write access to the root filesystem will fail.

Do not reboot or restart running instances that use the affected public images or they will become stuck in read-only mode. If your instances are already stuck in read-only mode, you can remotely restore read-write mode to the root filesystem and then fix the issue.

Identify affected instances:

Identify potentially affected instances with the following gcloud compute disks list commands:

RHEL 7:

gcloud compute disks list --filter="sourceImage ~ rhel-7-v201[4-6].* OR sourceImage ~ rhel-7-v20170[1-7].*" --uri

CentOS 7:

gcloud compute disks list --filter="sourceImage ~ centos-7-v201[4-6].* OR sourceImage ~ centos-7-v20170[1-7].*" --uri

If these disks are attached to your instances, you can correct the issue on those instances. The process to correct affected instances depends on the image version that you used to create the instance.

Instances created using RHEL 7 images between rhel-7-v20160418 and rhel-7-v20170719, or CentOS 7 images between centos-7-v20160418 and centos-7-v20170719:

If the instance uses automatic updates, yum-cron automatically installs the fixed package and removes the broken mount option from the /etc/fstab file. If the instance does not have automatic updates enabled, you can fix the instance with the following process:

1. Connect to the instance using SSH. If the connection fails, the instance might be stuck in read-only mode. However, you can attempt to recover it. If you have connected to the affected instance over SSH before, your public SSH keys are already on the root file system and will continue to function. Run a remote command over ssh to remount the filesystem in rw mode. For example, you can use the following gcloud command to remount the root file system:

   gcloud compute ssh [INSTANCE_NAME] --command "sudo mount -o remount,rw /dev/sda1 /"
   

   After the filesystem is mounted in read-write mode again, connect to the instance over SSH.

2. Run sudo yum -y update to update all installed packages including the gce-disk-expand package, which contains the fix.

The instance can now reboot without mounting the root filesystem in read-only mode.

Instances created using RHEL 7 images created before rhel-7-v20160418 or CentOS 7 images created before centos-7-v20160418:

1. Connect to the instance using SSH. If the connection fails, the instance might be stuck in read-only mode. However, you can attempt to recover it. If you have connected to the affected instance over SSH before, your public SSH keys are already on the root file system and will continue to function. Run a remote command over ssh to remount the filesystem in rw mode. For example, you can use the following gcloud command to remount the root file system:

   gcloud compute ssh [INSTANCE_NAME] --command "sudo mount -o remount,rw /dev/sda1 /"
   

   After the filesystem is mounted in read-write mode again, connect to the instance over SSH.

2. Edit the /etc/fstab file and remove any barrier=1 mount options in that file. The default mount option must be the only mount option set for the root file system entry. You can correct this broken mount option with the following command:

   sudo sed -i 's/defaults,barrier[^ ,]*/defaults/' /etc/fstab
   

   After you remove the barrier=1 mount option, the root file system entry in the /etc/fstab file should look similar the following example, but with a different value for the UUID:

   UUID=b5e54172-67e3-4d52-95f4-4314e71b25fd / xfs defaults 0 0
   

The instance can now reboot without mounting the root filesystem in read-only mode.

CentOS image v20131120 introduced a breaking change where iptables are turned on by default

The v20131120 release of CentOS 6 image, centos-6-v20131120, has a breaking change where iptables are turned on by default. This prevents external traffic from reaching CentOS instances that are running centos-6-v20131120, even if there is a relevant Firewall Rule resource permitting the connection.

As a workaround, users will need to disable iptables or update iptables to permit the desired connection (in addition to permitting the traffic using firewall rules). To disable iptables, run:

# Save your iptable settings
user@centos-instance:~$ sudo service iptables save

# Stop the iptables service
user@centos-instance:~$ sudo service iptables stop

# Disable iptables on start up
user@centos-instance:~$ sudo chkconfig iptables off

To update iptables, review the iptables documentation.

Google-provided images have known bug with the ext4/scsi driver in the stable Debian and CentOS kernels

For centos-6-v20131120 and debian-7-wheezy-v20131120 images, a known ext4 bug might cause a memory leak and eventual crash of a virtual machine instance under heavy persistent disk load. For details, see this Linux Kernel Mailing list thread.

Instance names longer than 32 characters can cause problems with various UNIX tools

Date Reported: June 2012

Although instance names can be up to 63 characters, names that are longer than 32 characters may cause some tools to be unreliable, including tools that may run during boot. As a workaround, choose instance names that are shorter than 32 characters.

Instances using OS Login return a login message after connection

On some instances that use OS Login, you might receive the following error message after the connection is established:

/usr/bin/id: cannot find name for group ID 123456789

Ignore the error message.

Known issues for Windows VM instances

The following are known issues with Windows images:

• Although Windows instances can use the NVMe interface with local SSD, support for NVMe on Windows is in Beta and we do not guarantee the same performance as Linux instances.
• On Windows 2008 R2, installing Python 2.7.9 or newer requires Visual C++. Python 2.7.8 avoids this dependency but we recommend installing the latest version.
• Compute Engine does not yet support IPv6. Even if you enable IPv6 by selecting the option in a Windows instance, the setting is ignored.
• Once a new instance is created, you cannot connect to it instantly. All new Windows instances use the System Preparation (sysprep) tool to set up your instance, which can take 5-10 mins to complete.
• Windows Server images cannot activate without a network connection to kms.windows.googlecloud.com, and stop functioning if they do not initially authenticate within 30 days. Software activated by the KMS must reactivate every 180 days, but the KMS attempts to reactivate every 7 days. Create an external IP for your Windows instances so they can authenticate periodically.
• Kernel software accessing non-emulated Model Specific Registers will generate General Protection Faults, which can cause a system crash depending on the guest operating system.