Troubleshooting RDP

In some situations, you might not be able to connect to your Compute Engine Windows virtual machine (VM) instance with RDP. This might be due to configuration errors, network errors, or the boot process might not have completed.

This page describes a number of tips and approaches to troubleshoot and resolve common RDP issues.

Ensure the VM is online and ready

Once the boot process is completed, which might take a few minutes, the status of your Windows VM is available through serial console output. Review the current state of the VM using one of the following methods:

  • Serial Console 1 (read only)

    1. Open your Windows instance from the VM instances page in Google Cloud console.
    2. Click the drop-down arrow next to Connect to serial console, and select Serial port 1, or the gcloud SDK command below:

      gcloud compute connect-to-serial-port [INSTANCE_NAME] --port 1
    3. Confirm that the instance has started, and has attempted to run startup scripts.

      Booting from Hard Disk 0...
      GCEWindowsAgent: GCE Agent Started (version x.x.x@x)
      GCEMetadataScripts: No startup scripts to run.
  • Serial Console 2 (interactive)

    1. Enable interactive access on the serial console by using the VM instance page , or the gcloud SDK command below:

      gcloud compute instances add-metadata [INSTANCE_NAME]^
    2. Open your Windows instance from the VM instances page in Google Cloud console.

    3. Click the drop-down arrow next to Connect to serial console, and select Serial port 2. Alternatively, you can run the following gcloud SDK command:

      gcloud compute connect-to-serial-port [INSTANCE_NAME] --port 2
    4. Confirm that the cmd process is available to use.

      Available SAC cmd command.

  • VM Screenshot

    1. Enable the virtual display if it is not already enabled.

    2. Capture a screenshot of the VM.

    3. Review the screenshot to see that the instance is ready.

    If the VM is accessible by RDP, the screenshot will look similar to the following:

    Login screen for Windows Server 2016 Datacenter.

    The VM isn't accessible by RDP if you see one of the following:

    • The UEFI loading screen
    • A Windows blue screen error
    • The Windows Update progress screen

    If the screenshot displays a blue screen error, review Troubleshooting blue screen errors. If the screenshot displays a loading screen or an update screen, wait for loading or the update to complete before continuing.

Check your VPC firewall rules

Compute Engine automatically provisions new projects with a firewall rule that allows RDP traffic. If you have an existing project, or have modified the configurations, the default firewall rule that permits RDP might not exist. Confirm that a rule allows RDP traffic to connect to the network that your affected instance is on.

To check if the default-allow-rdp firewall rule exists on your project, check the Firewall rules page, or run the following gcloud SDK command:

gcloud compute firewall-rules list

To create a new rule if one does not exist, create a rule with the following command:

gcloud compute firewall-rules create allow-rdp --allow tcp:3389

Check your Windows instance password

Each Compute Engine Windows instance must have a local password set if it is not already on a domain or custom image. Confirm you have the correct password set by connecting to the Windows SAC through the gcloud command-line tool or Cloud Console.

If you have problems connecting, try creating or resetting the password on the Windows instance.

Verify the external IP address

Ensure that you're connecting to the correct external IP address for the instance. View the IP for the instance from the VM instance page or by using the following gcloud SDK command:

gcloud compute instances list

Use of Windows Remote Desktop Services (RDS)

If you have Windows Remote Desktop Services (formerly known as Terminal Services) installed on your instance, then the conditions of the Client Access Licenses (CALs) are enforced. With these CALs, RDP connections will fail under any of the following conditions:

  • You have used all your available licenses
  • Your license is installed, but not configured or activated correctly
  • Your RDS trial period of 180 days has expired

Symptoms that you may not have enough valid licenses include messages such as:

  • This remote session was disconnected because there are no Remote Desktop License Servers available to provide a license.
  • The remote session was disconnected because of an error related to licensing in terminal server.
  • The remote session was disconnected because there are no Remote Desktop client access licenses available for this computer.

If your RDP connections fail, you can use the admin switch to connect to the instance for administrative purposes. This can be done on a Windows machine by using the native Remote Desktop Connection client.

%systemroot%/system32/mstsc.exe /admin

To resolve issues with RDP connections, purchase new licenses for your instance. For more details about CALs, review the Microsoft documentation. Alternatively, if Remote Desktop Services is not required, uninstall the service and use the regular RDP connections.

Check the OS configuration

If the environment and configurations for the instance are correct, the operating system on the instance might be misconfigured. You can use the interactive serial console to connect to the instance and troubleshoot the problem.

Connect to the instance through one of the available command line methods, and run the following commands to ensure that the instance is accepts connections:

  1. Check to see that the 'Remote Desktop Service' is running:

    • Command: net start | find "Remote Desktop Services"
    • Pass: Remote Desktop Service
    • Fail: (Remote Desktop Service missing from output)
    • Solution: net start "Remote Desktop Services"
  2. Check that Remote Connections are enabled:

    • Command: reg query "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server" /v fDenyTSConnections
    • Pass: fDenyTSConnections REG_DWORD 0x0
    • Fail: fDenyTSConnections REG_DWORD 0x1
    • Solution: reg add "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server" /f /v fDenyTSConnections /t REG_DWORD /d 0
  3. Ensure that the Windows firewall has Remote Desktop Connections enabled:

    • Command: netsh advfirewall firewall show rule name="Remote Desktop - User Mode (TCP-In)"
    • Pass: Enabled:Yes, Direction: In, Profiles: Public, Grouping: Remote Desktop, LocalIP: Any, RemoteIP: Any, Protocol:TCP, LocalPort: 3389, RemotePort: Any, Edge traversal: No, Action: Allow

    • Fail: (unexpected results, such as enabled = No)

    • Solution: netsh firewall set service remotedesktop enable

  4. Check to see what port number is configured for RDP connections on the remote instance:

    • Command: reg query "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v PortNumber
    • Pass: PortNumber REG_DWORD [PORT NUMBER]
    • Fail: (unexpected port number)
    • Solution: reg add "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /f /v PortNumber /t REG_DWORD /d [PORT NUMBER]

  5. Ensure that connected user account has permissions for remote connections:

    • Command: net localgroup "Remote Desktop Users"
    • Pass: (target local/domain username in resulting list)
    • Fail: (target local/domain username missing)
    • Solution: net localgroup "Remote Desktop Users" /add [DOMAIN\USERNAME]

  6. Reset client/server security negotiation to its default value:

    • Command: reg query "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v SecurityLayer
    • Pass: SecurityLayer REG_DWORD 0x1
    • Fail: SecurityLayer REG_DWORD 0x0 (or 0x2)
    • Solution: reg add "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v SecurityLayer /t REG_DWORD /d 1 /f
  7. Reset user network level authentication to its default value:

    • Command: reg query "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v UserAuthentication
    • Pass: UserAuthentication REG_DWORD 0x0
    • Fail: UserAuthentication REG_DWORD 0x1
    • Solution: reg add "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp" /v UserAuthentication /t REG_DWORD /d 0 /f
  8. Verify that your MTU size is no greater than the MTU of the network:

    • Command: netsh interface ipv4 show subinterfaces
    • Pass: When the number after the MTU matches the MTU of the VPC network.
    • Fail: When the number after MTU is larger than the MTU of the VPC network.
    • Solution: netsh interface ipv4 set subinterface Ethernet mtu=MTU_OF_VPC_NETWORK

      Note: VPC networks have a default maximum transmission unit (MTU) of 1460 bytes. However, the network MTU can be set to 1500 bytes. See Maximum transmission unit for background on network MTUs.
  9. Ensure that your antivirus/endpoint protection client settings allow for the configured port number and services.

Use your disk on a new instance

If the other troubleshooting steps do not resolve your connection issue, you might be able to delete the existing instance and preserve your persistent disk. Remount the disk to a new instance and make another troubleshooting attempt, or simply recover existing data from the disk.

gcloud compute instances delete [INSTANCE_NAME] --keep-disks=boot
gcloud compute instances create [NEW_INSTANCE_NAME] --disk name=[DISK_NAME],boot=yes,auto-delete=no