Performing other tasks with your instances

These are other tasks that you might want to perform with your instances.

Before you begin

• If you want to use the command-line examples in this guide:
  1. Install or update to the latest version of the gcloud command-line tool.
  2. Set a default region and zone.
• If you want to use the API examples in this guide, set up API access.

Copy files between an instance and local computer

Use gcloud compute scp to transfer files between a Linux instance and local computer:

gcloud compute scp my-instance:~/file-1 \
    my-instance:~/file-2 \
    ~/local-dir

Note: An instance must have an external IP address and a Linux image to copy files using the gcloud compute command.

To copy files from your local machine to your instance:

gcloud compute scp ~/local-dir/file-1 \
    my-instance:~/remote-destination

Detect if you are running in Compute Engine

It is common for systems to want to detect if they are running within a specific cloud environment. Use the following instructions to help you detect if you are running in Compute Engine.

Use the metadata server

You can easily detect if your applications or scripts are running within a Compute Engine instance by using the metadata server. When you make a request to the server, any response from the metadata server will contain the Metadata-Flavor: Google header. You can look for this header to reliably detect if you are running in Compute Engine.

For example, the following curl request returns a Metadata-Flavor: Google header, indicating that the request is being made from within a Compute Engine instance.

me@my-inst:~$ curl metadata.google.internal -i


HTTP/1.1 200 OK
Metadata-Flavor: Google
Content-Type: application/text
Date: Thu, 10 Apr 2014 19:24:27 GMT
Server: Metadata Server for VM
Content-Length: 22
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN


0.1/

computeMetadata/

Use other methods

Linux

For Linux instances, the dmidecode tool can be used to access the DMI/SMBIOS information in /proc/mem directly. Run the following command and the dmidecode tool should print out "Google Compute Engine" to indicate that you are running in Compute Engine:

my@myinst:~$ sudo dmidecode -s system-product-name | grep "Google Compute Engine"
Google Compute Engine

Windows

On Windows instances, "Google" is listed as the system manufacturer and model. You can use utilities like msinfo32.exe to look up this information. For example, msinfo32 displays the following information:

msinfo32 screen displaying Google as manufacturer and model (click to enlarge)

If you need to programmatically determine this information on a Windows instance, you can create a Windows Management Instrumentation (WMI) application with some modifications.

How to handle instance failures

Unfortunately, individual instances will experience failures from time to time. This can be due to a variety of reasons, including unexpected outages, hardware error, or other system failures. As a way to mitigate such situations, you should use persistent disks and back up your data routinely.

If an instance fails, it will be restarted automatically, with the same root persistent disk, metadata, and instance settings as when it failed. To control the automatic restart behavior for an instance, see How to set scheduling options.

In general, you should design your system to be robust enough that the failure of a single instance should not be catastrophic to your application. For more information, see Designing robust systems.

Identify an instance through the UUID

Each virtual machine has a universally unique identifier (UUID) that can be accessed through the dmidecode tool for Linux images. A UUID is calculated from the project ID, zone, and instance name of the virtual machine. An instance's UUID is unique among Compute Engine virtual machines and stable for the lifetime of the instance. UUIDs will persist through virtual machine restarts and if the virtual machine is deleted and recreated in the same project and zone, with the same instance name, the UUID will also be the same.

Note: UUIDs are only available on virtual machines created on or after 10/30/2014. If you have an existing instance that was created before that date, you can recreate your instance and it will be given a UUID.

To find an instance's UUID on a Linux instance, run the following command from your virtual machine:

me@myinst:~$ sudo dmidecode -t system | grep UUID

The tool should print a response similar to the following:

  UUID: FE0C672D-324F-25F1-052C-6C50FA8B7397

Install packages and configure an instance

The instance creator has administrator privileges on any instance that they add to a project, and is automatically on the SUDO list.

When you are logged into an instance as the administrator, you can install packages and configure the instance the same way you would a normal Linux box. For example, you can install Apache, as shown here:

user@myinst:~$ sudo apt-get update && sudo apt-get install apache2
Reading package lists... Done

Building dependency tree
Reading state information... Done
The following extra packages will be installed:

[...]

You can move files between your local computer and instance using gcloud compute scp as described in Copying files between an instance and local computer.

Note that your machine needs access to the Internet to be able to run apt-get. This means that it needs either an external IP address, or access to an Internet proxy.

Compute Engine changes a special attribute in a virtual machine's metadata server shortly before any attempts to live migrate or terminate and restart the virtual machine as part of a pending infrastructure maintenance event. The maintenance-event attribute will be updated before and after an event, allowing you to detect when these events are imminent. You can use this information to help automate any scripts or commands you want to run before and/or after a maintenance event.

For more information, see the Transparent maintenance notice section in the Metadata server documentation page.

List all instances

You can see a list of all instances in a project by calling instances list:

gcloud compute instances list

NAME               ZONE          MACHINE_TYPE  INTERNAL_IP    EXTERNAL_IP     STATUS
example-instance   us-central1-a n1-standard-1 10.105.155.92  173.255.114.53  RUNNING
example-instance-2 us-central1-a n1-standard-1 10.181.215.203 146.148.32.59   RUNNING

By default, gcloud compute provides an aggregate listing of all your resources across all available zones. If you want a list of resources from just a single zone, provide the --zones flag in your request.

In the API, you need to make requests to two different methods to get a list of aggregate resources or a list of resources within a zone. To make a request for an aggregate list, make a GET request to that resource's aggregatedList URI, replacing the project with your own project ID:

https://compute.googleapis.com/compute/v1/projects/myproject/aggregated/instances

In the client libraries, make a request to the instances().aggregatedList function:

def listAllInstances(auth_http, gce_service):
  request = gce_service.instances().aggregatedList(project=PROJECT_ID)
  response = request.execute(auth_http)

  print response

To make a request for a list of instances within a zone, make a GET request to the following URI, replacing the project with your own project ID and the zone with the zone of the instances:

https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-f/instances

In the API client libraries, make a instances().list request:

def listInstances(auth_http, gce_service):
  request = gce_service.instances().list(project="myproject",
    zone="us-central1-f")
  response = request.execute(auth_http)

  print response

Reduce costs with preemptible instances

Compute Engine offers preemptible instances that you can create and run at a much lower price than regular instances. If your applications are fault-tolerant and can withstand instance instability, then preemptible instances can reduce your Compute Engine costs significantly. See the Preemptible Instances documentation for more information.

Set up network time protocol (NTP) for instances

Many software systems that depend on careful sequencing of events rely on a stable, consistent system clock. System logs written by most services include a timestamp, which helps debug issues that occur between various components of your system. To help keep system clocks in sync, Compute Engine instances are preconfigured to use network time protocol (NTP).

In addition to keeping server time in sync, NTP is helpful in the rare case of a leap second. A leap second is a one-second adjustment made to UTC time to account for changes in the Earth's rotation. Leap seconds don’t happen at routine intervals, because the Earth's rotation speed varies irregularly in response to climatic and geological events. Previous leap seconds have wreaked havoc on a variety of services and applications on the web. NTP servers help ensure that all servers report the same time during the event of a leap second.

This section describes how to configure NTP servers on your virtual machines to behave properly in the case of a leap second.

Google NTP servers and leap smearing

Leap seconds in Unix time are commonly implemented by repeating the last second of the day. This can cause problems with software that expects timestamps to only ever increase. To get around this problem, the time servers at Google "smear" the extra second over twenty hours—ten before and ten after the leap second event—so that computers do not see the extra second all at once as a repeated timestamp. This reduces risk in systems that depend on a consistent timestamp. It is recommended that all Compute Engine virtual machine instances are configured to use the internal Google NTP services.

Configure NTP for your instances

Google can’t predict how external NTP services, such as pool.ntp.org, will handle the leap second. If at all possible, it is recommended that you do not use external NTP sources with Compute Engine virtual machines. Even worse, using both Google's NTP service and an external service can result in unpredictable changes in the system time. Using only a single external NTP source is preferable to using a mix, but external NTP services, such as pool.ntp.org, will likely use stepping to handle the leap second. As a result, your virtual machines may see a repeated timestamp.

The safest approach is to configure your Compute Engine virtual machines to use a single NTP server—the internal NTP server provided by Google. Do not mix external NTP servers and Google NTP servers, as this could result in unexpected behavior.

To ensure your virtual machines are correctly configured, follow these instructions.

Linux

1. Use ssh to connect to your instance.

   Console

   1. Go to the VM instances page in the Cloud Console.

   2. Click the ssh button next to the instance you want to configure.

      

   gcloud

   With the gcloud command-line tool, run:

   gcloud compute instances ssh INSTANCE
   

2. On your instance, run ntpq -p to check the current state of your NTP configuration:

   $ ntpq -p
   
   
        remote           refid      st t when poll reach   delay   offset  jitter
   
   ==============================================================================
   *metadata.google 255.28.23.83     2 u   27   64    1    0.634   -2.537   2.285
   *217.162.232.173 130.149.17.8     2 u  191 1024  176   79.245    3.589  27.454
   

   If you see a single record pointing at metadata.google or metadata.google.internal, you do not need to make any changes. If you see multiple sources, mixed between metadata.google and a public source such as pool.ntp.org, you need to update your sources to remove any external NTP servers.

   For this example, there are two records, one pointing at metadata.google and another pointing to an external address. Because there are multiple sources, you would need to update your NTP servers to remove the *217.162.232.173 address.

   1. Configure your NTP servers to remove external sources.

      To configure your NTP servers, edit the /etc/ntp.conf file in your favorite text editor. Find the servers section of the configuration, and remove all non-Google NTP sources:

   $ vim /etc/ntp.conf
   

   ...
   # You do need to talk to an NTP server or two (or three).
   #server ntp.your-provider.example
   ...
   server metadata.google.internal iburst
   

   After editing your /etc/ntp.conf file, restart the NTP service. The command to restart may vary based on the Linux distribution:

   $ sudo service ntp reload

   1. Verify your configuration. Run the ntpq -p command again to verify that your changes:

   $ ntpq -p
   
        remote           refid      st t when poll reach   delay   offset  jitter
   ==============================================================================
   *metadata.google 255.28.23.83     2 u   27   64    1    0.634   -2.537   2.285
   

Windows

1. Go to the VM instances page in the Cloud Console.

2. Click the RDP button next to the Windows instance you want to log into.

   

3. After you are logged in, right-click on the PowerShell icon and select Run as administrator.

   

4. When the command-prompt loads, run the following command to see the current NTP configuration:

   PS C:\Windows\system32>w32tm /query /configuration
   [Configuration]
   ...
   Type: NTP (Local)
   NtpServer: metadata.google.internal,
   ...
   

   If you see a single record pointing at metadata.google or metadata.google.internal, you do not need to make any changes. If you see multiple sources, mixed between metadata.google and a public source, you need to remove the external server. Follow the Windows' guide for configuring your NTP server.

The leap smearing feature of Google’s NTP servers is a convenient way to manage the risk involved with replaying a second on time sensitive systems. Other NTP services may provide an acceptable work around for most software systems. It is just important not to mix Google leap smearing NTP services with public NTP stepping services.

To synchronize devices outside Google Cloud to smeared time, use Google Public NTP. It uses the same smear provided to Compute Engine virtual machine instances.