Before you begin
- If you want to use the command-line examples in this guide:
- If you want to use the API examples in this guide, set up API access.
Copy files between an instance and local computer
gcloud compute scp
to transfer files between a Linux instance and local computer:
gcloud compute scp my-instance:~/file-1 \ my-instance:~/file-2 \ ~/local-dir
To copy files from your local machine to your instance:
gcloud compute scp ~/local-dir/file-1 \ my-instance:~/remote-destination
Detect if you are running in Compute Engine
It is common for systems to want to detect if they are running within a specific cloud environment. Use the following instructions to help you detect if you are running in Compute Engine.
Use the metadata server
You can easily detect if your applications or scripts are running within a
Compute Engine instance by using the metadata server. When you make a
request to the server, any response from the metadata server will contain the
Metadata-Flavor: Google header. You can look for this header to
reliably detect if you are running in Compute Engine.
For example, the following curl request returns a
header, indicating that the request is being made from within a
Compute Engine instance.
me@my-inst:~$ curl metadata.google.internal -i HTTP/1.1 200 OK Metadata-Flavor: Google Content-Type: application/text Date: Thu, 10 Apr 2014 19:24:27 GMT Server: Metadata Server for VM Content-Length: 22 X-XSS-Protection: 1; mode=block X-Frame-Options: SAMEORIGIN 0.1/ computeMetadata/
Use other methods
For Linux instances, the
tool can be used to access the DMI/SMBIOS information in
directly. Run the following command and the dmidecode tool should print out
"Google Compute Engine" to indicate that you are running in Google Compute Engine:
my@myinst:~$ sudo dmidecode -s system-product-name | grep "Google Compute Engine" Google Compute Engine
On Windows instances, "Google" is listed as the system manufacturer and model. You can use utilities like msinfo32.exe to look up this information. For example, msinfo32 displays the following information:
If you need to programmatically determine this information on a Windows instance, you can create a Windows Management Instrumentation (WMI) application with some modifications.
How to handle instance failures
Unfortunately, individual instances will experience failures from time to time. This can be due to a variety of reasons, including unexpected outages, hardware error, or other system failures. As a way to mitigate such situations, you should use persistent disks and back up your data routinely.
If an instance fails, it will be restarted automatically, with the same root persistent disk, metadata, and instance settings as when it failed. To control the automatic restart behavior for an instance, see How to set scheduling options.
In general, you should design your system to be robust enough that the failure of a single instance should not be catastrophic to your application. For more information, see Designing Robust Systems.
Identify an instance through the UUID
Each virtual machine has a universally unique identifier (UUID) that can be accessed through the dmidecode tool for Linux images. A UUID is calculated from the project ID, zone, and instance name of the virtual machine. An instance's UUID is unique among Compute Engine virtual machines and stable for the lifetime of the instance. UUIDs will persist through virtual machine restarts and if the virtual machine is deleted and recreated in the same project and zone, with the same instance name, the UUID will also be the same.
To find an instance's UUID on a Linux instance, run the following command from your virtual machine:
me@myinst:~$ sudo dmidecode -t system | grep UUID
The tool should print a response similar to the following:
Install packages and configure an instance
The instance creator has administrator privileges on any instance that they add to a project, and is automatically on the SUDO list.
When you are logged into an instance as the administrator, you can install packages and configure the instance the same way you would a normal Linux box. For example, you can install Apache, as shown here:
user@myinst:~$ sudo apt-get update && sudo apt-get install apache2 Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: [...]
You can move files between your local computer and instance using
gcloud compute scp as described in Copying files between an instance and local computer.
Compute Engine changes a special attribute in a virtual machine's
metadata server shortly before any attempts to live migrate or terminate and
restart the virtual machine as part of a pending infrastructure maintenance
maintenance-event attribute will be updated before and after an
event, allowing you to detect when these events are imminent. You can use this
information to help automate any scripts or commands you want to run before
and/or after a maintenance event.
List all instances
You can see a list of all instances in a project by calling
gcloud compute instances list
NAME ZONE MACHINE_TYPE INTERNAL_IP EXTERNAL_IP STATUS example-instance us-central1-a n1-standard-1 10.105.155.92 188.8.131.52 RUNNING example-instance-2 us-central1-a n1-standard-1 10.181.215.203 184.108.40.206 RUNNING
gcloud compute provides an aggregate listing of all your resources
across all available zones. If you want a list of resources from just a single
zone, provide the
--zones flag in your request.
In the API, you need to make requests to two different methods to get a list
of aggregate resources or a list of resources within a zone. To make a request
for an aggregate list, make a
GET request to that resource's
URI, replacing the project with your own project ID:
In the client libraries, make a request to the
def listAllInstances(auth_http, gce_service): request = gce_service.instances().aggregatedList(project=PROJECT_ID) response = request.execute(auth_http) print response
To make a request for a list of instances within a zone, make a
request to the following URI, replacing the project with your own project ID and
the zone with the zone of the instances:
In the API client libraries, make a
def listInstances(auth_http, gce_service): request = gce_service.instances().list(project="myproject", zone="us-central1-f") response = request.execute(auth_http) print response
Reduce costs with preemptible instances
Google Compute Engine offers preemptible instances that you can create and run at a much lower price than regular instances. If your applications are fault-tolerant and can withstand instance instability, then preemptible instances can reduce your Compute Engine costs significantly. See the Preemptible Instances documentation for more information.
Set up network time protocol (NTP) for instances
Many software systems that depend on careful sequencing of events rely on a stable, consistent system clock. System logs written by most services include a timestamp, which helps debug issues that occur between various components of your system. To help keep system clocks in sync, Compute Engine instances are preconfigured to use network time protocol (NTP).
In addition to keeping server time in sync, NTP is helpful in the rare case of a leap second. A leap second is a one-second adjustment made to UTC time to account for changes in the Earth's rotation. Leap seconds don’t happen at routine intervals, because the Earth's rotation speed varies irregularly in response to climatic and geological events. Previous leap seconds have wreaked havoc on a variety of services and applications on the web. NTP servers help ensure that all servers report the same time during the event of a leap second.
This section describes how to configure NTP servers on your virtual machines to behave properly in the case of a leap second.
Google NTP servers and leap smearing
Leap seconds in Unix time are commonly implemented by repeating the last second of the day. This can cause problems with software that expects timestamps to only ever increase. To get around this problem, the time servers at Google "smear" the extra second over twenty hours—ten before and ten after the leap second event—so that computers do not see the extra second all at once as a repeated timestamp. This reduces risk in systems that depend on a consistent timestamp. It is recommended that all Compute Engine virtual machine instances are configured to use the internal Google NTP services.
Configure NTP for your instances
Google can’t predict how external NTP services, such as pool.ntp.org, will handle the leap second. If at all possible, it is recommended that you do not use external NTP sources with Compute Engine virtual machines. Even worse, using both Google's NTP service and an external service can result in unpredictable changes in the system time. Using only a single external NTP source is preferable to using a mix, but external NTP services, such as pool.ntp.org, will likely use stepping to handle the leap second. As a result, your virtual machines may see a repeated timestamp.
The safest approach is to configure your Compute Engine virtual machines to use a single NTP server—the internal NTP server provided by Google. Do not mix external NTP servers and Google NTP servers, as this could result in unexpected behavior.
To ensure your virtual machines are correctly configured, follow these instructions.
Use ssh to connect to your instance.
- Go to the VM instances page in the GCP Console.
Click the ssh button next to the instance you want to configure.
gcloudcommand-line tool, run:
gcloud compute instances ssh INSTANCE
On your instance, run
ntpq -pto check the current state of your NTP configuration:
$ ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== *metadata.google 255.28.23.83 2 u 27 64 1 0.634 -2.537 2.285 *220.127.116.11 18.104.22.168 2 u 191 1024 176 79.245 3.589 27.454
If you see a single record pointing at
metadata.google.internal, you do not need to make any changes. If you see multiple sources, mixed between metadata.google and a public source such as pool.ntp.org, you need to update your sources to remove any external NTP servers.
For this example, there are two records, one pointing at
metadata.googleand another pointing to an external address. Since there are multiple sources, you would need to update your NTP servers to remove the
Configure your NTP servers to remove external sources.
To configure your NTP servers, edit the
/etc/ntp.conffile in your favorite text editor. Find the
serverssection of the configuration, and remove all non-Google NTP sources:
$ vim /etc/ntp.conf
... # You do need to talk to an NTP server or two (or three). #server ntp.your-provider.example ... server metadata.google.internal iburst
After editing your
/etc/ntp.conffile, restart the NTP service. The command to restart may vary based on the Linux distribution:
$ sudo service ntp reload
- Verify your configuration. Run the
ntpq -pcommand again to verify that your changes:
$ ntpq -p remote refid st t when poll reach delay offset jitter ============================================================================== *metadata.google 255.28.23.83 2 u 27 64 1 0.634 -2.537 2.285
- Go to the VM instances page in the GCP Console.
Click the RDP button next to the Windows instance you want to log into.
Once logged in, right-click on the PowerShell icon and select Run as administrator.
When the command-prompt loads, run the following command to see the current NTP configuration:
PS C:\Windows\system32>w32tm /query /configuration [Configuration] ... Type: NTP (Local) NtpServer: metadata.google.internal, ...
If you see a single record pointing at
metadata.google.internal, you do not need to make any changes. If you see multiple sources, mixed between
metadata.googleand a public source, you need to remove the external server. Follow the Windows' guide for configuring your NTP server.
The leap smearing feature of Google’s NTP servers is a convenient way to manage the risk involved with replaying a second on time sensitive systems. Other NTP services may provide an acceptable work around for most software systems. It is just important not to mix Google leap smearing NTP services with public NTP stepping services.
To synchronize devices outside Google's cloud to smeared time, use Google Public NTP. It uses the same smear provided to Compute Engine virtual machine instances.