Running startup scripts

Compute Engine lets you create and run your own startup scripts on your virtual machine (VM) instances to perform automated tasks every time your instance boots up. Startup scripts can perform actions such as installing software, performing updates, turning on services, and any other tasks defined in the script.

For example, a startup script that installs and starts an Apache server could look like this:

#! /bin/bash
apt update
apt -y install apache2
cat <<EOF > /var/www/html/index.html
<html><body><h1>Hello World</h1>
<p>This page was created from a startup script.</p>
</body></html>
EOF

To specify a startup script, you use startup script metadata keys with the metadata server. You can use the Google Cloud Console, gcloud command-line tool, or the Compute Engine API to provide a startup script.

Before you begin

If you want to use the command-line examples in this guide:
1. Install or update to the latest version of the gcloud command-line tool.
2. Set a default region and zone.
If you want to use the API examples in this guide, set up API access.
Understand what the metadata server is.
Configure your text editor to save ASCII text with Unix line endings.

Permissions required for this task

To perform this task, you must have the following permissions:

All permissions required to create a new instance
compute.instances.setMetadata on the instance

Running a startup script

The instance always runs startup scripts as root after the network is available.

A startup script can be of any file type. If a startup script is present, Compute Engine does the following:

Copies the startup script to a local file in the instance.
Sets run permissions on the file.
Runs the file.

You could, for example, provide a Python script instead of a bash script. Keep in mind that Compute Engine runs the script verbatim, regardless of the type of script.

To execute a script that is not bash, add a shebang (a line that starts with #!) at the top of the file. This tells the operating system which interpreter to use. For example, if you use a Python script, you can add the following shebang line:

#! /usr/bin/python

Using a local startup script

A local startup script is a script that is located on your local computer. To use a local startup script, pass a local startup script file to the instance or provide the contents of a startup script directly to the metadata server. The examples in the following subsections show how to add startup-script metadata from a local file or by direct input.

Local startup scripts are subject to the metadata value length limit of 256 KB. If your startup script exceeds this limit, you cannot load it locally. Instead, save your file to Cloud Storage and specify the script URL during instance creation time. For more information, see Providing a startup script stored on Cloud Storage.

Providing a startup script file

You can only pass a local startup script file by using the gcloud command-line tool. Include the --metadata-from-file flag, followed by a metadata key pair, startup-script=PATH_TO_FILE, replacing PATH_TO_FILE with a relative path to the startup script:

gcloud compute instances create example-instance \
  --metadata-from-file startup-script=examples/scripts/install.sh

Providing startup script contents directly

Alternatively, you can use the Google Cloud Console, gcloud command-line tool, or the Compute Engine API to type or paste the contents of your startup script directly.

Console

In Cloud Console, specify a startup script directly in the Startup script section:

In the Cloud Console, go to the VM Instances page.
Go to the VM Instances page
Click Create instance.
On the Create a new instance page, fill in the properties for your instance. For advanced configuration options, expand the Management, security, disks, networking, sole tenancy section.
In the Automation section, under Startup script, supply the contents of your startup script.
Click Create to create the instance.

gcloud

With the gcloud command-line tool, use the --metadata flag to supply the contents of your startup script by using the startup-script=contents key-value pair, replacing contents with the content of your startup script.

For example, the following command creates an instance that, on startup, performs some system updates, installs Apache, and launches a single web page. You can run this command and then visit the instance's external IP address to see the contents of the index.html page.

On a Linux desktop, run the command like so:

gcloud compute instances create example-instance --tags http-server \
  --metadata startup-script='#! /bin/bash
# Installs apache and a custom homepage
  sudo su -
  apt update
  apt -y install apache2
  cat <<EOF > /var/www/html/index.html
  <html><body><h1>Hello World</h1>
  <p>This page was created from a start up script.</p>
  </body></html>
  EOF'

If you are on a Windows desktop, you can run this command on a cmd terminal:

gcloud compute instances create example-instance --tags http-server \
  --metadata startup-script="
  apt update;
  apt -y install apache2;
  echo \"This page was created from a start up script.\" ^> /var/www/html/index.html"

Similarly, if you are using PowerShell, you can run this command by using the stop parsing special character (--%) to pass an exact command to the gcloud tool:

gcloud --% compute instances create example-instance --tags http-server \
  --metadata startup-script="
  apt update;
  apt -y install apache2;
  echo \"This page was created from a start up script.\" ^> /var/www/html/index.html"

API

In the API, provide a startup script as part of the metadata property in your request, using startup-script as the metadata key:

POST https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-a/instances

{
  ...
  "metadata": {
    "items": [
      {
       "key": "startup-script",
       "value": "#! /bin/bash\n\n# Installs apache and a custom homepage\napt update\napt -y install apache2\ncat <<EOF > /var/www/html/index.html\n<html><body><h1>Hello World</h1>\n<p>This page was created from a start up script.</p>\n</body></html>"
      }
    ]
  }
  ...
}

Providing a startup script for Windows instances

You can run startup scripts on Windows instances by using unique, Windows-specific metadata keys. Choose from any of the specialized metadata keys listed in the following table. Each metadata key must match the type of script you want to run. You can also specify multiple scripts by passing different metadata keys to your instance. You can specify each metadata key only one time for each VM instance.

If you need to manually run the startup scripts on a VM instance after the VM instance has started, an administrator can do this by using the following command:

C:\Program Files\Google\Compute Engine\metadata_scripts\run_startup_scripts.cmd

The following table shows the types of startup scripts that are supported, when they run, and which metadata key to use to specify the script. For information about how to use these metadata keys, see Using a local startup script. Note that these scripts only run if you prepare your image by using the GCESysprep command.

Type of script	Runs: During `GCESysprep`² Before boot	Runs: After `GCESysprep`² completes On every subsequent boot
`url` startup scripts	metadata key: `sysprep-specialize-script-url`	metadata key: `windows-startup-script-url`
`cmd` startup scripts	metadata key: `sysprep-specialize-script-cmd`	metadata key: `windows-startup-script-cmd`
`bat` startup scripts	metadata key: `sysprep-specialize-script-bat`	metadata key: `windows-startup-script-bat`
`ps1`¹ startup scripts	metadata key: `sysprep-specialize-script-ps1`	metadata key: `windows-startup-script-ps1`

¹ Windows PowerShell
² GCESysprep

Providing a startup script stored on Cloud Storage

You can store your script on Cloud Storage and provide the URL to the script when creating your instance. This lets you access your startup script from anywhere, and it also bypasses the metadata server limit.

Setting access permissions to your script

Before you can use a startup script from Cloud Storage, you must have permission to access the script. Check the access control settings on the bucket and file to ensure you have permission.

By default, if you are a project owner or editor, you can access files from the same project, unless there are explicit access controls that disallow it.

Providing a startup script

Create a bucket. You can create a bucket or use an existing bucket. For information about creating a bucket, see Create a bucket in the Google Cloud Console or Create a bucket in gsutil.
Upload the file to the bucket. Follow the instructions to use gsutil or the Cloud Console to upload an object.
Give the URL to the startup script file when you create an instance.
Console
1. In the Google Cloud Console, go to the VM instances page.
  
  Go to the VM instances page
2. Click Create Instance.
3. On the Create a new instance page, fill in the properties for your instance.
4. In the Identity and API access section, select a service account that has access to read your startup script file in Cloud Storage. For example, the service account must have the permissions of the Storage Object Viewer role.
5. Expand the Management, security, disks, networking, sole tenancy section.
6. In the Metadata section, provide startup-script-url as the metadata key.
7. In the Value box, provide a URL to the startup script file, either in the gs://BUCKET/FILE or https://storage.googleapis.com/BUCKET/FILE format.
8. Click Create to create the instance.
gcloud
Using the gcloud command-line tool, create an instance with the --scopes and --metadata flags, and specify the startup-script-url key. The --scopes flag gives the VM access to Cloud Storage to download the startup script. You can provide the Cloud Storage URL to the script in either gs://BUCKET/FILE or https://storage.googleapis.com/BUCKET/FILE format.
```
gcloud compute instances create example-instance --scopes storage-ro \
  --metadata startup-script-url=gs://BUCKET/FILE.sh
```
API
In the API, provide a startup script as part of the metadata property in your request, using startup-script-url as the metadata key. Also, provide a list of scopes that includes a Cloud Storage scope, so that the VM can access the startup script file:
```
POST https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-a/instances

{
  ...
  "serviceAccounts": [
    {
      "email": "default",
      "scopes": [
        "https://www.googleapis.com/auth/devstorage.read_only"
      ]
    }
  ],
  "metadata": {
    "items": [
      {
       "key": "startup-script-url",
       "value": "gs://BUCKET/FILE"
      }
    ]
  },
  "tags": {
    "items": []
  },
  "machineType": "zones/us-central1-a/machineTypes/n1-standard-1",
  "name": "example-instance"
}
```
Windows
Using the gcloud command-line tool, create an instance by using the --scopes and --metadata flags, and specify the windows-startup-script-url key. The --scopes flag gives the VM access to Cloud Storage to download the startup script. You can provide the Cloud Storage URL to the script in either gs://BUCKET/FILE or https://storage.googleapis.com/BUCKET/FILE format.
```
gcloud compute instances create example-windows-instance --scopes storage-ro \
  --metadata windows-startup-script-url=gs://BUCKET/FILE.ps1
```

Applying a startup script to running instances

If you have a running instance, add a startup script to the instance and the next time the instance restarts, the startup script will run. The startup script also runs on all subsequent reboots.

Complete the following instructions to set a startup script on a running instance:

Console

In the Google Cloud Console, go to the VM instances page.

Go to the VM instances page
Click the instance to which you want to add a startup script.
On the instance details page, complete the following steps:
1. Click the Edit button.
2. Under Custom metadata, click Add item.
3. Add your startup script using one of the following keys:
  - startup-script: supply the startup script contents directly by using this key.
  - startup-script-url: supply a Cloud Storage URL to the start script file by using this key.

gcloud

Using the gcloud command-line tool, use the instances add-metadata command to add metadata to the instance. Use any of the available startup script keys:

--metadata startup-script=CONTENTS: supply the startup script contents directly by using this key.
--metadata startup-script-url=URL: supply a Cloud Storage URL to the startup script file by using this key.
--metadata-from-file startup-script=FILE: supply a locally stored startup script file.

For example:

gcloud compute instances add-metadata EXAMPLE_INSTANCE \
  --metadata-from-file startup-script=PATH_TO_FILE

gcloud compute instances add-metadata EXAMPLE_INSTANCE \
  --metadata startup-script-url=gs://BUCKET/FILE

API

In the API, make a request to the instances().setMetadata method, providing the new metadata and a fingerprint value.

A fingerprint is a random string of characters generated by Compute Engine that is used to perform optimistic locking. Provide the matching fingerprint value in order to perform your request. The fingerprint changes after each request, and if you provide a mismatched fingerprint, your request is rejected. In this way, only one update can be made at a time, preventing collisions.

To get the current fingerprint of an instance, perform an instances().get request:

GET https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-a/instances/example-instance

And copy the fingerprint value:

{

 ...
 "name": "example-instance",
 "metadata": {
  "kind": "compute#metadata",
  "fingerprint": "zhma6O1w2l8="
 },
 "...
}

Next, make a request to the instances().setMetadata method, using one of the following metadata keys and the fingerprint value:

startup-script: supply the startup script contents directly by using this key.
startup-script-url: supply a Cloud Storage URL to the start script file by using this key.

For example:

POST https://compute.googleapis.com/compute/v1/projects/myproject/zones/us-central1-a/instances/example-instance/setMetadata

    {
     "fingerprint": "zhma6O1w2l8=",
     "items": [
      {
       "key": "startup-script-url",
       "value": "gs://BUCKET/FILE"
      }
     ]
    }

Rerunning a startup script

You can rerun your startup scripts on your VM instance by connecting to the instance and running one of the following commands.

On Debian, CentOS, and RHEL images

sudo google_metadata_script_runner startup

GCEMetadataScripts: Starting startup scripts (version YYMMDD.NN).
GCEMetadataScripts: Found startup-script in metadata.
GCEMetadataScripts: startup-script exit status 0
GCEMetadataScripts: Finished running startup scripts.

On Container-Optimized OS, Ubuntu, and SLES images

sudo google_metadata_script_runner --script-type startup --debug

Viewing startup script logs

Startup script output is written to the following log files:

CentOS and RHEL: /var/log/messages
Debian: /var/log/daemon.log
Ubuntu: /var/log/syslog
SLES: /var/log/messages

You can also use the journalctl command to view startup script output.

sudo journalctl -u google-startup-scripts.service

Setting custom values in startup scripts

When you run startup scripts across instances, there might be situations in which you would like to use custom values in your startup script. For example, you might want to run a startup script on different instances and have each instance print out a custom message.

You can specify these custom values as metadata key-value pairs during instance creation time, and reference them in your startup scripts. For more information about creating custom metadata keys, see Setting custom metadata.

After you have set a custom metadata key pair, you can modify your startup script to query for the new metadata. For example, the same script above can be modified as follows:

#! /bin/bash
VALUE_OF_FOO=$(curl http://metadata.google.internal/computeMetadata/v1/instance/attributes/foo -H "Metadata-Flavor: Google")
apt update
apt -y install apache2
cat <<EOF > /var/www/html/index.html
<html><body><h1>Hello World</h1>
<p>The value of foo: $VALUE_OF_FOO</p>
</body></html>
EOF

Troubleshooting

Network connectivity to the metadata server

Specific to Linux VMs, Compute Engine waits for a connection to the metadata server before attempting to get information from the metadata server such as a custom startup or shutdown script. If the metadata server is not responding or the network is not yet configured, the VM doesn't finish booting up.
Specific to Linux VMs with images earlier than v20160606, the serial port output shows "Waiting for metadata server, attempt N" where N is the number of attempts that have been made.

This issue can last for up to seven minutes due to a transient network issue that resolves on its own. If the issue does not resolve itself after seven minutes, recreate your VM instance.

What's next

Add a shutdown script.
Use the interactive serial console to troubleshoot instances.