This page describes how to enable interactive access to an instance's serial console to debug boot and networking issues, troubleshoot malfunctioning instances, interact with the GRand Unified Bootloader (GRUB), and perform other troubleshooting tasks.
A virtual machine instance has four virtual serial ports. Interacting with a serial port is similar to using a terminal window, in that input and output is entirely in text mode and there is no graphical interface or mouse support. The instance's operating system, BIOS, and other system-level entities often write output to the serial ports, and can accept input such as commands or answers to prompts. Typically, these system-level entities use the first serial port (port 1) and serial port 1 is often referred to as the serial console.
By default, you can call the
method to read information that your instance has written to its serial ports,
but you cannot write information for your instance to read. However, if you
run into problems accessing your instance through SSH or need to troubleshoot
an instance that is not fully booted, you can enable interactive access to the
serial console, which lets you connect to and interact with any of
your instance's serial ports. For example, you can directly run commands
and respond to prompts in the serial port.
Before you begin
- Read the VM instances documentation.
- If you want to use the command-line examples in this guide:
Enabling interactive access on the serial console
To enable interactive access to an instance's serial console, you must apply a special metadata key/value pair. You can apply this metadata on specific instances or apply the metadata to the project, which enables interactive access to the serial console of all instances in that project.
Apply this special metadata to the project or instance to enable interactive access:
For example, using
gcloud, you can apply this metadata to a specific
instance like so:
gcloud compute instances add-metadata [INSTANCE_NAME] \ --metadata=serial-port-enable=1
To apply the metadata to the project:
gcloud compute project-info add-metadata --metadata=serial-port-enable=1
If you apply the metadata to the project, you can still disable it for
a specific instance by setting
serial-port-enable=0 in the metadata of the
particular instance. This will override the project metadata.
Connecting to a serial console
After enabling interactive access for an instance's serial console, you can
connect to the serial console using the Google Cloud Platform Console,
gcloud, or a
third-party SSH client.
The serial console authenticates users with
SSH keys. Specifically, you must add your
public SSH key to the project or instance metadata, and store your private key
on the local machine from which you want to connect.
gcloud and the
Google Cloud Platform Console will automatically add SSH keys to the project for you.
If you are using a third-party client, you might need to add SSH keys manually.
- Go to the VM instances page.
- Click the instance you want to connect to.
- Scroll to the bottom of the page and look for the Serial port section.
- If you want to connect to a serial port other than the default serial port 1, click the down arrow next to the Connect to serial port button and change the port number accordingly.
- Click the Connect to serial port button to connect to port 1 by default. For Windows instances, pull down the dropdown menu next to the button and connect to Port 2 to access the serial console.
gcloud compute connect-to-serial-port subcommand to connect
gcloud. For example:
gcloud compute connect-to-serial-port [INSTANCE_NAME]
[INSTANCE_NAME] is the name of the instance for which you want to access
the serial console.
By default, the
connect-to-serial-port command connects to port 1 of the
serial console. If you are connecting to a Windows VM instance, connect
to port 2 instead:
gcloud compute connect-to-serial-port [INSTANCE_NAME] --port 2
To connect to any other port, provide a different port number using the
--port flag. You can provide a port number from 1 through 4, inclusively.
To learn more about port numbers, see
Understanding serial port numbering.
Other SSH clients
You can connect to an instance's serial console using other third-party SSH clients, as long as the client allows you to connect to TCP port 9600.
For example, the following SSH command connects to the default serial port
(1) of an instance named
example-instance with the username
jane in a
project with the project ID
myproject. The instance is in zone
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 \ email@example.com
In detail, you can connect to the serial console of an instance using the following login and address information:
[PROJECT_ID]is the project ID for this instance.
[ZONE]is zone of the instance.
[INSTANCE_NAME]is the name of the instance.
[USERNAME]is the username you are using to connect to your instance. Typically, this is the username on your local machine.
[OPTIONS]are additional options you can specify for this connection. For example, you can specify a certain serial port and specify any of the advanced options below. The port number can be 1 through 4, inclusively. To learn more about port numbers, see Understanding serial port numbering. If omitted, you will connect to serial port 1.
If you are connecting to a Windows VM instance, connect through port 2 using the following command:
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 \ [PROJECT_ID].[ZONE].[INSTANCE_NAME].[USERNAME].firstname.lastname@example.org
If you are having trouble connecting using a third-party SSH client, you can
gcloud compute connect-to-serial-port with the
command-line option to see the SSH command that it would have run on your
behalf, and compare the options with the command you are using.
Setting up a secure connection
When you use a third-party SSH client that is not
gcloud, you can ensure that
you're protected against impersonation or man-in-the-middle attacks by checking
Google's Serial Port server SSH key. Follow these instructions to set up your
system to check the server SSH key:
- Download Google's Serial Port server SSH key.
- Open your known hosts file, generally located at
Add the contents of the server SSH key, with
ssh-serialport.googleapis.comprepended to the key. For example, if the server key contains the line
ssh-rsa AAAAB3NzaC1yc..., then
~/.ssh/known_hostsshould have a line:
ssh-serialport.googleapis.com ssh-rsa AAAAB3NzaC1yc...
For security reasons, Google may occasionally change the Google Serial Port server SSH key. If your client fails to authenticate the server key, immediately abort the connection attempt and follow the directions above to download a new Google Serial Port server SSH key.
If, after updating the host key, you continue to receive a host authentication error from your client, stop attempts to connect to the serial port and contact Google support. Do not provide any credentials over a connection where host authentication has failed.
Disconnecting from the serial console
To disconnect from the serial console:
- Press the
~.(tilde, followed by a period).
You can discover other commands by typing
~? or by examining the man page
Do not try to disconnect using any of the following methods:
CTRL+ALT+DELETEkey combination or other similar combinations. This will not work because the serial console does not recognize PC keyboard combinations.
logoutcommand does not work because the guest is not aware of any network or modem connections. Using this command causes the console to close and then reopen again, and you remain connected to the session. If you would like to enable
logoutcommands for your session, you can enable it by setting the
Connecting to a serial console with a login prompt
If you are trying to troubleshoot an issue with an instance that has booted completely, or trying to troubleshoot an issue that occurs after the instance has booted past single user mode, you might be prompted for login information when trying to acccess the serial console.
By default, Google-supplied system images are not configured to allow password-based logins for local users. If your instance is running an image that is preconfigured with serial port logins, you need to set up a local password on the virtual machine instance so you can login to the serial console, if prompted.
Setting up a local password
The following instructions describe how to set up a local password for a user on a virtual machine instance so that the user can log on to the serial console of that instance using the specified password.
Connect to the instance:
gcloud compute ssh [INSTANCE_NAME]
On the instance, create a local password with the following command. This sets a password for the user that you are currently logged in as.
sudo passwd `whoami`
Follow the prompts to create a password.
- Next, log out of the instance and connect to the serial console.
- Enter in your login information when prompted.
Setting up a login on other serial ports
Login prompts are enabled on port 1 by default on most Linux operating systems. However, port 1 can often be overwhelmed by logging data and other information being printed to the port. Alternatively, you can choose to enable a login prompt on another port instead, such as port 2 (ttyS1), by executing one of the following commands on your instance. You can see a list of available ports for an instance in Understanding serial port numbering.
The following table lists images preconfigured with a serial console login and the default ports.
|Operating System||Port(s) with a login prompt by default||Service Management|
To enable login prompts on additional serial ports, use the following instructions.
For Linux operating systems using
Enable the service temporarily till next reboot:
sudo systemctl start serial-getty@ttyS1.service
Enable the service permanently, starting with the next reboot:
sudo systemctl enable serial-getty@ttyS1.service
For Linux operating systems using
Create a new
/etc/init/ttyS1.conffile by copying and modifying an existing
ttyS0.conffile to reflect
ttyS1. For example:
On Ubuntu 14.04:
sudo sh -c "sed -e s/ttyS0/ttyS1/g < /etc/init/ttyS0.conf > /etc/init/ttyS1.conf"
On RHEL 6.8 and CentOS 6.8
sudo sh -c "sed -ne '/^# # ttyS0/,/^# exec/p' < /etc/init/serial.conf | sed -e 's/ttyS0/ttyS1/g' -e 's/^# *//' > /etc/init/ttyS1.conf"
Start on a login prompt on
sudo start ttyS1
For Linux operating systems using
sysvinit, run the following commands:
sudo sed -i~ -e 's/^#T\(\)/T\1/' /etc/inittab sudo telinit q
Understanding serial port numbering
Each virtual machine instance has four serial ports. For consistency with the
API, each port is numbered 1 through 4. Linux and other similar systems number
their serial ports 0 through 3. For example, on many operating system images, the
corresponding devices are
/dev/ttyS3. Windows refers to
serial ports as
COM4. To connect to what Windows considers
COM3 and Linux considers
ttyS2, you would specify port 3. Use
the table below to help you figure out which port you want to connect to.
|Virtual Machine Instance Serial Ports||Standard Linux Serial Ports||Windows COM Ports|
Note that many Linux images use port 1 (
/dev/ttyS0) for logging messages from
the kernel and system programs.
Sending a serial break
The Magic SysRq key feature allows you to perform low-level tasks regardless of the system's state. For example, you can sync filestems, reboot the instance, kill processes, unmount filesystems and so on, using the Magic SysRq key feature.
To send a Magic SysRq command using a simulated serial break:
- Press the
~B(tilde, followed by uppercase
- Type the desired Magic SysRq command.
Viewing serial console logs
Compute Engine provides audit logs to track who has connected and disconnected from an instance's serial console. To view logs, you must have permissions for the Logs Viewer or be a project viewer or editor.
- Go to the Logs page in the Cloud Platform Console.
- Expand the drop-down menu and select
GCE VM Instance.
- In the search bar, type
ssh-serialport.googleapis.comand hit Enter.
A list of audit logs describing connection and disconnections from a serial console appears. Expand any of the entries to get more information:
For any of the audit logs, you can:
- Expand the
- Look for
methodNameto see activity this log applies to (either a connection or disconnection request). For example, if this log tracks a disconnection from the serial console, the method name would say
"google.ssh-serialport.v1.disconnect". Similarly, a connection log would say
"google.ssh-serialport.v1.connect". An audit log entry is recorded at the beginning and end of each session on the serial console.
There are different audit log properties for different log types. For example, audit logs relating to connections will have some properties that are specific to connection logs, while audit logs for disconnections will have their own set of properties. There are certain audit log properties that are also shared between both log types.
All serial console logs
||The IP address and port number from which the connection originated.|
||A string containing the project ID, zone, instance name, and
serial port number to indicate which serial console this pertains to.
|Properties identifying the instance ID, zone, and project ID.|
||A timestamp indicating when the session began or ended.|
||A ID string uniquely identifying the session; you can use this to associate a Disconnect entry with the corresponding Connection entry.|
||Any options that were specified with the request, including the serial port number.|
||The username specified for this request. This is used to select the public key to match.|
||For successful connection requests, a
||The amount of time the session lasted, in seconds.|
Failed connection logs
When a connection fails, Compute Engine creates an audit log entry. A failed connection log looks very similar to a successful connection entry, but has the following properties to indicate a failed connection.
The canonical Google API error code that best describes the error. The following are possible error codes that might appear:
||The human-readable message for this entry.|
Tips and tricks
If you are having trouble connecting using a standard SSH client, but
gcloud compute connect-to-serial-portconnects successfully, it might be helpful to run
gcloud compute connect-to-serial-portwith the
--dry-runcommand-line option to see the SSH command that it would have run on your behalf, and compare the options with the command you are using.
Setting the bit rate, also known as baud rate: You can set any bit rate you like, such as
stty 9600, but the feature normally forces the effective rate to 115200 bps (~11.5kB/sec). This is because many OS images default to slow bit rates such as 9600 on the serial console, and would boot slowly.
Some OS images have inconvenient defaults on the serial port. For instance, on CentOS 7
stty icrnlis required to tell the console to do the right thing with the Enter key (which sends a
^M). The bash shell might mask this until you try to set a password, and then wonder why it seems stuck at the
Some OS images have job control keys that are disabled by default if you attach a shell to a port in certain ways. Some examples of these keys include
setsidcommand may fix this. Otherwise, if you see a
job control is disabled in this shellmessage, be careful not to run commands that you will need to interrupt.
You might find it helpful to tell the system the size of the window you’re using, so that bash and editors can manage it properly. Otherwise, you might experience odd display behavior as bash or editors attempt to manipulate the display based on incorrect assumptions about the number of rows and columns available. Use the
stty rows Y cols Xcommand and
stty -ato see what the setting is. For example:
stty rows 60 cols 120(if your window is 120 chars by 60 lines).
If you connect using SSH from machine A to machine B, and then to machine C (and so on), creating a nested SSH session, and you want to use
~commands such as to disconnect, or send a serial break signal, you will need to add enough extra
~characters to the command to get to the right SSH client. A command following a single
~will be interpreted by the SSH client on machine A; two consecutive
ENTER~~) will be interpreted by the client on machine B, and so forth. You only need to press
ENTERonce, because that is passed all the way through to the innermost SSH destination. This is true for any use of SSH clients which provide the
If you lose track of how many
~characters you need, press the
ENTERkey and then type
~characters one at a time until the instance echoes the
~back. This indicates that you have reached the end of the chain and you now know that to send a
~command to the most nested SSH client, you need one less
~than the number you typed.
Controlling max connections
You can set the
max-connections property to control how many concurrent
connections can be made to this serial port at a time. The default and
maximum number of connections is 5. For example:
gcloud compute connect-to-serial-port [INSTANCE_NAME] --port [PORT_NUMBER] --extra-args max-connections=3
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 [PROJECT_ID].[ZONE].[INSTANCE_NAME].[USERNAME].email@example.com
Setting replay options
By default, each time you connect to the serial console, you will receive a replay of the last 10 lines of data, regardless of whether the last 10 lines have been seen by another SSH client. You can change this setting and control how many and which lines are returned by providing the following options:
Nto the number of lines you want replayed. For example, if
Nwas 50, then the last 50 lines of the console output is included.
replay-bytes=N: Replays the most recent
Nbytes. You can also set
newwhich replays all output that has not yet been sent to any client.
replay-from=N: Replays output starting from an absolute byte index that you provide. You can get the current byte index of serial console output by making a
getSerialPortOutputrequest. If you set
replay-from, all other replay options are ignored.
gcloud, append the following to your
N is the specified number of lines (or bytes or absolute byte index,
depending on which replay option you are selecting):
If you are using a third-party SSH client, provide this option in your SSH command:
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 \ myproject.us-central1-f.example-instance.jane.port=3.replay-lines=N@ssh-serialport.googleapis.com
You can also use a combination of these options as well. For example:
Replay the specified number of lines OR replay all output not previously sent to any client, whichever is larger. The first client to connect with this flag combination will see all the output that has been sent to the serial port, and clients that connect subsquently will only see the last
gcloud compute connect-to-serial-port [INSTANCE_NAME] --port [PORT_NUMBER] --extra-args replay-lines=N,replay-bytes=new
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 [PROJECT_ID].[ZONE].[INSTANCE_NAME].[USERNAME].replay-lines=N.firstname.lastname@example.org
Replay lines up to, but not more than, the number of lines or bytes described by these flags, whichever is less. This option will not replay more than
gcloud compute connect-to-serial-port [INSTANCE_NAME] --port [PORT_NUMBER] --extra-args replay-lines=N,replay-bytes=M
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 [PROJECT_ID].[ZONE].[INSTANCE_NAME].[USERNAME].replay-lines=N.replay-bytes=M@ssh-serialport.googleapis.com
Handling dropped output
The most recent 1 MiB of output for each serial port is always available and
generally, your SSH client should not miss any output from the serial port.
If, for some reason, your SSH client stops accepting output for a period of
time but does not disconnect, and more than 1 MiB of new data is produced,
your SSH client might miss some output. In these scenarios, when your SSH
client is not accepting data fast enough to keep up with the output on the
serial console port, you can set the
on-dropped-output property to determine
how the console behaves.
Set any of the following applicable options with this property:
insert-stderr-note: Insert a note on the SSH client's
stderrindicating that output was dropped. This is the default option.
ignore: Silently drops output and does nothing.
disconnect: Terminate the connnection.
gcloud compute connect-to-serial-port [INSTANCE_NAME] --port [PORT_NUMBER] --extra-args on-dropped-output=ignore
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 [PROJECT_ID].[ZONE].[INSTANCE_NAME].[USERNAME].email@example.com
Enabling disconnect using exit or logout commands
You can enable disconnecting on exit or logout commands by setting the
on-dtr-low property to
disconnect when you connect to the serial console.
gcloud, append the following to your
If you are using a third-party SSH client, provide this option in your SSH command:
ssh -i [PRIVATE_SSH_KEY_FILE] -p 9600 \ firstname.lastname@example.org
Enabling this option might cause your instance to disconnect one or more times when you are rebooting the instance, as the operating system resets the serial ports while booting up.
The default setting for this option is
none, where nothing happens when the
DTR line changes. If you change this to none, you can reboot your instance
without being disconnected from the serial console but the console will not
disconnect through normal means such as
logout commands, or normal
key combinations like Ctrl+d.