Connecting apps to instances using SSH

Your applications can use Service Accounts to run automated tasks and interact with other Google Cloud APIs. Allowing your applications to manage their own SSH keys and connect to instances can be useful for automating system management processes. This tutorial shows how to configure apps to access your instances over SSH connections. The sample app in this tutorial uses a service account and OS Login for SSH key management.

All code used in this tutorial is hosted on the GoogleCloudPlatform/python-docs-samples GitHub page.

Objectives

The tutorial covers the following tasks:

• Create a service account and configure it to provide OS Login SSH access for apps that connect to your instances.
• Create an instance that is associated with your service account.
• Configure the sample app on your instance to use the service account for managing its own SSH keys and establishing SSH connections.
• Run the app on an instance where the service account is associated.
• Run the app outside of a Compute Engine where you must provide the service account key manually and specify additional SSH parameters.

Costs

This tutorial uses billable components of Google Cloud including Compute Engine.

New Google Cloud users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

4. On your personal user account, obtain the following IAM roles for your project:
   • compute.instanceAdmin.v1
   • compute.networkAdmin
   • compute.osAdminLogin
   • iam.serviceAccountAdmin
   • iam.serviceAccountKeyAdmin
   • iam.serviceAccountUser
5. Learn how to use Cloud Shell to run gcloud command-line tool commands.

Create and configure the service account and the example instances

Create a service account and two instances to use for this tutorial. You use the service account to grant SSH access to your application, and that application will connect from one instance to the other over SSH.

Use the following steps to configure the test environment:

1. In the Google Cloud Console, open Cloud Shell.

   Open Cloud Shell

2. Export an environment variable to set your project ID for future commands:

   export PROJECT_ID='[PROJECT_ID]'
   

3. Create a new service account named ssh-account:

   gcloud iam service-accounts create ssh-account --project $PROJECT_ID \
      --display-name "ssh-account"
   

4. Create a network named ssh-example to use for this tutorial:

   gcloud compute networks create ssh-example --project $PROJECT_ID
   

5. Create a firewall rule that enables all SSH connections to instances on the ssh-example network:

   gcloud compute firewall-rules create ssh-all --project $PROJECT_ID \
      --network ssh-example --allow tcp:22
   

6. Create an instance in us-central1-f named target. This instance serves as the remote instance that your service account will connect to over SSH. Use the --metadata flag to enable OS Login on this specific instance. Include the --no-service-account and --no-scopes flags because this instance does not need to run any API requests for this specific example:

   gcloud compute instances create target --project $PROJECT_ID \
      --zone us-central1-f --network ssh-example \
      --no-service-account --no-scopes \
      --machine-type e2-micro --metadata=enable-oslogin=TRUE
   

7. Grant the compute.osAdminLogin IAM role to the service account, so it can establish SSH connections specifically to the instance named target. The compute.osAdminLogin role also grants your service account superuser privileges on the instance. Although you could grant this role at the project level so that it applies to all instances in your project, grant this role at the instance level to control SSH access in a more granular way. You can grant additional permissions to your service account later if you find that your applications require access to other resources in your project:

   gcloud compute instances add-iam-policy-binding target \
      --project $PROJECT_ID --zone us-central1-f \
      --member serviceAccount:ssh-account@$PROJECT_ID.iam.gserviceaccount.com \
      --role roles/compute.osAdminLogin
   

8. Create an instance in us-central1-f named source. Associate the instance with the ssh-account service account. Also, specify the cloud-platform scope, which is required for the service account to execute API requests on this instance:

   gcloud compute instances create source \
      --project $PROJECT_ID --zone us-central1-f \
      --service-account ssh-account@$PROJECT_ID.iam.gserviceaccount.com  \
      --scopes https://www.googleapis.com/auth/cloud-platform \
      --network ssh-example --machine-type e2-micro
   

The service account can now manage its own SSH key pairs and can use SSH to connect specifically to the target instance. Because the source instance is associated with the ssh-account service account that you created, the Cloud Client Libraries for Python can use Application Default Credentials to authenticate as the service account and use the roles that you granted to that service account earlier.

Next, configure and run an app that can SSH from one instance to another instance.

Run an SSH app on an instance

When apps running on your instances require SSH access to other instances, you can manage the SSH key pairs for your service account and execute SSH commands programmatically. For this example, run a sample app using the following process:

1. Connect to the source instance using the gcloud command-line tool:

   gcloud compute ssh source --project $PROJECT_ID --zone us-central1-f
   

2. On the source instance, install pip and the Python client library:

   my-username@source:~$ sudo apt update && sudo apt install python-pip -y && pip install --upgrade google-api-python-client
   

3. Download the service_account_ssh.py sample app from GoogleCloudPlatform/python-docs-samples:

   my-username@source:~$ curl -O https://raw.githubusercontent.com/GoogleCloudPlatform/python-docs-samples/master/compute/oslogin/service_account_ssh.py
   

4. Run the sample app, which uses argparse to accept variables from the command line. In this example, instruct the app to install and run cowsay on the target instance. For this command, add your project ID manually:

   my-username@source:~$ python service_account_ssh.py \
       --cmd 'sudo apt install cowsay -y && cowsay "It works!"' \
       --project [PROJECT_ID] --zone us-central1-f --instance target
   
   ⋮
   ___________
     It works!
   -----------
          \   ^__^
           \  (oo)\_______
              (__)\       )\/\
                  ||----w |
                  ||     ||
   
   

If the app runs correctly, you receive the output from the cowsay app. You can modify the --cmd flag to include any command that you want. Alternatively, you can write your own app that imports service_account_ssh.py and calls it directly.

Run exit to disconnect from the source instance and return to Cloud Shell.

Run an SSH app outside of Compute Engine

In the previous example, you ran the app on a Compute Engine instance where the Cloud Client Libraries for Python could use Application Default Credentials to use the service account that is associated with the source instance. If you run this app outside of a Compute Engine instance, the client library can't access the service account and its permissions unless you provide the service account key manually.

1. Obtain the external IP address for the target instance that you created earlier in this tutorial. You can find this address either in the console on the Instances page or by running the following command from the gcloud command-line tool:

   gcloud compute instances describe target \
      --project $PROJECT_ID --zone us-central1-f
   

2. Create a service account key for the ssh-account service account that you used in the previous example, and download the key file to your local workstation.

3. Copy the service account key to the system where you want to run this example.

4. Open a terminal on the system where you want to run this example.

5. Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to point to the path where your service account key .json file is located. If your key is in your Downloads folder, you might set an environment variable like the following example:

   $ export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/key.json"
   

6. Install the prerequisites on this system:

   1. Install Python and pip. On Debian-based systems, you can use apt to complete this step:

      $ sudo apt update && sudo apt install python python-pip -y
      

   2. Use pip to install the Cloud Client Libraries for Python:

      $ pip install --upgrade google-api-python-client
      

7. Download the sample app:

   $ curl -O https://raw.githubusercontent.com/GoogleCloudPlatform/python-docs-samples/master/compute/oslogin/service_account_ssh.py
   

8. Run the sample app. When you run the app outside of Compute Engine, the metadata server is not available, so you must specify the service account email manually. You must also specify the external IP address for the target instance that you obtained earlier.

   $ python service_account_ssh.py \
       --cmd 'sudo apt install cowsay -y && cowsay "It works!"' \
       --account ssh-account@[PROJECT_ID].iam.gserviceaccount.com \
       --project [PROJECT_ID] --hostname [TARGET_EXTERNAL_IP]
   
   ⋮
   ___________
     It works!
   -----------
          \   ^__^
           \  (oo)\_______
              (__)\       )\/\
                  ||----w |
                  ||     ||
   
   

If the app runs correctly, you receive the output from the cowsay app.

How the sample app works

The service_account_ssh.py sample app operates using the following process:

1. Initialize the OS Login API object.
2. If you don't provide the service account email address manually, the app reads instance metadata to identify the service account that's associated with the instance. If you run this app outside of Compute Engine, you must provide the service account address manually.
3. Call the create_ssh_key() method to generate a temporary SSH key for the service account on the instance where this example runs and add the public key to the service account with an expiration timer that you can specify.
4. Call the getLoginProfile() method from the OS Login API to get the POSIX user name that the service account uses.
5. Call the run_ssh() method to execute a remote SSH command as the service account.
6. Print the response from the remote SSH command.
7. Remove the temporary SSH key files.
8. OS Login removes the public key files automatically when they pass the expiration time.

compute/oslogin/service_account_ssh.py

View on GitHub

def main(cmd, project, instance=None, zone=None,
         oslogin=None, account=None, hostname=None):
    """Run a command on a remote system."""

    # Create the OS Login API object.
    oslogin = oslogin or googleapiclient.discovery.build('oslogin', 'v1')

    # Identify the service account ID if it is not already provided.
    account = account or requests.get(
        SERVICE_ACCOUNT_METADATA_URL, headers=HEADERS).text
    if not account.startswith('users/'):
        account = 'users/' + account

    # Create a new SSH key pair and associate it with the service account.
    private_key_file = create_ssh_key(oslogin, account)

    # Using the OS Login API, get the POSIX user name from the login profile
    # for the service account.
    profile = oslogin.users().getLoginProfile(name=account).execute()
    username = profile.get('posixAccounts')[0].get('username')

    # Create the hostname of the target instance using the instance name,
    # the zone where the instance is located, and the project that owns the
    # instance.
    hostname = hostname or '{instance}.{zone}.c.{project}.internal'.format(
        instance=instance, zone=zone, project=project)

    # Run a command on the remote instance over SSH.
    result = run_ssh(cmd, private_key_file, username, hostname)

    # Print the command line output from the remote instance.
    # Use .rstrip() rather than end='' for Python 2 compatability.
    for line in result:
        print(line.decode('utf-8').rstrip('\n\r'))

    # Shred the private key and delete the pair.
    execute(['shred', private_key_file])
    execute(['rm', private_key_file])
    execute(['rm', private_key_file + '.pub'])


if __name__ == '__main__':

    parser = argparse.ArgumentParser(
        description=__doc__,
        formatter_class=argparse.RawDescriptionHelpFormatter)
    parser.add_argument(
        '--cmd', default='uname -a',
        help='The command to run on the remote instance.')
    parser.add_argument(
        '--project',
        help='Your Google Cloud project ID.')
    parser.add_argument(
        '--zone',
        help='The zone where the target instance is located.')
    parser.add_argument(
        '--instance',
        help='The target instance for the ssh command.')
    parser.add_argument(
        '--account',
        help='The service account email.')
    parser.add_argument(
        '--hostname',
        help='The external IP address or hostname for the target instance.')
    args = parser.parse_args()

    main(args.cmd, args.project, instance=args.instance, zone=args.zone,
         account=args.account, hostname=args.hostname)

The create_ssh_key() method generates a new SSH key pair. Then, the method calls users().importSshPublicKey() from the OS Login API to associate the public key with the service account. The users().importSshPublicKey() method also accepts an expiration value, which indicates how long the public key remains valid.

compute/oslogin/service_account_ssh.py

View on GitHub

def create_ssh_key(oslogin, account, private_key_file=None, expire_time=300):
    """Generate an SSH key pair and apply it to the specified account."""
    private_key_file = private_key_file or '/tmp/key-' + str(uuid.uuid4())
    execute(['ssh-keygen', '-t', 'rsa', '-N', '', '-f', private_key_file])

    with open(private_key_file + '.pub', 'r') as original:
        public_key = original.read().strip()

    # Expiration time is in microseconds.
    expiration = int((time.time() + expire_time) * 1000000)

    body = {
        'key': public_key,
        'expirationTimeUsec': expiration,
    }
    oslogin.users().importSshPublicKey(parent=account, body=body).execute()
    return private_key_file

As a best practice, configure your service accounts to regularly generate new key pairs for themselves. In this example, the service account creates a new key pair for each SSH connection that it establishes, but you could modify this to run on a schedule that better meets the needs of your app.

The request body for users().importSshPublicKey() includes the expirationTimeUsec value, which tells OS Login when the key should expire. Each account can have only up to 32 KB of SSH key data, so it is best to configure your public SSH keys to expire shortly after your service account has completed its operations.

After your service account configures its SSH keys, it can execute remote commands. In this example, the app uses the run_ssh() method to execute a command on a remote instance and return the command output.

compute/oslogin/service_account_ssh.py

View on GitHub

def run_ssh(cmd, private_key_file, username, hostname):
    """Run a command on a remote system."""
    ssh_command = [
        'ssh', '-i', private_key_file, '-o', 'StrictHostKeyChecking=no',
        '{username}@{hostname}'.format(username=username, hostname=hostname),
        cmd,
    ]
    ssh = subprocess.Popen(
        ssh_command, shell=False, stdout=subprocess.PIPE,
        stderr=subprocess.PIPE)
    result = ssh.stdout.readlines()
    return result if result else ssh.stderr.readlines()

Cleaning up

To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, either delete the project that contains the resources, or keep the project and delete the individual resources.

1. In the Google Cloud Console, open Cloud Shell.

   Open Cloud Shell

2. Delete the instance named source:

   gcloud compute instances delete source \
      --project $PROJECT_ID --zone us-central1-f
   

3. Delete the instance named target:

   gcloud compute instances delete target \
      --project $PROJECT_ID --zone us-central1-f
   

4. Delete the ssh-account service account:

   gcloud iam service-accounts delete ssh-account --project $PROJECT_ID
   

5. Delete the network named ssh-example:

   gcloud compute networks delete ssh-example --project $PROJECT_ID
   

What's next

• Download and view the full code sample. The full sample includes a small example of using all of these methods together. Feel free to download it, change it, and run it to suit your needs.
• Review the Compute Engine API reference and OS Login API reference to learn how to perform other tasks with these APIs.
• Start creating your own apps.