Google Cloud Platform

How to dynamically generate GCP IAM credentials with a new HashiCorp Vault secrets engine

Applications often require secrets such as credentials at build- or run-time. These credentials are an assertion of a service’s or user’s identity that they can use to authenticate to other services. On Google Cloud Platform (GCP), you can manage services or temporary users using Cloud Identity and Access Management (IAM) service accounts, which are identities whose credentials your application code can use to access other GCP services. You can access a service account from code running on GCP, in your on-premises environment, or even another cloud.

Protecting service account keys is critical—you should tightly control access to them, rotate them, and make sure they're not committed in code. But managing these credentials shouldn’t get harder as your number of services increases. HashiCorp Vault is a popular open source tool for secret management that allows users to store, manage and control access to tokens, passwords, certificates, API keys and many other secrets. Vault supports pluggable mechanisms known as secrets engines for managing different secret types. These engines allow developers to store, rotate and dynamically generate many kinds of secrets in Vault. Because Vault's secrets engines are pluggable, they each provide different functionality. Some engines store static data, others manage PKI and certificates, and others manage database credentials.

Today, we're pleased to announce a Google Cloud Platform IAM secrets engine for HashiCorp Vault. This allows a user to dynamically generate IAM service account credentials with which a human, machine or application can access specific Google Cloud resources using their own identity. To limit the impact of a security incident, Vault allows credentials to be easily revoked.

This helps address some common use cases, for example:

  • Restricting application access for recurring jobs: An application runs a monthly batch job. Instead of a hard-coded, long-lived credential, the application can request a short-lived GCP IAM credential at the start of the job. After a short time, the credential automatically expires, reducing the surface area for a potential attack.
  • Restricting user access for temporary users: A contracting firm needs 90 days of read-only access to build dashboards. Instead of someone generating this credential and distributing it to the firm, the firm requests this credential through Vault. This creates a 1-1 mapping of the credential to its users in audit and access logs.

Getting started with the IAM service account secret engine

Let’s go through an example for generating new service account credentials using Vault. This example assumes the Vault server is already up and running.

You can also watch a demo of the backend in our new video below.

First, enable the secrets engine:

  $ vault secrets enable gcp

Then, set up the engine with initial config and role sets:

  $ vault write gcp/config \
credentials=@path/to/creds.json \
 ttl=3600 \
 max_ttl=86400

This config supplies default credentials that Vault will use to generate the service account keys and access tokens, as well as TTL metadata for the leases Vault assigns to these secrets when generated.

Role sets define the sets of IAM roles, bound to specific resources, that you assign to generated credentials. Each role set can generate one of two types of secrets: either `access_token` for one-use OAuth access tokens or `service_account_key` for long-lived service account keys. Here are some examples for both types of rolesets:

  # Create role sets
$ vault write gcp/roleset/token-role-set \
    project="myproject" \
    secret_type="access_token" \
    bindings=@token_bindings.hcl
    token_scopes="https://www.googleapis.com/auth/cloud-platform"
$ vault write gcp/roleset/key-role-set \
    project="myproject" \
    secret_type="service_account_key"
    bindings=””

The above bindings param expects a string (or, using the special Vault syntax ‘@’, a path to a file containing this string) with the following HCL (or JSON) format

  resource "path/to/my/resource" {
    roles = [
      "roles/viewer",
      "roles/my-other-role",
    ]
}
resource "path/to/another/resource" {
    roles = [
      "roles/editor",
    ]
}

Creating a new role set generates a new service account for a role set. When a user generates a set of credentials, they specify a role set (and thus service account) under which to create the credentials.

Once you have set up the secrets engine, a Vault client can easily generate new secrets:

  $ vault read gcp/key/key-role-set
Key                 Value
---                 -----
lease_id            gcp/key/key-roleset/
lease_duration      1h
lease_renewable     true
key_algorithm       KEY_ALG_RSA_2048
key_type            TYPE_GOOGLE_CREDENTIALS_FILE
private_key_data    
$ vault read gcp/token/token-role-set
Key                 Value
---                 -----
lease_id           gcp/token/test/
lease_duration     59m59s
lease_renewable    false
token              ya29.c.restoftoken...

These credentials can then be used to make calls to GCP APIs as needed and can be automatically revoked by Vault.

To learn more, check out the GCP IAM service account secret engine documentation.

How WePay uses HashiCorp Vault on GCP

WePay is an online payment service provider who uses HashiCorp Vault on GCP. It currently runs HashiCorp Vault servers as virtual machines on Google Compute Engine for two primary use cases:
  • Plain vanilla secret storage using a configuration service: WePay has a service-oriented architecture built on Google Kubernetes Engine. Each microservice stores secrets such as passwords, tokens, private keys, and certificates in a centralized configuration service. This in turn uses the generic "kv" (key value) HashiCorp Vault secrets engine to manage application secrets. The configuration service authenticates services that talk to it, and authorizes those services to access their secrets at deployment time. Secrets are segmented by service using base paths, i.e., superSecurePaymentSystem would only be able to access secrets in the superSecurePaymentSystem path.
  • Key management using a key management service: WePay needs a way to centrally manage the provisioning, deprovisioning and rotation of encryption keys used in its applications. A central key management service generates encryption keys, and stores these in HashiCorp Vault using the "kv" secret engine.
WePay has its complete infrastructure built in GCP, and the introduction of the GCP IAM service account secrets engine will help to put in stronger security practices. WePay is exploring options on how to use the GCP IAM service account secrets engine in its infrastructure, and is excited by the possibilities.

Continuing work for HashiCorp Vault on GCP

We're excited to see what amazing applications and services users will build using the new HashiCorp Vault GCP secrets engine. This feature release is part of our long-standing ongoing partnership with HashiCorp since 2013. We're excited to continue working together to help HashiCorp users make the best use of GCP services and features. To get you up and running with, HashiCorp Vault for IAM service accounts, check out our solution brief “Using Vault on Compute Engine for Secret Management” for an overview of best practices, and a new video on authentication options.

As always, both HashiCorp and Google welcome contributions from the open-source community. Give us a tweet or open an issue on GitHub if you have any questions!