HashiCorp and Google expand collaboration, easing secret and infrastructure management
Maya Kaczorowski
Security & Privacy Product Manager
Emily Ye
Software Engineer
Open source technology encourages collaboration and innovation to address real world problems, including projects supported by Google Cloud. As part of our broad engagement with the open source community, we’ve been working with HashiCorp since 2013 to enable customers who use HashiCorp tools to make optimal use of Google Cloud Platform (GCP) services and features.
A longstanding, productive collaboration
Google and HashiCorp have dedicated engineering teams focused on enhancing and expanding GCP support in HashiCorp products. We're focused on technical and shared go-to-market efforts around HashiCorp products in several critical areas of infrastructure.- Cloud provisioning: The Google Cloud provider for HashiCorp Terraform allows management of a broad array of GCP resource types, with Bigtable and BigQuery being the most recent additions. Today, HashiCorp also announced support for GCP in the Terraform Module Registry to give users easy access to templates for setting up and running their GCP-based infrastructure. We plan to continue to broaden the number of GCP services that can be provisioned with Terraform, allowing Terraform users to adopt a familiar workflow across multiple cloud and on-premises environments. Using Terraform to move workloads to GCP simplifies the cloud adoption process for Google customers that use Terraform today in cross-cloud environments.
- Cloud security and secret management: We're working to enhance the integration between HashiCorp Vault and GCP, including Vault authentication backends for IAM and signed VM metadata. This is in addition to work being done by HashiCorp for Kubernetes authentication.
Using HashiCorp Vault with Google Cloud and Kubernetes
Applications often require access to small pieces of sensitive data at build or run time, referred to as secrets. HashiCorp Vault is a popular open source tool for secret management, which allows a developer to store, manage and control access to tokens, passwords, certificates, API keys and other secrets. Vault has many options for authentication, known as authentication backends. These allow developers to use many kinds of credentials to access Vault, including tokens, or usernames and passwords.As of today, developers on Google Cloud now have two authentication backends which they can use to validate a service’s identity to their instance of Vault:
- GCP IAM service accounts: a new Google Cloud Platform IAM authentication backend for Vault allows you to use an existing IAM identity to authenticate to Vault.
- Google Compute Engine instance identity tokens: announced today, this uses an instance’s signed metadata token to authenticate to Vault.
With these authentication backends, it’s easier for a particular service running on Google Cloud to get access to a secret it needs at build or run time stored in Vault.
Fleetsmith is a secure cloud-based solution for managing a company’s Mac computers, that fully integrates with G Suite. They’ve been testing out the new Compute Engine metadata backend, and are currently using Vault on GCP for PKI and secret management. Learn more about how Fleetsmith did this in their blogpost.
Fleetsmith and Google have shared values when it comes to security, and we built our product on Google Cloud Platform in part due to Google's high bar for security. We're excited about this new integration because it strengthens the security model for us as Google Cloud customers using Vault.
— Jesse Endahl, CPO and CSO, Fleetsmith
If you’re using Vault for managing secrets in Kubernetes specifically, today HashiCorp announced a new Kubernetes authentication backend. This uses Kubernetes pod service accounts to authenticate to Vault, providing an alternative to storing secrets in directly in `etcd`.
Running HashiCorp Vault on Google Cloud
You may already be running your own instance of HashiCorp Vault. Users can run Vault in either Compute Engine or Google Container Engine, and then use one of our new authentication backends to authenticate to Vault.
WePay, an online payment service provider, uses HashiCorp Vault on GCP:
Managing usernames, passwords and certificates is a challenge in a microservice world, where we have to securely manage many secrets for hundreds of microservices. WePay chose to use HashiCorp Vault to store secrets because it provides us with rotation, tight control and out-of-the-box audit logging for our secrets and other sensitive data. WePay runs Vault server infrastructure on Google Compute Engine for secret storage, key management and service to service authentication, for use by our microservice architecture based on Google Container Engine.
— Akshath Kumar, Site Reliability Engineer, WePay
eBay also uses HashiCorp Vault on GCP:
As a strong contributor and supporter of free open source software with vital projects such as regressr and datameta, eBay is a user of Hashicorp’s software products, including vaultproject.io on the Google Cloud Platform.
— Mitch Wyle, Director of Applied Science and Engineering, eBay
Today, we’re publishing a solution on how to best set up and run HashiCorp Vault on Compute Engine. For best practices for running Vault on Compute Engine, read the solution brief “Using Vault on Compute Engine for Secret Management”.
Using HashiCorp Terraform to manage your resources on Google Cloud
When you’re testing new code or software, you might want to spin up a test environment to simulate your application. HashiCorp Terraform is an infrastructure management and deployment tool that allows you to programmatically configure infrastructure across a variety of providers, including cloud providers like Google Cloud.Using Terraform on Google Cloud, you can programmatically manage projects, IAM policies, Compute Engine resources, BigQuery datasets and more. To get started with Terraform for Google Cloud, check out the Terraform Google Cloud provider documentation, take a look at our tutorial for managing GCP projects with Terraform, which you can follow on our community page, or watch our Terraform for Google Cloud demo.
Google has released a number of Terraform modules that make working with Google Cloud even easier. These modules let you quickly compose your architectures as code and reuse architectural patterns for resources like load balancing, managed instance groups, NAT gateways and SQL databases. The modules can be found on the Terraform Module Registry.
