Ensuring scale and compliance of your Terraform deployment with Cloud Build

Terraform is an open source Infrastructure as Code tool that is popular with platform developers building reusable cloud automation. The Terraform Provider for Google Cloud Platform continues to add support for the latest Google Cloud features, such as Anthos on GKE, and our teams continue to expand Terraform integrations including Cloud Foundation Toolkit and Terraform Validator.

How do teams use Terraform on Google Cloud? While the simplest approach is to run terraform init, plan and apply directly from your terminal, it cannot be recommended for automating your production deployments. First, there is a decision on how to store your Terraform state in a way that is secure, compliant and enables team collaboration. Secondly there’s a question of scale and reliability. Over the course of even the simplest cloud deployment, Terraform can end up making thousands of Create/Read/Update/Delete API calls to the endpoints used by the Terraform providers, some of which will inevitably hit quota issues or need to be retried for other reasons. For platform administrators, who are looking to ensure the best deployment practices for their curated Terraform solutions, while benefiting from the simplicity of Google Cloud Console, there’s Terraform Private Catalog integration that we enabled earlier this year.

Outside of Private Catalog, Cloud Build and Cloud Storage have been the recommended approach to use Terraform on Google Cloud. Using a remote backend prevents race conditions and simplifies sharing reusable modules between different configurations. With Cloud Build you can configure a GitOps CI/CD pipeline to automatically plan and apply your Terraform configuration when changes are pushed into the repo. 

These are widely popularized benefits explored in Managing infrastructure as code with Terraform, Cloud Build, and GitOps. In addition, there are lesser known advantages of Cloud Build, particularly for enterprise customers: Cloud Build’s concurrency capabilities and VPC-SC support, Cloud Storage versioning, security and compliance. Let’s explore these benefits in more detail.

Cloud Build’s ability to scale makes it capable to process multiple Terraform deployments across the regions globally and simultaneously. By default, Cloud Build supports 30 concurrent builds, with additional builds queued and processed after the running builds complete. In some cases it may not be enough. Customers who initiate parallel deployments to multiple zones, or those who provision infrastructure on behalf of multiple tenants, often require running more concurrent deployments to complete all of them within the allotted deployment window. Cloud Build private pool feature allows up to 100 concurrent builds, which may be further adjusted upon request. This is an example of creating a private pool and then using it when submitting a build:

A full step-by-step example of creating a private pool and submitting 80+ Terraform deployments with Cloud Build simultaneously is available here.

Using Cloud Build removes the need to build a custom high-scale Terraform provisioning service and provides observability and diagnostics for each of the build instances launched and their results. 

Using Cloud Build with private pools enables recommended security features, such as VPC Service Controls that allows setting a secure perimeter to protect against data exfiltration, with additional restrictions to further restrict it to using the specified private pools. This makes it unnecessary to configure a dedicated bastion host inside the perimeter, which improves the overall security posture.

Beyond just using Cloud Storage for remote storage, additional reasons to use Cloud Storage include versioning, security and compliance. Enabling versioning protects against state file corruption and allows you to view earlier versions. Versioning can be enabled with gsutil command:

In addition to versioning, you can use Customer-Supplied Encryption Keys to encrypt the Terraform state file. After you generated the key you can specify it as encryption_key parameter of your backend object:

Once encrypted you can still view the contents of your state by adding encryption_key option to boto configuration file. 

Finally, Cloud Storage is one of the Google Cloud services covered by FedRAMP High, which is important for enterprises that are seeking their own FedRAMP on top of Google Cloud (for more details see Compliance resource center).

To summarize, using Cloud Build and Cloud Storage for your Terraform deployments enables high scalability, security and compliance with simpler configuration and via the familiar gcloud and Google Cloud console interface. Please check out this sample for step-by-step guidance.