Manage infrastructure with Workload Identity Federation and Terraform Cloud

Introduction

Terraform Cloud (TFC) can help manage infrastructure as code (IaC) development for large enterprises. As the number of Google Cloud projects grows, managing access controls for Terraform Cloud projects and workspaces can become complex. Don't worry, we have a solution that is designed to be more secure than using Google Cloud service account keys, and also scales well for hundreds or even thousands of Google Cloud projects, TFC workspaces, and TFC projects using Workload Identity Federation.

An enterprise scenario

Consider a fictitious financial services firm example.com that offers banking, lending, insurance and brokerage service to customers.  Their Google Cloud resource hierarchy is shown below: Business Units > Environment Folders > Application Projects.

As IaC codebases grow to manage the infrastructure for the entire organization, it can be difficult to control access for a large number of deployment pipelines. The following solution guidance addresses three key challenges:

1. How do I ensure dev IaC code only creates or deletes resources only in the dev environment, not production?

2. How do I prevent Banking IaC code from accidentally creating or deleting Brokerage business unit resources?

3. How many TFC projects, workspaces, Google Cloud Workload identity pools and service accounts for Terraform are optimal for my enterprise use cases?

Solution architecture

At a high level, Terraform Cloud workspaces integrate with Workload Identity Federation to authenticate with Google Cloud, then impersonate Google Cloud service accounts to manage resources in application projects more securely. TFC workspaces are granted permission to impersonate the right service account.

The following diagram shows the components and how they interact.

This solution requires the following setup in Terraform Cloud and Google Cloud. 

Terraform Cloud setup

1. Create one TFC project per business unit

2. Create workspaces under TFC projects, one for each environment

3. Configure TFC workspace with the right pool id for deployment pipelines to run

Google Cloud setup

In Google Cloud Workload identity pools and service accounts are set up for TFC workspaces use:

1. Setup Workload identity pools

a. Create one Workload identity pool for each environment. 

b. Create Terraform Cloud as Workload identity pool provider. 

2. Setup service accounts for managing resources

a. Create a service account for Terraform (TF SA) workspace impersonation

b. Grant TF SAs the minimum roles needed to run IaC in application projects, following the principle of least privilege

3. Grant TF SA impersonation permission to only one TFC workspace. 

3. Enable data access logs for IAM and STS (Security Token Service) for auditability and troubleshooting

Use Workload identity pool to authenticate with Google Cloud and impersonate a service account to manage Google Cloud resources. For more information on keyless authentication, you can review our blogs on enabling keyless authentication from GitHub Actions and configuring Workload Identity Federation for GitHub actions and Terraform Cloud.

Putting all the pieces together 

Here is an example of banking-prod IaC code managing resources in  prj-banking-1-prod project.

A successful run of a plan result would look like this:

Projects and Workspaces in Terraform Cloud 

IaC code in TFC is organized into Projects and Workspaces for easier management. We create four projects In our case — banking, insurance, brokerage and lending — one for each business unit.

Terraform workspace access control

To access TFC workspaces, ensure appropriate RBAC controls and approval processes are in place with naming conventions of <business unit>-<environment>, where the environment determines the Workload identity pool it gets access to. For example, banking-prod gets access to the terraform-pool-prod in Google Cloud.

Terraform workspace configuration

Here is an example configuration that connects to the Production pool provider for banking-prod workspace. This step is executed after Google Cloud setup is completed.

Workload identity pool management

A Workload identity pool is an entity in Google Cloud that allows management of external identities. Each environment should have its own Workload identity pool. Set strict IAM policies for pool projects to ensure only authorized users have access.

Following pools are created for our example.com organization use case.

The following IaC code illustrates the creation of one Identity Pool with a single TFC Provider in that pool.

The Pool provider is created with TFC OIDC assertions mapped to Google Cloud attributes. Refer to https://app.terraform.io/.well-known/openid-configuration for the claims supported by TFC ID Token. 

The diagram below shows Provider configuration created by IaC code.

Workload identity pool security 

Use attribute conditions to restrict the use of example.com Workload identity pools to the example.com TFC organization and for one specific environment by configuring the Terraform identity pool provider resource.

Deny unauthorized access to TFC Workspaces

• Unauthorized access to a workload identity pool by a workspace is denied.  
  Dev TFC Workspace configuration with production pool.

Error received when TF plan is run in TFC.

• Unauthorized access to a service account by a workspace is denied. 
  Dev TFC Workspace configuration with a production service account.

Error received when TF plan is run:

Service Account management

• Create Google Cloud service accounts for use by Terraform Cloud workspaces to manage Google Cloud resources in application projects. For our enterprise use case we create a separate service account for each TFC workspace. 

• Create these service accounts in separate Google Cloud projects with proper IAM applied to those projects.

An example that grants impersonation permission only to TFC workspace banking-dev.

• Grant minimum roles to service accounts based on app project requirements. For example, grant sa-tf-banking-prod service account  roles/storage.admin, roles/compute.admin to manage Compute and Cloud Storage services in  prj-banking-1-prod project.

IAM in prj-banking-1-prod for Service account sa-tf-banking-prod will look like this: 

Auditability 

Enable data access logs for IAM and STS in pool projects to track who is accessing resources in those projects, what resources they are accessing, and identify any unauthorized access. 

Example IAM and STS log entries:

Get started today

Refer to the following detailed documentation on concepts and best practices for using Workload Identity Federation. 

• Workload identity federation | IAM Documentation | Google Cloud

• Configure workload identity federation with deployment pipelines | IAM Documentation | Google Cloud

• Best practices for using workload identity federation | IAM Documentation | Google Cloud

• Best practices for using service accounts in pipelines | IAM Documentation | Google Cloud