Store Terraform state in a Cloud Storage bucket

Stay organized with collections Save and categorize content based on your preferences.

In this tutorial, you learn how to store Terraform state in a Cloud Storage bucket.

By default, Terraform stores state locally in a file named terraform.tfstate. This default configuration can make Terraform usage difficult for teams when multiple users run Terraform at the same time and each machine has its own understanding of the current infrastructure.

To help you avoid such issues, this page shows you how to configure a remote state that points to a Cloud Storage bucket. Remote state is a feature of Terraform backends.

Costs

Cloud Storage incurs costs for storage, read and write operations, network egress, and replication.

The Cloud Storage bucket in this tutorial has Object Versioning enabled to keep the history of your deployments. Enabling Object Versioning increases storage costs, which you can mitigate by configuring Object Lifecycle Management to delete old state versions.

Before you begin

  1. Make sure you have the necessary Cloud Storage permissions on your user account:
    • storage.buckets.create
    • storage.buckets.list
    • storage.objects.get
    • storage.objects.create
    • storage.objects.delete
    • storage.objects.update

    Go to the IAM page

    Learn more about roles and permissions.

    As a best practice, we recommend that access to the bucket and the state files stored there is controlled. Only a small set of users (for example, the main cloud administrator and the person acting as the alternative or backup administrator) should have admin permissions for the bucket. The other developers should have permissions to only write and read objects in the bucket.

  2. Start Cloud Shell.

    Cloud Shell is a Compute Engine virtual machine. The service credentials associated with this virtual machine are automatic, so there is no need to set up or download a service account key.

  3. Enable the Cloud Storage API:

    gcloud services enable storage.googleapis.com
    

Configure Terraform to store state in a Cloud Storage bucket

In the following steps, you create a Cloud Storage bucket and change the backend configuration to your new bucket and your Google Cloud project.

Create the bucket

  1. Add the following google_storage_bucket Terraform resource to a Terraform config file, such as main.tf.

    resource "random_id" "bucket_prefix" {
      byte_length = 8
    }
    
    resource "google_storage_bucket" "default" {
      name          = "${random_id.bucket_prefix.hex}-bucket-tfstate"
      force_destroy = false
      location      = "US"
      storage_class = "STANDARD"
      versioning {
        enabled = true
      }
    }

    In the code snippet, the location field is hard-coded to US (which means a multi-region bucket in the US is created). You can change this field to a location of your choice.

  2. Run terraform apply to create the storage bucket.

If the apply fails, it might be because the bucket name isn't globally unique. The next section shows you how to resolve this issue.

Make the bucket name unique

To make the bucket name globally unique, you can include a Terraform plugin for creating random IDs.

// Terraform plugin for creating random IDs
resource "random_id" "instance_id" {
  byte_length = 8
}

Then you use the random ID in the bucket name attribute.

resource "google_storage_bucket" "default" {
  name = "bucket-tfstate-${random_id.instance_id.hex}"
  ...
}

Retry the terraform apply command.

Change the backend configuration

  1. Add the following text to a new Terraform configuration file called backend.tf.

    terraform {
     backend "gcs" {
       bucket  = "BUCKET_NAME"
       prefix  = "terraform/state"
     }
    }
    

    Make sure to update the BUCKET_NAME to match the name of your new Cloud Storage bucket.

  2. Run terraform init to configure your Terraform backend.

    Terraform detects that you already have a state file locally and prompts you to copy it to the new Cloud Storage bucket. Enter yes.

After running this command, your Terraform state is stored in the Cloud Storage bucket. Terraform pulls the latest state from this bucket before running a command, and pushes the latest state to the bucket after running a command.

Next steps