Terraform on Google Cloud FAQ

This page provides answers to common questions encountered when using Terraform to manage resources on Google Cloud, particularly concerning API interactions and getting started.

Getting started with Terraform

This section covers foundational concepts and initial steps for new Terraform users.

What is Infrastructure as Code (IaC) and why should I use Terraform?

Infrastructure as Code (IaC) is the practice of managing and provisioning computing infrastructure through machine-readable definition files. For a complete overview of the concepts and benefits of IaC, see What is Infrastructure as Code?.

Terraform is an open-source IaC tool used to define, provision, and manage cloud and on-premises resources. To learn about the advantages of using Terraform for your IaC workflow, see the Benefits of using Terraform.

How do I install Terraform and run my first configuration?

To get started with Terraform, you first need to download and install the Terraform CLI on your local machine. Instructions are available on the HashiCorp Terraform website. After installation, you can create a Terraform configuration file, define a resource (like a Cloud Storage bucket), and then use terraform init to initialize your working directory, terraform plan to preview your changes, and terraform apply to apply them.

What is HashiCorp Configuration Language (HCL) and where can I learn its syntax?

HashiCorp Configuration Language (HCL) is the configuration language used by Terraform. It is designed to be human-readable and machine-friendly, for clear and efficient writing and understanding of infrastructure definitions. HCL supports various features like variables, expressions, functions, and modules. You can learn HCL syntax through the official HashiCorp Terraform documentation which provides comprehensive guides and examples.

Where can I find examples of Terraform configurations for Google Cloud resources?

You can find numerous examples of Terraform configurations for Google Cloud:

  • HashiCorp Terraform Registry: The official Terraform Registry for the Google Cloud provider contains documentation and examples for every resource and data source.

  • Google Cloud Terraform Samples: Google provides a variety of Terraform samples that demonstrate how to deploy and manage common Google Cloud resources.

  • GitHub repositories: Many open-source repositories, including the terraform-google-modules GitHub organization, offer examples and reusable modules.

How can I manage and test complex Terraform configurations, especially when dealing with many resources?

For complex configurations, consider using Terraform's features designed for scalability and maintainability:

  • Modules: Encapsulate and reuse common infrastructure patterns.

  • Workspaces: Manage multiple distinct instances of a single configuration.

  • terraform plan and terraform validate: Use these commands frequently to validate syntax and preview changes without actual deployment.

  • Targeting resources (use with caution): For testing specific parts, you can temporarily use -target with terraform apply or terraform destroy, though this is generally discouraged for routine operations due to state management complexities.

  • Dedicated environments: Deploy to development or staging environments for testing before production.

Google Cloud API questions

These questions address common inquiries regarding Terraform's interaction with Google Cloud APIs, including public and private APIs.

Can I use Terraform to manage or import internal or private Google Cloud APIs like dataproc-control.googleapis.com?

No. Internal or private Google Cloud APIs are part of Google's managed Service Infrastructure and are not exposed for direct customer management, enablement, or import using Terraform. These APIs are handled automatically by Google Cloud. Attempting to manage them directly with Terraform will result in errors.

For a comprehensive explanation, refer to the Understanding Google Cloud APIs and Terraform guide.

What's the difference between enabling an API and importing a resource in Terraform?

  • Enabling an API: This means activating a specific Google Cloud service for your project, and granting that project the necessary permissions to use that service. When using Terraform on Google Cloud, this is typically done using the google_project_service resource. This is a prerequisite for creating resources that rely on that API.

  • Importing a resource: This refers to bringing an existing Google Cloud resource (for example, a Compute Engine instance, a Cloud Storage bucket) that was created outside of Terraform under Terraform's management. You import resources, not APIs themselves.

For more details, see the Understanding Google Cloud APIs and Terraform Guide.

What if I don't explicitly manage or import dataproc-control.googleapis.com? Will it impact my ability to use Dataproc?

No, it won't impact your ability to use Dataproc. dataproc-control.googleapis.com is an internal API used by Dataproc for its own operational control. Its functionality is managed automatically by Google Cloud, and it does not require explicit enablement, import, or management by you using Terraform. Your Dataproc clusters and jobs will function correctly without any manual intervention regarding this internal API.

How do I troubleshoot 403 Permission Denied errors in Terraform?

403 Permission Denied errors typically indicate that the service account or user credentials used by Terraform lack the necessary IAM permissions to perform a requested action on a specific Google Cloud resource. To troubleshoot:

  1. Identify the affected resource and API method: The error message usually specifies the resource type and the API call that failed.

  2. Check IAM roles: Verify the principal (service account or user) has the correct IAM roles assigned at the appropriate level (project, folder, organization, or resource). Use the IAM troubleshooter in the Google Cloud console.

  3. Verify service enablement: Confirm that the required Google Cloud API service is enabled for your project (for example, using gcloud services enable or google_project_service).

  4. Review organization policies: Check if any organization policies are restricting the action.

Quota errors occur when your project attempts to consume more resources or make more API requests than allowed by its current quotas. To resolve this:

  1. Identify the specific quota: The error message will usually specify the API and the quota limit exceeded.

  2. Check current quotas: Visit the Quotas page in the Google Cloud console to view current usage and limits.

  3. Request a quota increase: If you need more capacity, you can request a quota increase directly from the Quotas page.

  4. Consider user_project_override: For some resources, if your credentials project differs from the resource project, API requests might be billed against the credentials project's quota. Using user_project_override (see Provider Reference) can sometimes resolve this by forcing the quota to be billed to the resource's project.

What is a user-managed default service account and how can I manage its permissions with Terraform?

Certain Google Cloud services automatically create user-managed service accounts (often referred to as default service accounts) when a project is created or a service is enabled. These typically have broad permissions. While they are user-managed, they are created by Google. You can manage their permissions using IAM resources like google_project_iam_member to modify their roles. If you want to take action on the default service accounts themselves, such as removing their default high-privilege roles or deleting the accounts entirely, you can use the google_project_default_service_accounts resource. Google also provides guidance on default service account types.

What is a Google-managed service account and how do I reference it in Terraform configurations?

Google-managed service accounts are created and fully managed by Google for certain services. They exist outside user projects and are not directly configurable by users in the same way user-managed service accounts are. However, you might need to grant them IAM permissions to interact with your resources. You can reference the email address of a Google-managed service account for a specific service using the google_project_service_identity data source or resource in Terraform to then apply IAM policies to it. For example, this is common for services like Cloud Build or Cloud Composer.

What happens when I terraform destroy a resource that has disable_on_destroy configured?

The disable_on_destroy argument on google_project_service and some other resources (for example, google_storage_bucket) controls whether the underlying cloud resource is disabled or deleted when the Terraform resource is destroyed.

  • If disable_on_destroy is true (or unset, as it's often the default), terraform destroy will attempt to disable (for APIs) or delete (for buckets) the corresponding cloud resource.

  • If disable_on_destroy is false, terraform destroy will remove the resource from the Terraform state but leave the actual cloud resource (e.g., the enabled API or the bucket) intact in your Google Cloud project. This is often preferred for critical services that shouldn't be accidentally disabled or deleted.