Terraform Pub/Sub Tutorial (2nd gen)

---

Note: This content applies only to Cloud Functions (2nd gen). See Cloud Functions version comparison for more information.

This tutorial demonstrates how to deploy a Pub/Sub function by uploading a function source code zip file to a Cloud Storage bucket, using Terraform to provision the resources. Terraform is an open source tool that lets you provision Google Cloud resources with declarative configuration files

This tutorial uses a Node.js function as an example, but it also works with Python, Go, and Java functions. The instructions are the same regardless of which of these runtimes you are using.

Objectives

• Learn how to use Terraform to deploy a Pub/Sub function.

Costs

In this document, you use the following billable components of Google Cloud:

• Cloud Functions
• Cloud Build
• Cloud Storage
• Artifact Registry

For details, see Cloud Functions pricing.

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Make sure that billing is enabled for your Google Cloud project.

4. Enable the Cloud Functions, Cloud Build, Artifact Registry, and Cloud Storage APIs.

   Enable the APIs

5. Install the Google Cloud CLI.

6. To initialize the gcloud CLI, run the following command:

   gcloud init

7. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

8. Make sure that billing is enabled for your Google Cloud project.

9. Enable the Cloud Functions, Cloud Build, Artifact Registry, and Cloud Storage APIs.

   Enable the APIs

10. Install the Google Cloud CLI.

11. To initialize the gcloud CLI, run the following command:

    gcloud init

If you already have the gcloud CLI installed, update it by running the following command:

gcloud components update

12. Prepare your development environment.
    

    Go to the Node.js setup guide

Setting up your environment

In this tutorial, you run commands in Cloud Shell. Cloud Shell is a shell environment with the Google Cloud CLI already installed, including the Google Cloud CLI, and with values already set for your current project. Cloud Shell can take several minutes to initialize:

Open Cloud Shell

Preparing the application

In Cloud Shell, perform the following steps:

1. Clone the sample app repository to your Cloud Shell instance:

   git clone https://github.com/GoogleCloudPlatform/nodejs-docs-samples.git

2. Change to the directory that contains the Cloud Functions sample code:

   cd nodejs-docs-samples/functions/v2/helloPubSub/

   The Node.js sample used in this tutorial is a basic "Hello World" Pub/Sub function.

3. Create a zip file containing the function source code that Terraform will upload to a Cloud Storage bucket:

   zip -r function-source.zip .
   

   Note that the root of the zip file must be the root directory of your function source code, so the above command includes the files within the helloworld directory but does not include the directory itself.

   Note: For production, it is recommended to include only necessary function source files.

Create your main.tf file

1. In the nodejs-docs-samples/functions/ directory, create a main.tf file for the Terraform configuration:

   touch main.tf
   

2. Copy this Terraform configuration into your main.tf file:

   terraform {
     required_providers {
       google = {
         source  = "hashicorp/google"
         version = ">= 4.34.0"
       }
     }
   }
   
   resource "random_id" "bucket_prefix" {
     byte_length = 8
   }
   
   
   resource "google_service_account" "default" {
     account_id   = "test-gcf-sa"
     display_name = "Test Service Account"
   }
   
   resource "google_pubsub_topic" "default" {
     name = "functions2-topic"
   }
   
   resource "google_storage_bucket" "default" {
     name                        = "${random_id.bucket_prefix.hex}-gcf-source" # Every bucket name must be globally unique
     location                    = "US"
     uniform_bucket_level_access = true
   }
   
   data "archive_file" "default" {
     type        = "zip"
     output_path = "/tmp/function-source.zip"
     source_dir  = "function-source/"
   }
   
   resource "google_storage_bucket_object" "default" {
     name   = "function-source.zip"
     bucket = google_storage_bucket.default.name
     source = data.archive_file.default.output_path # Path to the zipped function source code
   }
   
   resource "google_cloudfunctions2_function" "default" {
     name        = "function"
     location    = "us-central1"
     description = "a new function"
   
     build_config {
       runtime     = "nodejs16"
       entry_point = "helloPubSub" # Set the entry point
       environment_variables = {
         BUILD_CONFIG_TEST = "build_test"
       }
       source {
         storage_source {
           bucket = google_storage_bucket.default.name
           object = google_storage_bucket_object.default.name
         }
       }
     }
   
     service_config {
       max_instance_count = 3
       min_instance_count = 1
       available_memory   = "256M"
       timeout_seconds    = 60
       environment_variables = {
         SERVICE_CONFIG_TEST = "config_test"
       }
       ingress_settings               = "ALLOW_INTERNAL_ONLY"
       all_traffic_on_latest_revision = true
       service_account_email          = google_service_account.default.email
     }
   
     event_trigger {
       trigger_region = "us-central1"
       event_type     = "google.cloud.pubsub.topic.v1.messagePublished"
       pubsub_topic   = google_pubsub_topic.default.id
       retry_policy   = "RETRY_POLICY_RETRY"
     }
   }

3. Edit the main.tf file to make sure it has the correct values for the following items. You need to edit this file whenever your configuration changes (for example, to use a different runtime or deploy a different function):

   • Runtime: In this example, the runtime is nodejs16.
   • Function entry point: In this example, the function entry point is helloPubSub.
   • Path to zip file: In this example, if you placed your main.tf file in the nodejs-docs-samples/functions/ directory as described above, the path is ./v2/helloPubSub/function-source.zip.

Note: The main.tf file should reliably generate a unique bucket name whenever you apply the configuration. However, if you get the error "Resource already exists," you need to edit main.tf to supply a different bucket name that will be globally unique.

Initialize Terraform

1. In Cloud Shell, run the following command to initialize Terraform:

   docker run -v $(pwd):/app -w /app hashicorp/terraform:0.12.0 init
   

   You use the public Terraform Docker image. Docker is already installed in Cloud Shell. The current working directory is mounted as a volume so the Docker container can read the Terraform configuration file.

2. Run this command to add the necessary plugins and build the .terraform directory:

   terraform init
   

Validate the Terraform configuration

Preview the Terraform configuration. This step is optional, but it allows you to verify that the syntax of main.tf is correct. This command shows a preview of the resources that will be created:

terraform plan

Apply the Terraform configuration

Deploy the function by applying the configuration. When prompted, enter yes:

terraform apply

Triggering the function

To test the Pub/Sub function:

1. Publish a message to the topic (in this example, the topic name is functions2-topic):

   gcloud pubsub topics publish TOPIC_NAME --message="Friend"

2. Read the function logs to see the result, where FUNCTION_NAME is the name of your function (in this example, the function name is simply function):

   gcloud beta functions logs read FUNCTION_NAME --gen2

   You should see logging output that includes your new "Friend" message.

   Note: Logs might take a few moments to appear. If you don't see them immediately, check again in a minute or two.

Clean up

After completing the tutorial, you can delete everything that you created so that you don't incur any further costs.

Terraform lets you remove all the resources defined in the configuration file by running the terraform destroy command:

terraform destroy

Enter yes to allow Terraform to delete your resources.