Schedule and run a cron job using Terraform

Schedule and run a cron job using Terraform

This quickstart shows you how to use Terraform to create a Cloud Scheduler cron job. Terraform is an infrastructure as code (IaC) tool that lets you predictably create, change, and improve your cloud infrastructure by using code. You can learn more about using Terraform to provision infrastructure on Google Cloud.

Cloud Scheduler has a free tier and running this quickstart shouldn't result in any costs. For more information, see Pricing.

In this quickstart, you:

1. Use Terraform to create a cron job for Cloud Scheduler.
2. Set a recurring schedule for the job.
3. Specify a Pub/Sub topic as the job target.
4. Run the job.
5. Verify that the job has run successfully.

Before you begin

1. Sign in to your Google Account.

   If you don't already have one, sign up for a new account.

2. Install the Google Cloud CLI.

3. To initialize the gcloud CLI, run the following command:

   gcloud init

4. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Google Cloud project:

     gcloud projects create PROJECT_ID

     Replace PROJECT_ID with a name for the Google Cloud project you are creating.

   • Select the Google Cloud project that you created:

     gcloud config set project PROJECT_ID

     Replace PROJECT_ID with your Google Cloud project name.

5. Install the Google Cloud CLI.

6. To initialize the gcloud CLI, run the following command:

   gcloud init

7. Create or select a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   • Create a Google Cloud project:

     gcloud projects create PROJECT_ID

     Replace PROJECT_ID with a name for the Google Cloud project you are creating.

   • Select the Google Cloud project that you created:

     gcloud config set project PROJECT_ID

     Replace PROJECT_ID with your Google Cloud project name.

8. Cloud Shell has Terraform already integrated. If you need to install Terraform, see the HashiCorp Terraform documentation.

Create a Terraform configuration file

To use Terraform with Cloud Scheduler, you need to create a configuration file to describe your infrastructure and create an execution plan. You then apply the configuration file to your platform or service to perform operations that provision your infrastructure.

Complete the following steps to create a Terraform configuration file called main.tf:

1. Open a terminal and create a directory:

   mkdir terraform

2. Go to the terraform directory:

   cd terraform

3. Add a new file, main.tf, to the directory:

   nano main.tf

4. Add the following Terraform provider for Google Cloud resources to the main.tf file:

   1. Enable the Cloud Scheduler and Pub/Sub APIs:

      # Enable Cloud Scheduler API
      resource "google_project_service" "scheduler" {
        service            = "cloudscheduler.googleapis.com"
        disable_on_destroy = false
      }
      # Enable Pub/Sub API
      resource "google_project_service" "pubsub" {
        service            = "pubsub.googleapis.com"
        disable_on_destroy = false
      }

   2. Create a Pub/Sub topic as a resource to which messages can be sent by publishers:

      # Create Pub/Sub topic
      resource "google_pubsub_topic" "default" {
        name = "pubsub_topic"
      }

      This creates a topic called pubsub_topic.

   3. Create a subscription to receive messages published to the Pub/Sub topic:

      # Create Pub/Sub subscription
      resource "google_pubsub_subscription" "default" {
        name  = "pubsub_subscription"
        topic = google_pubsub_topic.default.name
      }

   4. Create a cron job using the google_cloud_scheduler_job resource:

      # Create a cron job using Cloud Scheduler
      resource "google_cloud_scheduler_job" "default" {
        name        = "test-job"
        description = "test job"
        schedule    = "30 16 * * 7"
        region      = "us-central1"
      
        pubsub_target {
          topic_name = google_pubsub_topic.default.id
          data       = base64encode("Hello world!")
        }
      }

      The following arguments are used in the sample:

      • name: the name of the job.
      • description: a description for the job.

      • schedule: the frequency for the job, using a format based on unix-cron.

        In this sample, 30 16 * * 7 means that the job will run at 16:30 on Sundays. For more information, see Cron job format and time zone.

      • region: the region where the job resides.

      • pubsub_target: the Pub/Sub topic target, including the full resource name of the topic (topic_name) to which the message payload (data) is published when the job is run.

      Other arguments are supported. For details, see the Terraform Registry argument reference.

Create the cron job

Deploy your Terraform resources to create the cron job.

1. Open a terminal and in the terraform directory, initialize Terraform:

   terraform init

2. Check that the changes you propose with Terraform match the expected plan:

   terraform plan

   You can ignore the note regarding not using the -out option.

3. Create the cron job:

   terraform apply

4. At the Enter a value prompt, type yes to proceed with the creation of resources.

5. Confirm that a job is created:

   gcloud scheduler jobs describe test-job --location=us-central1

   The output should be similar to the following:

   description: test job
   lastAttemptTime: '2024-04-04T13:56:00.669530Z'
   name: projects/PROJECT_ID/locations/us-central1/jobs/test-job
   pubsubTarget:
   data: dGVzdA==
   topicName: projects/PROJECT_ID/topics/pubsub_topic
   schedule: '30 16 * * 7'
   scheduleTime: '2024-04-04T13:58:00.737907Z'
   state: ENABLED

You've created a job that sends a message to a Pub/Sub topic at 16:30 on Sundays. You can now run the job.

Run your job

In addition to executing according to its specified schedule, you can force your job to execute immediately by running the following command.

gcloud scheduler jobs run test-job --location=us-central1

Note that due to some initial configuration, the first job created in a project can take a few minutes to run.

Verify the results

Verify that your Pub/Sub topic is receiving messages from your job.

1. Pull Pub/Sub messages from a subscription:

   gcloud pubsub subscriptions pull pubsub_subscription --limit 5
   

   If there are no messages pulled initially, run the command again.

2. View the results of running your job. The output should look similar to the following:

   DATA: Hello world!
   MESSAGE_ID: 5028933846601543
   ORDERING_KEY:
   ATTRIBUTES:
   DELIVERY_ATTEMPT:
   ACK_ID: RFAGFixdRkhRNxkIaFEOT14jPzUgKEUQAgVPAihdeTFXLkFacGhRDRlyfWB9[...]
   

Clean up

To avoid incurring charges to your Google Cloud account for the resources used on this page, delete the Google Cloud project with the resources.

1. Deleting your Google Cloud project stops billing for all the resources used within that project. Note that by default, any files in your Cloud Shell home directory (for example, any Terraform files) persist between sessions.
   Caution: Deleting a project has the following effects:

   • Everything in the project is deleted. If you used an existing project for the tasks in this document, when you delete it, you also delete any other work you've done in the project.
   • Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.

   Delete a Google Cloud project:

   gcloud projects delete PROJECT_ID

2. Alternatively, you can delete all the resources you created with Terraform.

   The terraform destroy command terminates all the resources specified in your Terraform state. It doesn't destroy resources that aren't managed by the current Terraform project. Your Terraform configuration file is not destroyed. For more information, see Destroy infrastructure.

   terraform destroy

What's next

• What is Terraform?
• About Cloud Scheduler
• Viewing Cloud Scheduler logs