Use Workflows with Cloud Run and Cloud Functions tutorial

This tutorial shows you how to use Workflows to link a series of services together. By connecting two public HTTP services (using Cloud Functions), an external REST API, and a private Cloud Run service, you can create a flexible, serverless application.

Objectives

In this tutorial, you will create a single workflow, connecting one service at a time:

  1. Deploy two Cloud Functions services: the first function generates a random number, and then passes that number to the second function which multiplies it.
  2. Using Workflows, connect the two HTTP functions together. Execute the workflow and return a result that is then passed to an external API.
  3. Using Workflows, connect an external HTTP API that returns the log for a given number. Execute the workflow and return a result that is then passed to a Cloud Run service.
  4. Deploy a Cloud Run service that allows authenticated access only. The service returns the math.floor for a given number.
  5. Using Workflows, connect the Cloud Run service, execute the entire workflow, and return a final result.

The following diagram shows both an overview of the process as well as a visualization of the final workflow:

Workflows visualization

Costs

This tutorial uses the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

Some of the steps in this document might not work correctly if your organization applies constraints to your Google Cloud environment. In that case, you might not be able to complete tasks like creating public IP addresses or service account keys. If you make a request that returns an error about constraints, see how to Develop applications in a constrained Google Cloud environment.

  1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
  2. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  3. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  4. Enable the Cloud Build, Cloud Functions, Cloud Run, Cloud Storage, Container Registry, Workflows APIs.

    Enable the APIs

  5. Install and initialize the Cloud SDK.
  6. In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  7. Make sure that billing is enabled for your Cloud project. Learn how to confirm that billing is enabled for your project.

  8. Enable the Cloud Build, Cloud Functions, Cloud Run, Cloud Storage, Container Registry, Workflows APIs.

    Enable the APIs

  9. Install and initialize the Cloud SDK.
  10. Update gcloud components:
    gcloud components update
  11. Log in using your account:
    gcloud auth login
  12. Create a service account for Workflows to use:

    export SERVICE_ACCOUNT=workflows-sa
    gcloud iam service-accounts create ${SERVICE_ACCOUNT}
    

  13. To allow the service account to call authenticated Cloud Run services, grant the run.invoker role to the Workflows service account:

    gcloud projects add-iam-policy-binding ${GOOGLE_CLOUD_PROJECT} \
        --member "serviceAccount:${SERVICE_ACCOUNT}@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com" \
        --role "roles/run.invoker"
    

  14. Set the default location used in this tutorial:
    gcloud config set project PROJECT_ID
    export REGION=REGION
    gcloud config set run/region ${REGION}
    gcloud config set workflows/location ${REGION}
    

    Replace:

    • PROJECT_ID with your Google Cloud project ID
    • REGION with the supported Workflows location of your choice

Deploy the first Cloud Functions service

After receiving an HTTP request, this HTTP function generates a random number between 1 and 100, and then returns the number in JSON format.

  1. Create a directory called randomgen and change to it:

    mkdir ~/randomgen
    cd ~/randomgen
    
  2. Create a text file with the filename main.py that contains the following Python code:

    import random, json
    from flask import jsonify
    
    def randomgen(request):
        randomNum = random.randint(1,100)
        output = {"random":randomNum}
        return jsonify(output)
  3. To support a dependency on Flask for HTTP processing, create a text file for the pip package manager. Give it the filename requirements.txt and add the following:

    flask>=1.0.2
    
  4. Deploy the function with an HTTP trigger, and allow unauthenticated access:

    gcloud functions deploy randomgen \
    --runtime python37 \
    --trigger-http \
    --allow-unauthenticated
    

    The function might take a few minutes to deploy. Alternatively, you can use the Cloud Functions interface in the Cloud Console to deploy the function.

  5. Once the function is deployed, you can confirm the httpsTrigger.url property:

    gcloud functions describe randomgen
    
  6. You can try out the function with the following curl command:

    curl $(gcloud functions describe randomgen --format='value(httpsTrigger.url)')
    

    A number is randomly generated and returned.

Deploy the second Cloud Functions service

After receiving an HTTP request, this HTTP function extracts the input from the JSON body, multiplies it by 2, and returns the result in JSON format.

  1. Create a directory called multiply and change to it:

    mkdir ~/multiply
    cd ~/multiply
    
  2. Create a text file with the filename main.py that contains the following Python code:

    import random, json
    from flask import jsonify
    
    def multiply(request):
        request_json = request.get_json()
        output = {"multiplied":2*request_json['input']}
        return jsonify(output)
  3. To support a dependency on Flask for HTTP processing, create a text file for the pip package manager. Give it the filename requirements.txt and add the following:

    flask>=1.0.2
    
  4. Deploy the function with an HTTP trigger, and allow unauthenticated access:

    gcloud functions deploy multiply \
    --runtime python37 \
    --trigger-http \
    --allow-unauthenticated
    

    The function might take a few minutes to deploy.Alternatively, you can use the Cloud Functions interface in the Cloud Console to deploy the function.

  5. Once the function is deployed, you can confirm the httpsTrigger.url property:

    gcloud functions describe multiply
    
  6. You can try out the function with the following curl command:

    curl $(gcloud functions describe multiply --format='value(httpsTrigger.url)') \
    -X POST \
    -H "content-type: application/json" \
    -d '{"input": 5}'
    

    The number 10 should be returned.

Connect the two Cloud Functions services in a workflow

A workflow is made up of a series of steps described using the Workflows syntax, which can be written in either YAML or JSON format. This is the workflow's definition. For a detailed explanation, see the Syntax reference page.

  1. Navigate back to your home directory:

    cd ~
    
  2. Create a text file with the filename workflow.yaml with the following content:

    - randomgen_function:
        call: http.get
        args:
            url: https://REGION-PROJECT_ID.cloudfunctions.net/randomgen
        result: randomgen_result
    - multiply_function:
        call: http.post
        args:
            url: https://REGION-PROJECT_ID.cloudfunctions.net/multiply
            body:
                input: ${randomgen_result.body.random}
        result: multiply_result
    - return_result:
        return: ${multiply_result}
    

    This links the two HTTP functions together and returns a final result.

  3. After creating the workflow, you can deploy it, which makes it ready for execution.

    gcloud workflows deploy WORKFLOW_NAME --source=workflow.yaml
    

    Replace WORKFLOW_NAME with the name of your workflow.

  4. Execute the workflow:

    gcloud workflows run WORKFLOW_NAME
    

    An execution is a single run of the logic contained in a workflow's definition. All workflow executions are independent, and the rapid scaling of Workflows allows for a high number of concurrent executions.

    After the workflow is executed, the output should resemble the following:

    result: '{"body":{"multiplied":120},"code":200,"headers":{"Alt-Svc":"h3-29=\":443\";
    ...
    startTime: '2021-05-05T14:17:39.135251700Z'
    state: SUCCEEDED
    

Connect a public REST service in the workflow

Using the Google Cloud Console, update the existing workflow and connect a public REST API (math.js) which can evaluate mathematical expressions; for example, curl https://api.mathjs.org/v4/?'expr=log(56)'.

  1. Open the Workflows page in the Google Cloud Console:
    Go to the Workflows page

  2. Select the name of the workflow you want to update.

  3. To edit the workflow's source, click Edit, and then click Next.

  4. Replace the source code in the workflow editor, with the following content:

    - randomgen_function:
        call: http.get
        args:
            url: https://REGION-PROJECT_ID.cloudfunctions.net/randomgen
        result: randomgen_result
    - multiply_function:
        call: http.post
        args:
            url: https://REGION-PROJECT_ID.cloudfunctions.net/multiply
            body:
                input: ${randomgen_result.body.random}
        result: multiply_result
    - log_function:
        call: http.get
        args:
            url: https://api.mathjs.org/v4/
            query:
                expr: ${"log(" + string(multiply_result.body.multiplied) + ")"}
        result: log_result
    - return_result:
        return: ${log_result}
    

    This links the external REST service to the Cloud Functions services, and returns a final result.

  5. Click Deploy.

Deploy a Cloud Run service

Deploy a Cloud Run service that, after receiving an HTTP request, extracts input from the JSON body, calculates its math.floor, and returns the result.

  1. Create a directory called floor and change to it:

    mkdir ~/floor
    cd ~/floor
    
  2. Create a text file with the filename app.py that contains the following Python code:

    import json
    import logging
    import os
    import math
    
    from flask import Flask, request
    
    app = Flask(__name__)
    
    @app.route('/', methods=['POST'])
    def handle_post():
        content = json.loads(request.data)
        input = float(content['input'])
        return f"{math.floor(input)}", 200
    
    if __name__ != '__main__':
        # Redirect Flask logs to Gunicorn logs
        gunicorn_logger = logging.getLogger('gunicorn.error')
        app.logger.handlers = gunicorn_logger.handlers
        app.logger.setLevel(gunicorn_logger.level)
        app.logger.info('Service started...')
    else:
        app.run(debug=True, host='0.0.0.0', port=int(os.environ.get('PORT', 8080)))

  3. In the same directory, create a Dockerfile with the following content:

    # Use an official lightweight Python image.
    # https://hub.docker.com/_/python
    FROM python:3.7-slim
    
    # Install production dependencies.
    RUN pip install Flask gunicorn
    
    # Copy local code to the container image.
    WORKDIR /app
    COPY . .
    
    # Run the web service on container startup. Here we use the gunicorn
    # webserver, with one worker process and 8 threads.
    # For environments with multiple CPU cores, increase the number of workers
    # to be equal to the cores available.
    CMD exec gunicorn --bind 0.0.0.0:8080 --workers 1 --threads 8 app:app

  4. Build the container image:

    export SERVICE_NAME=floor
    gcloud builds submit --tag gcr.io/${GOOGLE_CLOUD_PROJECT}/${SERVICE_NAME}
    
  5. Deploy the container image to Cloud Run, ensuring that it only accepts authenticated calls:

    gcloud run deploy ${SERVICE_NAME} \
    --image gcr.io/${GOOGLE_CLOUD_PROJECT}/${SERVICE_NAME} \
    --platform managed \
    --no-allow-unauthenticated
    

When you see the service URL, the deployment is complete. You will need to specify that URL when updating the workflow definition.

Connect the Cloud Run service in the workflow

  1. Open the Workflows page in the Google Cloud Console:
    Go to the Workflows page

  2. Select the name of the workflow you want to update.

  3. To edit the workflow's source, click Edit, and then click Next.

  4. Replace the source code in the workflow editor, with the following content:

    - randomgen_function:
        call: http.get
        args:
            url: https://REGION-PROJECT_ID.cloudfunctions.net/randomgen
        result: randomgen_result
    - multiply_function:
        call: http.post
        args:
            url: https://REGION-PROJECT_ID.cloudfunctions.net/multiply
            body:
                input: ${randomgen_result.body.random}
        result: multiply_result
    - log_function:
        call: http.get
        args:
            url: https://api.mathjs.org/v4/
            query:
                expr: ${"log(" + string(multiply_result.body.multiplied) + ")"}
        result: log_result
    - floor_function:
        call: http.post
        args:
            url: CLOUD_RUN_SERVICE_URL
            auth:
                type: OIDC
            body:
                input: ${log_result.body}
        result: floor_result
    - return_result:
        return: ${floor_result}
    

    Replace CLOUD_RUN_SERVICE_URL with your Cloud Run service URL.

    This connects the Cloud Run service in the workflow. Note that the auth key ensures that an authentication token is being passed in the call to the Cloud Run service. For more information, see Workflows authentication.

  5. Click Deploy.

Execute the final workflow

  1. Update the workflow, passing in the service account:

    cd ~
    gcloud workflows deploy workflow \
    --source=workflow.yaml \
    --service-account=${SERVICE_ACCOUNT}@${GOOGLE_CLOUD_PROJECT}.iam.gserviceaccount.com
    
  2. Execute the workflow:

    gcloud workflows run WORKFLOW_NAME
    

    The output should resemble the following:

    result: '{"body":{"multiplied":192},"code":200,"headers":{"Alt-Svc":"h3-29=\":443\";
    ...
    startTime: '2021-05-05T14:36:48.762896438Z'
    state: SUCCEEDED
    

Congratulations! You have deployed and executed a workflow that connects a series of services together.

To create more complex workflows using expressions, conditional jumps, Base64 encoding/decoding, subworkflows, and more, refer to the Workflows syntax reference and the Standard library overview.

Clean up

If you created a new project for this tutorial, delete the project. If you used an existing project and wish to keep it without the changes added in this tutorial, delete resources created for the tutorial.

Delete the project

The easiest way to eliminate billing is to delete the project that you created for the tutorial.

To delete the project:

  1. In the Cloud Console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

Delete tutorial resources

  1. Delete the Cloud Run service you deployed in this tutorial:

    gcloud run services delete SERVICE_NAME

    Where SERVICE_NAME is your chosen service name.

    You can also delete Cloud Run services from the Google Cloud Console.

  2. Remove the gcloud default configurations you added during the tutorial setup.

     gcloud config unset run/region
     gcloud config unset workflows/location
     gcloud config unset project
    

  3. Delete the workflow created in this tutorial:

    gcloud workflows delete WORKFLOW_NAME
    
  4. Delete the container images named gcr.io/PROJECT_ID/SERVICE_NAME from Container Registry.

What's next