Use Cloud Scheduler to invoke private Cloud Functions with OIDC
Dina Graves | Developer Programs Engineer | Google
Contributed by Google employees.
This tutorial shows you how to use Cloud Scheduler to invoke a private Cloud Function using HTTP targets and triggers and OIDC authentication.
Invoking a Cloud Function with authenticated HTTP that then performs an action is a common use case for Cloud Functions. Using Cloud Scheduler to schedule these invocations is a common use case for Cloud Scheduler.
To use Cloud Pub/Sub as an intermediary, see the Using Pub/Sub to trigger a Cloud Function tutorial.
Objectives
In this tutorial, you do the following:
- Create a service account with limited access.
- Create a Cloud Function that triggers on HTTP.
- Create a Cloud Scheduler job that targets an HTTP endpoint.
- Run the Cloud Scheduler job.
- Verify success of the job.
Costs
This tutorial uses billable components of Google Cloud, including the following:
Use the Pricing Calculator to generate a cost estimate based on your projected usage.
Before you begin
You can run the commands in this tutorial in Cloud Shell or with a local installation of gcloud.
We recommend that you create a new Google Cloud project for this tutorial, so that all created resources can be easily deleted when the tutorial is complete.
- In the Cloud console, on the project selector page, create a Cloud project. For details, see Create a project.
Make sure that billing is enabled for your project. For details, see Confirm billing is enabled.
Enable the Cloud Build, Cloud Scheduler, and Cloud Functions APIs:
- Enable the APIs in the Cloud console.
Enable the APIs from the command line:
gcloud services enable
cloudbuild.googleapis.com
cloudscheduler.googleapis.com
cloudfunctions.googleapis.com
Get the source code
The function used in this tutorial prints Hello, world
in the function logs and returns a formatted message.
This function is called from many different sources, so to ensure that every type of invocation is handled, this example uses the
Flask get_data()
method, decoding the data payload manually. If the
data is valid JSON, the function returns a message using the value of the name given. If it is not available, it will default to World
. If the payload is
invalid, an error is returned.
main.py:
import json
def hello_world(request):
request = request.get_data()
try:
request_json = json.loads(request.decode())
except ValueError as e:
print(f"Error decoding JSON: {e}")
return "JSON Error", 400
name = request_json.get("name") or "World"
msg = f'Hello, {name}!'
print(msg)
return msg
This example shows the Cloud Scheduler and Cloud Functions concepts in a simple proof of concept, but this architecture can be extended for more complex use cases, such as the following:
- Sending Slack notifications
- Performing BigQuery batch processes
- Running AutoML jobs
- Running nightly tests
- Cleaning up Cloud Storage buckets
Test the function locally
This function can be tested on your local machine by using the Functions Framework.
Install the Function Frameworks package on your machine:
pip install functions-framework==3.0.0
Use Functions Framework to target the
hello_world
method inmain.py
:functions-framework --target hello_world --debug
The function will be made available at a local address ("Running on http://...").
In a new terminal, use
curl
to send POST data to test the function:curl -d '{"name": "local function"}' http://0.0.0.0:8080
You should get the following result:
Hello, local function!
The terminal running the Functions Framework will show the logs for the invocation:
* Serving Flask app "hello_world" (lazy loading) * Environment: production * Debug mode: on * Running on http://x.x.x.x:8080/ (Press CTRL+C to quit) * Restarting with fsevents reloader * Debugger is active! Hello, local function! 127.0.0.1 - - [00/Jan/2020 00:00:00] "POST / HTTP/1.1" 200 -
Authentication architecture overview
To use a private Cloud Function, you need to send the request with sufficient authentication identification. Cloud Scheduler automates this for you by allowing you to specify multiple kinds of authentication headers. An OpenID Connect (OIDC) token is the most general way to provide this. By creating a service account that has only the ability to invoke functions, then assigning that as the identity of the function and as the OIDC service account, you can implement authentication scheduled invocation of your function.
See also:
Deploying the example
To deploy the example, you create a dedicated service account to serve as the identity for the function and scheduler, create and test a Cloud Function, and create and test a Cloud Scheduler job.
Create a service account
-
gcloud iam service-accounts create myserviceaccount \ --display-name "my service account"
Assign the role to allow this service account to invoke Cloud Functions:
gcloud projects add-iam-policy-binding ${PROJECT_ID} \ --member serviceAccount:myserviceaccount@${PROJECT_ID}.iam.gserviceaccount.com \ --role roles/cloudfunctions.invoker
Create a Cloud Function
Using the source code provided earlier, deploy the Cloud Function, ensuring that the previously created service account is associated with this function, and that unauthenticated access is disallowed:
gcloud functions deploy hello_world \
--trigger-http \
--region us-central1 \
--runtime python39 \
--service-account myserviceaccount@${PROJECT_ID}.iam.gserviceaccount.com \
--no-allow-unauthenticated
Test the function
After the function is deployed, it can be tested. Since the request must be authenticated, the most straightforward way of testing this function is in the Cloud console.
- Go to the Cloud console and navigate to the Cloud Functions page.
- Find the
hello_world
function, and click the Testing tab. In the Triggering event field, enter valid JSON that contains a name key and a value:
{"name": "testing event"}
Click the Test the function button.
The result should be
Hello, testing event!
.
You can also test this event in the console with curl
by specifying the authorization header:
FUNCTION_URL=$(gcloud functions describe hello_world --format 'value(httpsTrigger.url)')
curl -H "Authorization: Bearer $(gcloud auth print-identity-token)" \
$FUNCTION_URL \
-d '{"name": "testing event"}'
Create a scheduled job
Create a scheduled job to invoke the Cloud Function previously created, using the service account that you created.
- In the Cloud console, go to the Cloud Scheduler page.
- Click Schedule a job.
- In the Define the schedule section, enter the following:
- Name:
my-hourly-job
- Region:
us-central1
- Frequency:
0 * * * *
(hourly) - Timezone:
Australia/Sydney
- Name:
- Click Continue.
- In the Configure the execution section, enter the following:
- Target type:
HTTP
- URL: the URL of the Cloud Function deployed earlier (
$FUNCTION_URL
) - HTTP Method:
POST
- Body:
{"name": "Scheduler"}
- Auth Header: Add OIDC token
- Service account:
my service account
- Target type:
- Click Continue, and then click Create.
Alternatively, you can run the following equivalent gcloud
command:
gcloud scheduler jobs create http my-hourly-job \
--location us-central1 \
--schedule "0 * * * *" \
--time-zone "Australia/Sydney" \
--uri "https://${REGION}-${PROJECT_ID}.cloudfunctions.net/hello_world" \
--http-method POST \
--message-body '{"name": "Scheduler"}' \
--oidc-service-account-email myserviceaccount@${PROJECT_ID}.iam.gserviceaccount.com
You can confirm that the job was created by checking the Cloud Scheduler section of the Cloud console, or by listing the active jobs:
gcloud scheduler jobs list
Invoke a Cloud Function from a scheduled job
To test the configurations without having to wait for the next scheduled invocation, you can trigger the scheduled job by clicking Run Now on the scheduled job listing in the Cloud console. You can also use the following command:
gcloud scheduler jobs run my-hourly-job
This command return no output of its own.
Check success
You can check the Cloud Functions logs on the Cloud Logging page in the Cloud console or with the following command:
gcloud functions logs read hello_world
You should see something similar to the following:
LEVEL NAME EXECUTION_ID TIME_UTC LOG
D hello_world rndmid64qasr 2020-00-00 00:00:00.000 Function execution started
I hello_world rndmid64qasr 2020-00-00 00:00:00.000 Hello, Scheduler!
D hello_world rndmid64qasr 2020-00-00 00:00:00.000 Function execution took 16 ms, finished with status code: 200
Checking the logs again each hour should show that the scheduled event occurred and successfully ran.
Clean up
To avoid incurring charges to your Google Cloud account for the resources used in this tutorial, delete the project created specifically for this tutorial.
- In the Cloud console, go to the Manage resources page.
- In the project list, select the project that you want to delete, and then click Delete.
- In the dialog, type the project ID, and then click Shut down.
What's next
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see our Site Policies. Java is a registered trademark of Oracle and/or its affiliates.