Tokenizing sensitive cardholder data for PCI DSS

This tutorial shows how to set up an access-controlled credit and debit card tokenization service on Cloud Functions. To set up the service, the article uses these Google Cloud services: Identity and Access Management (IAM), Cloud Key Management Service (KMS), and Firestore in Datastore mode.

Tokenization is the process of substituting a benign placeholder value, or token, for sensitive information such as credit card data. Part 3 of the Payment Card Industry Data Security Standard (PCI DSS) requires that most of the data stored on a credit card be treated as sensitive information.

A token by itself is meaningless except as a means of looking up tokenized data in a particular context. However, you still need to ensure that your tokens don't contain any user-specific information and that they aren't directly decryptable. This way, if you lose control over your customers' payment card tokens, no one can use the tokens to compromise the cardholder data.

A service for handling sensitive information

You have many choices for the platform or service to host your cardholder data environment (CDE). This tutorial guides you through a sample deployment using Cloud Functions and helps you on the next steps toward a production-ready solution.

Cloud Functions is a serverless platform that hosts and executes code, and it's a convenient place to quickly launch an application that scales without intervention. Keep in mind that in a PCI DSS compliant CDE, you must limit all inbound and outbound traffic to authorized connections. Such fine-grained controls are not currently available for Cloud Functions. Therefore, you must implement compensating controls elsewhere (such as in your application) or choose a different platform. The same Tokenization service can be run in a containerized manner such as an autoscaling managed instance group or a Kubernetes cluster. These would be preferable production environments with their complete VPC network controls.

Cloud KMS is Google Cloud's secret-management service. Cloud KMS hosts your encryption keys, rotates them regularly, and encrypts or decrypts stored account data. Firestore stores the tokenized data.

IAM is used in this tutorial to provide tight controls on all of the resources used in the tokenization service. You need a special service account that has frequently expiring tokens to grant access to Cloud KMS and Firestore and to execute the tokenizer.

The following figure illustrates the tokenization app architecture.

tokenization app architecture

Firestore modes

Firestore is the next major version of Datastore. Firestore can run in Datastore mode, which uses the same API as Datastore and scales to millions of writes per second, or in Native mode, which uses a different API and is best for mobile and web apps.

To complete this tutorial, we recommend using Firestore in Datastore mode. The tutorial does not support Firestore in Native mode.

If you've already enabled Datastore in your Google Cloud project, you can use Datastore instead. However, the rest of this tutorial assumes that you're using Firestore in Datastore mode.

If you need help deciding which Firestore mode to use in your own app, see our guide to choosing between Native mode and Datastore mode.


  • Select a database service and mode.
  • Create a service account.
  • Set up Cloud KMS.
  • Choose a storage region.
  • Create two Cloud Functions.


This tutorial uses the following billable components of Google Cloud:

To generate a cost estimate based on your projected usage, use the pricing calculator. New Google Cloud users might be eligible for a free trial.

Before you begin

  1. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

    Go to project selector

  2. Throughout this tutorial, your project is referred to as [YOUR_PROJECT].
  3. Make sure that billing is enabled for your Cloud project. Learn how to check if billing is enabled on a project.

  4. Enable the Cloud Functions and Cloud KMS APIs.

    Enable the APIs

  5. Select Datastore mode for Firestore.
    Select Datastore mode

When you finish this tutorial, you can avoid continued billing by deleting the resources you created. For more information, see Clean up.

Creating the service account

To create a service account and download a JSON key for the service account, do the following:

  1. Create a service account:

    1. In the console, go to the Service Accounts page.

      Go to Service Accounts

    2. Select your project.

    3. Click Create Service Account.

    4. In the Service account name field, enter a Tokenization Service User. The console fills in the Service account ID field based on this name.

    5. Optional: In the Service account description field, enter a description for the service account.

    6. Click Create and continue.

    7. Click Select a role, and then select Cloud KMS CryptoKey Encrypter/Decrypter.

    8. Click Add another role.

    9. Click Select a role, and then select Cloud Datastore User.

    10. To finish creating the service account, click Done.

      You now have a service account user with the following email address:


      Don't close your browser window. You use it in the next step.

  2. Download a JSON key for the service account that you just created:

    1. In the console Service accounts page, click the email address for the service account that you created.
    2. In the Keys tab, click Add key, then click Create new key.
    3. Click Create. A JSON key file is saved to your computer.

      Make sure to store the key file securely, because it can be used to authenticate as your service account. You can move and rename this file if you want to.

    4. After you save the key file, on the Private key saved to your computer dialog, click Close.

Setting up Cloud KMS

  1. In the console, open Cryptographic Keys.

    Go to the Cryptographic Keys page

  2. Click + Create key ring. In the dialog that appears, do the following:

    1. Name the key ring tokenization-service-kr.
    2. For Key ring location, select global. This is a common choice that suffices for this tutorial. Before you make any production architecture decisions, however, make sure you understand the differences between the various Cloud KMS locations.
    3. Double-check your choices, because you can't delete or rename key rings after they are created.
    4. Click Create.

      Creating a key ring

    The system creates the key ring and forwards you to the key creation page.

  3. In the Create key dialog, do the following:

    1. Name the key cc-tokenization.
    2. For Purpose, select Symmetric encrypt/decrypt.
    3. Set Rotation period to a value you choose, and click Create.

    Tracking information about your keys

Creating Cloud Functions

This tutorial assumes you'll be using Cloud Shell. If you use a different terminal, be sure you have the latest version of the Google Cloud CLI.

  1. In the console, open Cloud Shell:

    Go to Cloud Shell

  2. Clone the GitHub project repository and move to the working folder:

    git clone gcp-community
    cd gcp-community/tutorials/pci-tokenizer/

    The gcs-cf-tokenizer folder contains the file index.js, which is the source for two different Cloud Functions that you will create. It also contains package.json, which tells Cloud Functions which packages to run.

  3. Apply the KMS configuration Copy the config template file and open it for editing:

    cp config/default.json config/local.json
    nano config/local.json

    The Node.js 10 runtime requires you to explicitly define the Google Cloud project ID:

    "project_id":              "YOUR_PROJECT_ID_HERE"
  4. Find the KMS configuration and apply the KMS values you created in the previous section:

    "location":                "global",
    "key_ring":                "tokenization-service-kr",
    "key_name":                "cc-tokenization"
  5. Deploy both of the Cloud Functions in KMS mode (see for more information on DLP mode):

    gcloud functions deploy tokenize --runtime=nodejs10 --trigger-http --entry-point=kms_crypto_tokenize --memory=256MB --source=.
    gcloud functions deploy detokenize --runtime=nodejs10 --trigger-http --entry-point=kms_crypto_detokenize --memory=256MB --source=.

    These commands create two separate Cloud Functions: one for turning the card number into a token, and another to reverse the process. The differing entry points direct execution to the proper starting function in index.js.

  6. Note the httpsTrigger URL in the output for each command. You need these URLs to call the functions.

  7. When the functions are deployed, open the Cloud Functions console.

    Open the Cloud Functions console

  8. Verify that the functions were created. If all went well, you will see your two functions with a check mark next to each one.

    Verifying that your Cloud Functions were created

Build authentication tokens

You execute all Cloud Functions as a single service account user. If you want more control over what a given Cloud Function can do, you can build rules into the code itself. The code you deployed in the previous steps requires an OAuth2 authorization token (referred to as auth_token in this document) that was generated on behalf of the special service account that you created (tokenization-service-user@[YOUR_PROJECT]

There are many ways to generate an auth_token that go beyond the scope of this tutorial. A development and testing tool called getToken.js can help you through the rest of this tutorial. You can find getToken.js in the src folder of the GitHub repository.

Before you can use the auth_token generator, you must provide service account credentials.

  1. Open the JSON file containing the service account credentials that you downloaded, and copy the entire contents to your clipboard, making sure not to truncate any data.
  2. In Cloud Shell, create and open an auth token file:

    nano ~/.tokenization-service-user.json
  3. Paste the contents of the JSON file.

  4. Press Control+O, Enter, and then Control+X to save and exit the editor.

  5. If you are running this process outside of Cloud Shell, define the GOOGLE_CLOUD_PROJECT environment variable:

  6. Generate a sample auth_token:

    npm install
    AUTH_TOKEN=$(GOOGLE_APPLICATION_CREDENTIALS=~/.tokenization-service-user.json node src/getToken.js)
    echo $AUTH_TOKEN

    These commands generate an auth_token string and store it in the environment variable $AUTH_TOKEN. For your convenience, the auth_token is also displayed. From here, you can use that auth_token to call your tokenization service.

Call the tokenizer

  1. Open the Cloud Functions console.

    Open the Cloud Functions console

  2. Click your function.

  3. Switch to the Trigger tab and copy the URL if you don't have it already. Use the trigger URL in place of {CF_URL} in the steps that follow.

    Copying the URL of your Cloud Function

  4. In your shell, save the trigger URL to an environment variable:

    export TOK_URL="{CF_URL}"
  5. Create some sample data to pass to the tokenizer:

    export TOK_CC=4000300020001000
    export TOK_MM=11
    export TOK_YYYY=2028
    export TOK_UID=543210
  6. Generate an auth_token as described in the previous section, and then call the tokenizer:

    CC_TOKEN=$(curl -s -X POST "$TOK_URL" -H "Content-Type:application/json" --data '{"auth_token":"'$AUTH_TOKEN'", "cc": "'$TOK_CC'", "mm": "'$TOK_MM'", "yyyy": "'$TOK_YYYY'", "userid": "'$TOK_UID'"}') ; echo $CC_TOKEN

    If this step works, a 64-character alphanumeric string is displayed. This string has been stored in the environment variable CC_TOK. You can retrieve the card information by invoking the detokenizer.

  7. To reverse the tokenization process, first retrieve the URL to the detokenize function the same way you retrieved it to tokenize.

  8. To detokenize, run the following command, replacing {CF2_URL} with your trigger URL:

    export DETOK_URL="{CF2_URL}"
    curl -s -X POST "$DETOK_URL" -H "Content-Type:application/json" --data '{"auth_token":"'$AUTH_TOKEN'", "cc_token": "'$CC_TOKEN'"}'

    The output looks something like the following:


    This data is what was originally sent into the tokenizer, decrypted, and retrieved by your app.

Verify the data

  1. Open the Datastore console. This console also supports Firestore in Datastore mode.

    Open the Datastore console

  2. If you don't see any records, set Kind to cc and Namespace to tokenizer.

The data you see is the tokenized data with an encrypted payment card in the cipher field. (The following screenshot doesn't show the cipher field.)

Successfully tokenized payment cards (cipher field not shown)

Expanding on this tutorial

The sample code on GitHub is an excellent start, but there is more to consider before moving to production.

If you are new to generating and refreshing access tokens, that generating a OAuth token for each request adds unnecessary API calls and can slow down your service and lead to the exhaustion of auth quotas.

If you choose to use Cloud Functions for payment card tokenization, you might need to do more work to satisfy your Qualified Security Assessor or Self-Assessment Questionnaire. Specifically, PCI DSS sections 1.2 and 1.3 require tight controls on inbound and outbound traffic. Cloud Functions and App Engine don't offer a two-way configurable firewall, so you must either create compensating controls or deploy the tokenization service on Compute Engine or Google Kubernetes Engine. If you would like to explore containerization, the GitHub code is Docker compatible and contains supporting documentation.

This sample code also pulls in the npm (Node.js package manager) dependencies on deployment. In your production environment, always pin dependencies to specific vetted versions. Then bundle these versions with the app itself or serve them from a private and trusted location. Either approach helps you avoid downtime resulting from an outage at the public npm repository or from a supply-chain attack that infects packages that you assumed were safe. If you pre-build and bundle the complete app, your deployment time typically decreases, which means faster launches and smoother scaling.

Clean up

To clean up the individual resources you used in this tutorial, take the following actions:

  1. Open the Datastore console.

    Open the Datastore console

  2. Select all the records you created and click Delete.

  3. Open the Cloud Functions console.

    Open the Cloud Functions console

  4. Select the functions you created and click Delete.

  5. Open the IAM console.

    Open the IAM console

  6. Select the service account named tokenization-service-user@[YOUR_PROJECT] and click Remove.

  7. In the IAM Cryptographic Keys page, open the key ring tokenization-service-kr. Delete the key cc-tokenization.

  8. In the Cloud Storage console, delete the bucket named staging.[YOUR_PROJECT]

Or, if you created a project just for this tutorial, you can delete the entire project.

  1. In the console, go to the Manage resources page.

    Go to Manage resources

  2. In the project list, select the project that you want to delete, and then click Delete.
  3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next