Automating the Classification of Data Uploaded to Cloud Storage

This tutorial shows how to implement an automated data quarantine and classification system using Cloud Storage and other Google Cloud Platform (GCP) products. The tutorial assumes that you are familiar with GCP and basic shell programming.

In every organization, data protection officers like you face an ever-increasing amount of data, data that must be protected and treated appropriately. Quarantining and classifying that data can be complicated and time consuming, especially given hundreds or thousands of files a day.

What if you could take each file, upload it to a quarantine location, and have it automatically classified and moved to the appropriate location based on the classification result? This tutorial shows you how to implement such a system by using Cloud Functions, Cloud Storage, and the Cloud Data Loss Prevention (DLP) API.

Objectives

• Create Cloud Storage buckets to be used as part of the quarantine and classification pipeline.
• Create a Cloud Pub/Sub topic and subscription to notify you when file processing is completed.
• Create a simple Cloud Function that invokes the DLP API when files are uploaded.
• Upload some sample files to the quarantine bucket to invoke the Cloud Function. The function uses the DLP API to inspect and classify the files and move them to the appropriate bucket.

Costs

This tutorial uses billable GCP components, including:

• Cloud Storage
• Cloud Functions
• Cloud Data Loss Prevention API

You can use the Pricing Calculator to generate a cost estimate based on your projected usage.

Before you begin

1. Sign in to your Google Account.

   If you don't already have one, sign up for a new account.

2. Select or create a Google Cloud Platform project.

   Note: If you don't plan to keep the resources you create in this tutorial, create a new project instead of selecting an existing project. After you finish, you can delete the project, removing all resources associated with the project and tutorial.

   Go to the Manage resources page

3. Make sure that billing is enabled for your Google Cloud Platform project.

   Learn how to enable billing

4. Enable the Cloud Functions, Cloud Storage, and Cloud Data Loss Prevention APIs.

   Enable the APIs

Granting permissions to service accounts

Your first step is to grant permissions to two service accounts: the Cloud Functions service account and the DLP service account.

Grant permissions to the App Engine default service account

1. In the GCP Console, open the IAM & Admin page and select the project you created:

   GO TO THE IAM & ADMIN PAGE

2. Locate the App Engine service account. This account has the format [PROJECT_ID]@appspot.gserviceaccount.com. Replace [PROJECT_ID] with your project ID.

3. Select the edit icon next to the service account.

4. Add the following roles:

   • Project > Owner
   • Cloud DLP > DLP Administrator
   • Service Management > DLP API service agent

5. Click Save.

Grant permissions to the DLP service account

1. In the GCP Console, open the IAM & Admin page and select the project you created:

   GO TO THE IAM & ADMIN PAGE

2. Locate the DLP API Service Agent service account. This account has the format service-[PROJECT_NUMBER]@dlp-api.iam.gserviceaccount.com. Replace [PROJECT_NUMBER] with your project number.

3. Select the edit icon next to the service account.

4. Add the role Project > Viewer, and then click Save.

Building the quarantine and classification pipeline

In this section, you build the quarantine and classification pipeline shown in the following diagram.

The numbers in this pipeline correspond to these steps:

1. You upload files to Cloud Storage.
2. You invoke a Cloud Function.
3. The DLP API inspects and classifies the data.
4. The file is moved to the appropriate bucket.

Create Cloud Storage buckets

Following the guidance outlined in the bucket and object naming guidelines, create three uniquely named buckets, which you use throughout this tutorial:

• Bucket 1: Replace [YOUR_QUARANTINE_BUCKET] with a unique name.
• Bucket 2: Replace [YOUR_SENSITIVE_DATA_BUCKET] with a unique name.
• Bucket 3: Replace [YOUR_NON_SENSITIVE_DATA_BUCKET] with a unique name.

---

console

1. In the GCP Console, open the Cloud Storage browser:

   GO TO Cloud Storage BROWSER

2. Click Create bucket.

3. In the Bucket name text box, enter the name you selected for [YOUR_QUARANTINE_BUCKET], and then click Create.

4. Repeat for the [YOUR_SENSITIVE_DATA_BUCKET] and [YOUR_NON_SENSITIVE_DATA_BUCKET] buckets.

gcloud

1. Open Cloud Shell:

   GO TO Cloud Shell

2. Create three buckets using the following commands:

   gsutil mb gs://[YOUR_QUARANTINE_BUCKET]
   gsutil mb gs://[YOUR_SENSITIVE_DATA_BUCKET]
   gsutil mb gs://[YOUR_NON_SENSITIVE_DATA_BUCKET]
   

Create a Cloud Pub/Sub topic and subscription

console

1. Open the Cloud Pub/Sub Topics page:

   GO TO CLOUD PUB/SUB TOPICS

2. Click Create a topic.

3. In the text box that has an entry of the format PROJECTS/[YOUR_PROJECT_NAME]/TOPICS/, append the topic name, like this:

   PROJECTS/[YOUR_PROJECT_NAME]/TOPICS/[PUB/SUB_TOPIC]

4. Click Create.

5. Select the newly created topic, click the three dots (...) that follow the topic name, and then select New subscription.

6. In the text box that has an entry of the format PROJECTS/[YOUR_PROJECT_NAME]/TOPICS/[PUB/SUB_TOPIC], append the subscription name, like this:

   PROJECTS/[YOUR_PROJECT_NAME]/TOPICS/[PUB/SUB_TOPIC]/[PUB/SUB_SUBSCRIPTION]

7. Click Create.

gcloud

1. Open Cloud Shell:

   GO TO Cloud Shell

2. Create a topic, replacing [PUB/SUB_TOPIC] with a name of your choosing:

   gcloud pubsub topics create [PUB/SUB_TOPIC]

3. Create a subscription, replacing [PUB/SUB_SUBSCRIPTION] with a name of your choosing:

   gcloud pubsub subscriptions create [PUB/SUB_SUBSCRIPTION] --topic [PUB/SUB_TOPIC]

Create the Cloud Functions

This section steps through deploying the Python script containing the following two Cloud Functions:

• A function that is invoked when an object is uploaded to Cloud Storage.
• A function that is invoked when a message is received in the Cloud Pub/Sub queue.

Creating the first function

console

1. Open the Cloud Functions Overview page:

   GO TO THE CLOUD FUNCTIONS OVERVIEW PAGE

2. Select the project for which you enabled Cloud Functions.

3. Click Create function.

4. In the Name box, replace the default name with create_DLP_job.

5. In the Trigger field, select Cloud Storage.

6. In the Bucket field, click browse, select your quarantine bucket by highlighting the bucket in the drop-down list, and then click Select.

7. Under Runtime, select Python 3.7.

8. Under Source code, check Inline editor.

9. Paste the following code into the main.py box, replacing the existing text:

   gcs-dlp-classification-python/main.py

   View on GitHub

   """ Copyright 2018, Google, Inc.
   
   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at
   
     http://www.apache.org/licenses/LICENSE-2.0
   
   Unless  required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
   
   Authors: Yuhan Guo, Zhaoyuan Sun, Fengyi Huang, Weimu Song.
   Date:    October 2018
   
   """
   
   from google.cloud import dlp
   from google.cloud import storage
   from google.cloud import pubsub
   import os
   
   # ----------------------------
   #  User-configurable Constants
   
   PROJECT_ID = '[PROJECT_ID_HOSTING_STAGING_BUCKET]'
   """The bucket the to-be-scanned files are uploaded to."""
   STAGING_BUCKET = '[YOUR_QUARANTINE_BUCKET]'
   """The bucket to move "sensitive" files to."""
   SENSITIVE_BUCKET = '[YOUR_SENSITIVE_DATA_BUCKET]'
   """The bucket to move "non sensitive" files to."""
   NONSENSITIVE_BUCKET = '[YOUR_NON_SENSITIVE_DATA_BUCKET]'
   """ Pub/Sub topic to notify once the  DLP job completes."""
   PUB_SUB_TOPIC = '[PUB/SUB_TOPIC]'
   """The minimum_likelihood (Enum) required before returning a match"""
   """For more info visit: https://cloud.google.com/dlp/docs/likelihood"""
   MIN_LIKELIHOOD = 'POSSIBLE'
   """The maximum number of findings to report (0 = server maximum)"""
   MAX_FINDINGS = 0
   """The infoTypes of information to match"""
   """For more info visit: https://cloud.google.com/dlp/docs/concepts-infotypes"""
   INFO_TYPES = [
       'FIRST_NAME', 'PHONE_NUMBER', 'EMAIL_ADDRESS', 'US_SOCIAL_SECURITY_NUMBER'
   ]
   
   # End of User-configurable Constants
   # ----------------------------------
   
   # Initialize the Google Cloud client libraries
   dlp = dlp.DlpServiceClient()
   storage_client = storage.Client()
   publisher = pubsub.PublisherClient()
   subscriber = pubsub.SubscriberClient()
   
   
   def create_DLP_job(data, done):
     """This function is triggered by new files uploaded to the designated Cloud Storage quarantine/staging bucket.
   
          It creates a dlp job for the uploaded file.
       Arg:
          data: The Cloud Storage Event
       Returns:
           None. Debug information is printed to the log.
       """
     # Get the targeted file in the quarantine bucket
     file_name = data['name']
     print('Function triggered for file [{}]'.format(file_name))
   
     # Prepare info_types by converting the list of strings (INFO_TYPES) into a list of dictionaries
     info_types = [{'name': info_type} for info_type in INFO_TYPES]
   
     # Convert the project id into a full resource id.
     parent = dlp.project_path(PROJECT_ID)
   
     # Construct the configuration dictionary.
     inspect_job = {
         'inspect_config': {
             'info_types': info_types,
             'min_likelihood': MIN_LIKELIHOOD,
             'limits': {
                 'max_findings_per_request': MAX_FINDINGS
             },
         },
         'storage_config': {
             'cloud_storage_options': {
                 'file_set': {
                     'url':
                         'gs://{bucket_name}/{file_name}'.format(
                             bucket_name=STAGING_BUCKET, file_name=file_name)
                 }
             }
         },
         'actions': [{
             'pub_sub': {
                 'topic':
                     'projects/{project_id}/topics/{topic_id}'.format(
                         project_id=PROJECT_ID, topic_id=PUB_SUB_TOPIC)
             }
         }]
     }
   
     # Create the DLP job and let the DLP api processes it.
     try:
       dlp.create_dlp_job(parent, inspect_job)
       print('Job created by create_DLP_job')
     except Exception as e:
       print(e)
   
   
   def resolve_DLP(data, context):
     """This function listens to the pub/sub notification from function above.
   
       As soon as it gets pub/sub notification, it picks up results from the
       DLP job and moves the file to sensitive bucket or nonsensitive bucket
       accordingly.
       Args:
           data: The Cloud Pub/Sub event
   
       Returns:
           None. Debug information is printed to the log.
       """
     # Get the targeted DLP job name that is created by the create_DLP_job function
     job_name = data['attributes']['DlpJobName']
     print('Received pub/sub notification from DLP job: {}'.format(job_name))
   
     # Get the DLP job details by the job_name
     job = dlp.get_dlp_job(job_name)
     print('Job Name:{name}\nStatus:{status}'.format(
         name=job.name, status=job.state))
   
     # Fetching Filename in Cloud Storage from the original dlpJob config.
     # See defintion of "JSON Output' in Limiting Cloud Storage Scans':
     # https://cloud.google.com/dlp/docs/inspecting-storage
   
     file_path = (
         job.inspect_details.requested_options.job_config.storage_config
         .cloud_storage_options.file_set.url)
     file_name = os.path.basename(file_path)
   
     info_type_stats = job.inspect_details.result.info_type_stats
     source_bucket = storage_client.get_bucket(STAGING_BUCKET)
     source_blob = source_bucket.blob(file_name)
     if (len(info_type_stats) > 0):
       # Found at least one sensitive data
       for stat in info_type_stats:
         print('Found {stat_cnt} instances of {stat_type_name}.'.format(
             stat_cnt=stat.count, stat_type_name=stat.info_type.name))
       print('Moving item to sensitive bucket')
       destination_bucket = storage_client.get_bucket(SENSITIVE_BUCKET)
       source_bucket.copy_blob(source_blob, destination_bucket,
                               file_name)  # copy the item to the sensitive bucket
       source_blob.delete()  # delete item from the quarantine bucket
   
     else:
       # No sensitive data found
       print('Moving item to non-sensitive bucket')
       destination_bucket = storage_client.get_bucket(NONSENSITIVE_BUCKET)
       source_bucket.copy_blob(
           source_blob, destination_bucket,
           file_name)  # copy the item to the non-sensitive bucket
       source_blob.delete()  # delete item from the quarantine bucket
     print('{} Finished'.format(file_name))
   

10. Adjust the following lines in the code that you pasted into the main.py box, replacing the variables with the project ID of your project, the corresponding buckets, and the Cloud Pub/Sub topic and subscription names that you created earlier.

    [YOUR_QUARANTINE_BUCKET]
    [YOUR_SENSITIVE_DATA_BUCKET]
    [YOUR_NON_SENSITIVE_DATA_BUCKET]
    [PROJECT_ID_HOSTING_STAGING_BUCKET]
    [PUB/SUB_TOPIC]
    

11. In the Function to execute text box, replace hello_gcs with create_DLP_job.

12. Paste the following code into the requirements.txt text box, replacing the existing text:

    gcs-dlp-classification-python/requirements.txt

    View on GitHub

    google-cloud-dlp
    google-cloud-pubsub
    google-cloud-storage
    
    

13. Click Save.

    A green checkmark beside the function indicates a successful deployment.

    

gcloud

1. Open a Cloud Shell session and clone the GitHub repository that contains the code and some sample data files:

   OPEN IN Cloud Shell

2. Change directories to the folder the repository has been cloned to:

   cd gcs-dlp-classification-python/

3. Adjust the following lines in the code in the main.py box, replacing the following bucket variables with the corresponding buckets you created earlier. Also replace the Cloud Pub/Sub topic and subscription variables with the names you chose.

   [YOUR_QUARANTINE_BUCKET]
   [YOUR_SENSITIVE_DATA_BUCKET]
   [YOUR_NON_SENSITIVE_DATA_BUCKET]
   [PROJECT_ID_HOSTING_STAGING_BUCKET]
   [PUB/SUB_TOPIC]
   

4. Deploy the function, replacing [YOUR_QUARANTINE_BUCKET] with your bucket name:

   gcloud functions deploy create_DLP_job --runtime python37 \
       --trigger-resource [YOUR_QUARANTINE_BUCKET] \
       --trigger-event google.storage.object.finalize
   

5. Validate that the function has successfully deployed:

   gcloud functions describe create_DLP_job

   A successful deployment is indicated by a ready status similar to the following:

   status:  READY
   timeout:  60s
   

When the Cloud Function has successfully deployed, continue to the next section to create the second Cloud Function.

Creating the second function

console

1. Open the Cloud Functions Overview page:

   GO TO THE Cloud Functions OVERVIEW PAGE

2. Select the project for which you enabled Cloud Functions.

3. Click Create function.

4. In the Name box, replace the default name with resolve_DLP.

5. In the Trigger field, select Cloud Pub/Sub.

6. In the Topic field, enter [PUB/SUB_TOPIC].

7. Under Source code, check Inline editor.

8. Under Runtime, select Python 3.7.

9. Paste the following code into the main.py box, replacing the existing text:

   gcs-dlp-classification-python/main.py

   View on GitHub

   """ Copyright 2018, Google, Inc.
   
   Licensed under the Apache License, Version 2.0 (the "License");
   you may not use this file except in compliance with the License.
   You may obtain a copy of the License at
   
     http://www.apache.org/licenses/LICENSE-2.0
   
   Unless  required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
   
   Authors: Yuhan Guo, Zhaoyuan Sun, Fengyi Huang, Weimu Song.
   Date:    October 2018
   
   """
   
   from google.cloud import dlp
   from google.cloud import storage
   from google.cloud import pubsub
   import os
   
   # ----------------------------
   #  User-configurable Constants
   
   PROJECT_ID = '[PROJECT_ID_HOSTING_STAGING_BUCKET]'
   """The bucket the to-be-scanned files are uploaded to."""
   STAGING_BUCKET = '[YOUR_QUARANTINE_BUCKET]'
   """The bucket to move "sensitive" files to."""
   SENSITIVE_BUCKET = '[YOUR_SENSITIVE_DATA_BUCKET]'
   """The bucket to move "non sensitive" files to."""
   NONSENSITIVE_BUCKET = '[YOUR_NON_SENSITIVE_DATA_BUCKET]'
   """ Pub/Sub topic to notify once the  DLP job completes."""
   PUB_SUB_TOPIC = '[PUB/SUB_TOPIC]'
   """The minimum_likelihood (Enum) required before returning a match"""
   """For more info visit: https://cloud.google.com/dlp/docs/likelihood"""
   MIN_LIKELIHOOD = 'POSSIBLE'
   """The maximum number of findings to report (0 = server maximum)"""
   MAX_FINDINGS = 0
   """The infoTypes of information to match"""
   """For more info visit: https://cloud.google.com/dlp/docs/concepts-infotypes"""
   INFO_TYPES = [
       'FIRST_NAME', 'PHONE_NUMBER', 'EMAIL_ADDRESS', 'US_SOCIAL_SECURITY_NUMBER'
   ]
   
   # End of User-configurable Constants
   # ----------------------------------
   
   # Initialize the Google Cloud client libraries
   dlp = dlp.DlpServiceClient()
   storage_client = storage.Client()
   publisher = pubsub.PublisherClient()
   subscriber = pubsub.SubscriberClient()
   
   
   def create_DLP_job(data, done):
     """This function is triggered by new files uploaded to the designated Cloud Storage quarantine/staging bucket.
   
          It creates a dlp job for the uploaded file.
       Arg:
          data: The Cloud Storage Event
       Returns:
           None. Debug information is printed to the log.
       """
     # Get the targeted file in the quarantine bucket
     file_name = data['name']
     print('Function triggered for file [{}]'.format(file_name))
   
     # Prepare info_types by converting the list of strings (INFO_TYPES) into a list of dictionaries
     info_types = [{'name': info_type} for info_type in INFO_TYPES]
   
     # Convert the project id into a full resource id.
     parent = dlp.project_path(PROJECT_ID)
   
     # Construct the configuration dictionary.
     inspect_job = {
         'inspect_config': {
             'info_types': info_types,
             'min_likelihood': MIN_LIKELIHOOD,
             'limits': {
                 'max_findings_per_request': MAX_FINDINGS
             },
         },
         'storage_config': {
             'cloud_storage_options': {
                 'file_set': {
                     'url':
                         'gs://{bucket_name}/{file_name}'.format(
                             bucket_name=STAGING_BUCKET, file_name=file_name)
                 }
             }
         },
         'actions': [{
             'pub_sub': {
                 'topic':
                     'projects/{project_id}/topics/{topic_id}'.format(
                         project_id=PROJECT_ID, topic_id=PUB_SUB_TOPIC)
             }
         }]
     }
   
     # Create the DLP job and let the DLP api processes it.
     try:
       dlp.create_dlp_job(parent, inspect_job)
       print('Job created by create_DLP_job')
     except Exception as e:
       print(e)
   
   
   def resolve_DLP(data, context):
     """This function listens to the pub/sub notification from function above.
   
       As soon as it gets pub/sub notification, it picks up results from the
       DLP job and moves the file to sensitive bucket or nonsensitive bucket
       accordingly.
       Args:
           data: The Cloud Pub/Sub event
   
       Returns:
           None. Debug information is printed to the log.
       """
     # Get the targeted DLP job name that is created by the create_DLP_job function
     job_name = data['attributes']['DlpJobName']
     print('Received pub/sub notification from DLP job: {}'.format(job_name))
   
     # Get the DLP job details by the job_name
     job = dlp.get_dlp_job(job_name)
     print('Job Name:{name}\nStatus:{status}'.format(
         name=job.name, status=job.state))
   
     # Fetching Filename in Cloud Storage from the original dlpJob config.
     # See defintion of "JSON Output' in Limiting Cloud Storage Scans':
     # https://cloud.google.com/dlp/docs/inspecting-storage
   
     file_path = (
         job.inspect_details.requested_options.job_config.storage_config
         .cloud_storage_options.file_set.url)
     file_name = os.path.basename(file_path)
   
     info_type_stats = job.inspect_details.result.info_type_stats
     source_bucket = storage_client.get_bucket(STAGING_BUCKET)
     source_blob = source_bucket.blob(file_name)
     if (len(info_type_stats) > 0):
       # Found at least one sensitive data
       for stat in info_type_stats:
         print('Found {stat_cnt} instances of {stat_type_name}.'.format(
             stat_cnt=stat.count, stat_type_name=stat.info_type.name))
       print('Moving item to sensitive bucket')
       destination_bucket = storage_client.get_bucket(SENSITIVE_BUCKET)
       source_bucket.copy_blob(source_blob, destination_bucket,
                               file_name)  # copy the item to the sensitive bucket
       source_blob.delete()  # delete item from the quarantine bucket
   
     else:
       # No sensitive data found
       print('Moving item to non-sensitive bucket')
       destination_bucket = storage_client.get_bucket(NONSENSITIVE_BUCKET)
       source_bucket.copy_blob(
           source_blob, destination_bucket,
           file_name)  # copy the item to the non-sensitive bucket
       source_blob.delete()  # delete item from the quarantine bucket
     print('{} Finished'.format(file_name))
   

10. Adjust the following lines in the code that you pasted into the main.py box, replacing the variables with the project ID of your project, the corresponding buckets, and the Cloud Pub/Sub topic and subscription names that you created earlier.

    [YOUR_QUARANTINE_BUCKET]
    [YOUR_SENSITIVE_DATA_BUCKET]
    [YOUR_NON_SENSITIVE_DATA_BUCKET]
    [PROJECT_ID_HOSTING_STAGING_BUCKET]
    [PUB/SUB_TOPIC]
    

11. In the Function to execute text box, replace helloPubSub with resolve_DLP.

12. Paste the following into the requirements.txt text box, replacing the existing text:

    gcs-dlp-classification-python/requirements.txt

    View on GitHub

    google-cloud-dlp
    google-cloud-pubsub
    google-cloud-storage
    
    

13. Click Save.

    A green checkmark beside the function indicates a successful deployment.

    

gcloud

1. Open (or reopen) a Cloud Shell session and clone the GitHub repository that contains the code and some sample data files:

   OPEN IN Cloud Shell

2. Change directories to the folder with the Python code:

   cd gcs-dlp-classification-python

3. Adjust the following lines in the code in the main.py box, replacing the following bucket variables with the corresponding buckets you created earlier. Also replace the Cloud Pub/Sub topic and subscription variables with the names you chose.

   [YOUR_QUARANTINE_BUCKET]
   [YOUR_SENSITIVE_DATA_BUCKET]
   [YOUR_NON_SENSITIVE_DATA_BUCKET]
   [PROJECT_ID_HOSTING_STAGING_BUCKET]
   [PUB/SUB_TOPIC]
   

4. Deploy the function, replacing [PUB/SUB_TOPIC] with your Cloud Pub/Sub topic:

   gcloud functions deploy resolve_DLP --runtime python37 --trigger-topic [PUB/SUB_TOPIC]

5. Validate that the function has successfully deployed:

   gcloud functions describe resolve_DLP

   A successful deployment is indicated by a ready status similar to the following:

   status:  READY
   timeout:  60s
   

When the Cloud Function has successfully deployed, continue to the next section.

Upload sample files to the quarantine bucket

The GitHub repository associated with this article includes sample data files. The folder contains some files that have sensitive data and other files that have nonsensitive data. Sensitive data is classified as containing one or more of the following INFO_TYPES values:

US_SOCIAL_SECURITY_NUMBER
EMAIL_ADDRESS
PERSON_NAME
LOCATION
PHONE_NUMBER

The data types that are used to classify the sample files are defined in the INFO_TYPES constant in the main.py file, which is initially set to [‘PHONE_NUMBER', ‘EMAIL_ADDRESS'].

1. If you have not already cloned the repository, open Cloud Shell and clone the GitHub repository that contains the code and some sample data files:

   OPEN IN Cloud Shell

2. Change folders to the sample data files:

   cd ~/dlp-cloud-functions-tutorials/sample_data/

3. Copy the sample data files to the quarantine bucket by using the gsutil command, replacing [YOUR_QUARANTINE_BUCKET] with the name of your quarantine bucket:

   gsutil -m  cp * gs://[YOUR_QUARANTINE_BUCKET]/

   The DLP API inspects and classifies each file uploaded to the quarantine bucket and moves it to the appropriate target bucket based on its classification.

4. In the Cloud Storage console, open the Storage Browser page:

   GO TO Cloud Storage BROWSER

5. Select one of the target buckets that you created earlier and review the uploaded files. Also review the other buckets that you created.

Cleaning up

After you've finished the current tutorial, you can clean up the resources you created on Google Cloud Platform so you won't be billed for them in the future. The following sections describe how to delete or turn off these resources.

Delete the project

Caution: Deleting a project has the following effects:
• Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project.
• Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project.
1. In the GCP Console, go to the Projects page.

   Go to the Projects page

2. In the project list, select the project you want to delete and click Delete .
3. In the dialog, type the project ID, and then click Shut down to delete the project.

What's next

• Try setting different valid data type values for INFO_TYPES.
• Learn more about inspecting storage and databases for sensitive data using the DLP API.
• Learn more about Cloud Functions.
• Try out other Google Cloud Platform features for yourself. Have a look at our tutorials.