Creating Cloud DLP de-identification templates

You can use templates to create and persist configuration information to use with Cloud Data Loss Prevention (DLP). Templates are useful for decoupling configuration such as what you inspect for and how you de-identify it from the implementation of your requests. Templates provide a robust way to re-use configuration and enable consistency across users and data sets.

The Cloud DLP supports two types of templates:

• De-identification templates: Templates for saving configuration information for de-identification jobs, including both infoType and record (structured dataset) transformations.
• Inspection templates: Templates for persisting configuration information for inspection scan jobs, including what predefined or custom detectors to use. For more information about inspection templates, see Creating Cloud DLP inspection templates.

For conceptual information about templates in Cloud DLP, see Templates.

The remainder of this topic shows you how to create de-identification templates for use with the Cloud DLP.

Create a de-identification template

It’s useful to note that a de-identification template is simply a reusable de-identification configuration plus some metadata. In API terms, the DeidentifyTemplate object is effectively a DeidentifyConfig object that includes a few more fields of metadata, such as a display name and a description. Therefore, to create a new de-identification template, the basic steps are:

1. Start with a DeidentifyConfig object.
2. Call or POST the create method of either the projects.deidentifyTemplates or organizations.deidentifyTemplates resource, including in your request a DeidentifyTemplate object that contains a display name, a description, and that DeidentifyConfig object.

The returned DeidentifyTemplate will be usable immediately. You can reference it in other calls or jobs by its name. You can list the existing templates by calling the *.deidentifyTemplates.list method. To view a specific template, call the *.deidentifyTemplates.get method. Note that the limit for the number of templates you can create is 1000.

If you've already had some experience de-identifying sensitive data from text, images, or structured content using Cloud DLP, you've already created a DeidentifyConfig object. One additional step turns it into a DeidentifyTemplate object.

REST example

The following JSON is an example of what you could send to the projects.deidentifyTemplates.create method. This JSON creates a new template with the given display name and description, and scans for matches on the infoTypes EMAIL_ADDRESS and GENERIC_ID. When it finds content matching those infoTypes, it will mask the first three characters with an asterisk (*) character:

JSON Input:

POST https://dlp.googleapis.com/v2/projects/[PROJECT_ID]/deidentifyTemplates?key={YOUR_API_KEY}

{
  "deidentifyTemplate":{
    "displayName":"Email and id masker",
    "description":"De-identifies emails and ids with a series of asterisks.",
    "deidentifyConfig":{
      "infoTypeTransformations":{
        "transformations":[
          {
            "infoTypes":[
              {
                "name":"GENERIC_ID"
              }
            ],
            "primitiveTransformation":{
              "replaceWithInfoTypeConfig":{

              }
            }
          },
          {
            "infoTypes":[
              {
                "name":"EMAIL_ADDRESS"
              }
            ],
            "primitiveTransformation":{
              "characterMaskConfig":{
                "charactersToIgnore":[
                  {
                    "charactersToSkip":"@"
                  }
                ],
                "maskingCharacter":"*"
              }
            }
          }
        ]
      }
    }
  }
}

JSON Output:

{
  "name":"projects/[PROJECT_ID]/deidentifyTemplates/[JOB-ID]",
  "displayName":"Email and id masker",
  "description":"De-identifies emails and ids with a series of asterisks.",
  "createTime":"2018-11-30T07:17:59.536022Z",
  "updateTime":"2018-11-30T07:17:59.536022Z",
  "deidentifyConfig":{
    "infoTypeTransformations":{
      "transformations":[
        {
          "infoTypes":[
            {
              "name":"GENERIC_ID"
            }
          ],
          "primitiveTransformation":{
            "replaceWithInfoTypeConfig":{

            }
          }
        },
        {
          "infoTypes":[
            {
              "name":"EMAIL_ADDRESS"
            }
          ],
          "primitiveTransformation":{
            "characterMaskConfig":{
              "maskingCharacter":"*",
              "charactersToIgnore":[
                {
                  "charactersToSkip":"@"
                }
              ]
            }
          }
        }
      ]
    }
  }
}

Note: This template was created at the project level, but you could also create it at the organization level using the organizations.deidentifyTemplates.create method.

To quickly try this out, you can use the APIs Explorer that's embedded below. For general information about using JSON to send requests to the Cloud DLP API, see the JSON quickstart.

Using de-identification templates

Once you’ve created a new template, you can use its name identifier anywhere deidentifyTemplateName is accepted, such as:

• projects.content.deidentify: De-identifies potentially sensitive data in content using the template as its configuration. Be aware that this method can use either an inspection template or a de-identification template.

Listing de-identification templates

To list all de-identification templates that have been created, use one of the *.*.list methods:

• organizations.deidentifyTemplates.list
• projects.deidentifyTemplates.list

Code examples

Following is sample code in several languages that demonstrates how to use Cloud DLP to list inspection templates. The process is identical to the one for listing de-identification templates.

Important: The code on this page requires that you first set up a Cloud DLP client. For more information about installing and creating a Cloud DLP client, see Cloud DLP client libraries. (Sending JSON to Cloud DLP REST endpoints does not require a client library.)

Java

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.cloud.dlp.v2.DlpServiceClient.ListInspectTemplatesPagedResponse;
import com.google.privacy.dlp.v2.InfoType;
import com.google.privacy.dlp.v2.InspectConfig;
import com.google.privacy.dlp.v2.InspectTemplate;
import com.google.privacy.dlp.v2.ListInspectTemplatesRequest;
import com.google.privacy.dlp.v2.LocationName;
import java.io.IOException;

class TemplatesList {

  public static void main(String[] args) throws Exception {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-project-id";
    listInspectTemplates(projectId);
  }

  // Lists all templates associated with a given project
  public static void listInspectTemplates(String projectId) throws IOException {
    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlpServiceClient = DlpServiceClient.create()) {

      // Create the request to be sent by the client
      ListInspectTemplatesRequest request =
          ListInspectTemplatesRequest.newBuilder()
              .setParent(LocationName.of(projectId, "global").toString())
              .setPageSize(1)
              .build();

      // Send the request
      ListInspectTemplatesPagedResponse response = dlpServiceClient.listInspectTemplates(request);

      // Parse through and process the response
      System.out.println("Templates found:");
      for (InspectTemplate template : response.getPage().getResponse().getInspectTemplatesList()) {
        System.out.printf("Template name: %s\n", template.getName());
        if (template.getDisplayName() != null) {
          System.out.printf("\tDisplay name: %s \n", template.getDisplayName());
          System.out.printf("\tCreate time: %s \n", template.getCreateTime());
          System.out.printf("\tUpdate time: %s \n", template.getUpdateTime());

          // print inspection config
          InspectConfig inspectConfig = template.getInspectConfig();
          for (InfoType infoType : inspectConfig.getInfoTypesList()) {
            System.out.printf("\tInfoType: %s\n", infoType.getName());
          }
          System.out.printf("\tMin likelihood: %s\n", inspectConfig.getMinLikelihood());
          System.out.printf("\tLimits: %s\n", inspectConfig.getLimits().getMaxFindingsPerRequest());
        }
      }
    }
  }
}

Node.js

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The project ID to run the API call under
// const projectId = 'my-project';

// Helper function to pretty-print dates
const formatDate = date => {
  const msSinceEpoch = parseInt(date.seconds, 10) * 1000;
  return new Date(msSinceEpoch).toLocaleString('en-US');
};

async function listInspectTemplates() {
  // Construct template-listing request
  const request = {
    parent: `projects/${projectId}/locations/global`,
  };

  // Run template-deletion request
  const [templates] = await dlp.listInspectTemplates(request);

  templates.forEach(template => {
    console.log(`Template ${template.name}`);
    if (template.displayName) {
      console.log(`  Display name: ${template.displayName}`);
    }

    console.log(`  Created: ${formatDate(template.createTime)}`);
    console.log(`  Updated: ${formatDate(template.updateTime)}`);

    const inspectConfig = template.inspectConfig;
    const infoTypes = inspectConfig.infoTypes.map(x => x.name);
    console.log('  InfoTypes:', infoTypes.join(' '));
    console.log('  Minimum likelihood:', inspectConfig.minLikelihood);
    console.log('  Include quotes:', inspectConfig.includeQuote);

    const limits = inspectConfig.limits;
    console.log('  Max findings per request:', limits.maxFindingsPerRequest);
  });
}

listInspectTemplates();

Python

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

def list_inspect_templates(project):
    """Lists all Data Loss Prevention API inspect templates.
    Args:
        project: The Google Cloud project id to use as a parent resource.
    Returns:
        None; the response from the API is printed to the terminal.
    """

    # Import the client library
    import google.cloud.dlp

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Convert the project id into a full resource id.
    parent = f"projects/{project}"

    # Call the API.
    response = dlp.list_inspect_templates(request={"parent": parent})

    for template in response:
        print("Template {}:".format(template.name))
        if template.display_name:
            print("  Display Name: {}".format(template.display_name))
        print("  Created: {}".format(template.create_time))
        print("  Updated: {}".format(template.update_time))

        config = template.inspect_config
        print(
            "  InfoTypes: {}".format(", ".join([it.name for it in config.info_types]))
        )
        print("  Minimum likelihood: {}".format(config.min_likelihood))
        print("  Include quotes: {}".format(config.include_quote))
        print(
            "  Max findings per request: {}".format(
                config.limits.max_findings_per_request
            )
        )

Go

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

import (
	"context"
	"fmt"
	"io"
	"time"

	dlp "cloud.google.com/go/dlp/apiv2"
	"github.com/golang/protobuf/ptypes"
	"google.golang.org/api/iterator"
	dlppb "google.golang.org/genproto/googleapis/privacy/dlp/v2"
)

// listInspectTemplates lists the inspect templates in the project.
func listInspectTemplates(w io.Writer, projectID string) error {
	// projectID := "my-project-id"

	ctx := context.Background()

	client, err := dlp.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("dlp.NewClient: %v", err)
	}

	// Create a configured request.
	req := &dlppb.ListInspectTemplatesRequest{
		Parent: fmt.Sprintf("projects/%s/locations/global", projectID),
	}

	// Send the request and iterate over the results.
	it := client.ListInspectTemplates(ctx, req)
	for {
		t, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			return fmt.Errorf("Next: %v", err)
		}
		fmt.Fprintf(w, "Inspect template %v\n", t.GetName())
		c, err := ptypes.Timestamp(t.GetCreateTime())
		if err != nil {
			return fmt.Errorf("CreateTime Timestamp: %v", err)
		}
		fmt.Fprintf(w, "  Created: %v\n", c.Format(time.RFC1123))
		u, err := ptypes.Timestamp(t.GetUpdateTime())
		if err != nil {
			return fmt.Errorf("UpdateTime Timestamp: %v", err)
		}
		fmt.Fprintf(w, "  Updated: %v\n", u.Format(time.RFC1123))
		fmt.Fprintf(w, "  Display Name: %q\n", t.GetDisplayName())
		fmt.Fprintf(w, "  Description: %q\n", t.GetDescription())
	}

	return nil
}

PHP

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

/**
 * List DLP inspection configuration templates.
 */
use Google\Cloud\Dlp\V2\DlpServiceClient;

/** Uncomment and populate these variables in your code */
// $callingProjectId = 'The project ID to run the API call under';

// Instantiate a client.
$dlp = new DlpServiceClient();

$parent = "projects/$callingProjectId/locations/global";

// Run request
$response = $dlp->listInspectTemplates($parent);

// Print results
$templates = $response->iterateAllElements();

foreach ($templates as $template) {
    printf('Template %s' . PHP_EOL, $template->getName());
    printf('  Created: %s' . PHP_EOL, $template->getCreateTime()->getSeconds());
    printf('  Updated: %s' . PHP_EOL, $template->getUpdateTime()->getSeconds());
    printf('  Display Name: %s' . PHP_EOL, $template->getDisplayName());
    printf('  Description: %s' . PHP_EOL, $template->getDescription());

    $inspectConfig = $template->getInspectConfig();
    if ($inspectConfig === null) {
        print('  No inspect config.' . PHP_EOL);
    } else {
        printf('  Minimum likelihood: %s' . PHP_EOL, $inspectConfig->getMinLikelihood());
        printf('  Include quotes: %s' . PHP_EOL, $inspectConfig->getIncludeQuote());
        $limits = $inspectConfig->getLimits();
        printf('  Max findings per request: %s' . PHP_EOL, $limits->getMaxFindingsPerRequest());
    }
}

C#

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

using Google.Api.Gax.ResourceNames;
using Google.Cloud.Dlp.V2;
using System;
using Google.Api.Gax;

public class InspectTemplateList
{
    public static PagedEnumerable<ListInspectTemplatesResponse, InspectTemplate> List(string projectId)
    {
        var client = DlpServiceClient.Create();

        var response = client.ListInspectTemplates(
            new ListInspectTemplatesRequest
            {
                Parent = new LocationName(projectId, "global").ToString(),
            }
        );

        // Uncomment to list templates
        //PrintTemplates(response);

        return response;
    }

    public static void PrintTemplates(PagedEnumerable<ListInspectTemplatesResponse, InspectTemplate> response)
    {
        foreach (var template in response)
        {
            Console.WriteLine($"Template {template.Name}:");
            Console.WriteLine($"\tDisplay Name: {template.DisplayName}");
            Console.WriteLine($"\tDescription: {template.Description}");
            Console.WriteLine($"\tCreated: {template.CreateTime}");
            Console.WriteLine($"\tUpdated: {template.UpdateTime}");
            Console.WriteLine("Configuration:");
            Console.WriteLine($"\tMin Likelihood: {template.InspectConfig?.MinLikelihood}");
            Console.WriteLine($"\tInclude quotes: {template.InspectConfig?.IncludeQuote}");
            Console.WriteLine($"\tMax findings per request: {template.InspectConfig?.Limits.MaxFindingsPerRequest}");
        }
    }
}

Deleting de-identification templates

To delete a specific de-identification template, use one of the *.*.delete methods:

• organizations.deidentifyTemplates.delete
• projects.deidentifyTemplates.delete

With each *.*.delete method, you include the resource name of the template to be deleted.

Code examples

Following is sample code in several languages that demonstrates how to use Cloud DLP to delete an inspection template. The process is identical to the one for listing de-identification templates.

Important: The code on this page requires that you first set up a Cloud DLP client. For more information about installing and creating a Cloud DLP client, see Cloud DLP client libraries. (Sending JSON to Cloud DLP REST endpoints does not require a client library.)

Java

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

import com.google.cloud.dlp.v2.DlpServiceClient;
import com.google.privacy.dlp.v2.DeleteInspectTemplateRequest;
import java.io.IOException;

class TemplatesDelete {

  public static void main(String[] args) throws Exception {
    // TODO(developer): Replace these variables before running the sample.
    String projectId = "your-project-id";
    String templateId = "your-template-id";
    deleteInspectTemplate(projectId, templateId);
  }

  // Delete an existing template
  public static void deleteInspectTemplate(String projectId, String templateId) throws IOException {
    // Construct the template name to be deleted
    String templateName = String.format("projects/%s/inspectTemplates/%s", projectId, templateId);

    // Initialize client that will be used to send requests. This client only needs to be created
    // once, and can be reused for multiple requests. After completing all of your requests, call
    // the "close" method on the client to safely clean up any remaining background resources.
    try (DlpServiceClient dlpServiceClient = DlpServiceClient.create()) {

      // Create delete template request to be sent by the client
      DeleteInspectTemplateRequest request =
          DeleteInspectTemplateRequest.newBuilder().setName(templateName).build();

      // Send the request with the client
      dlpServiceClient.deleteInspectTemplate(request);
      System.out.printf("Deleted template: %s\n", templateName);
    }
  }
}

Node.js

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The project ID to run the API call under
// const projectId = 'my-project';

// The name of the template to delete
// Parent project ID is automatically extracted from this parameter
// const templateName = 'projects/YOUR_PROJECT_ID/inspectTemplates/#####'
async function deleteInspectTemplate() {
  // Construct template-deletion request
  const request = {
    name: templateName,
  };

  // Run template-deletion request
  await dlp.deleteInspectTemplate(request);
  console.log(`Successfully deleted template ${templateName}.`);
}

deleteInspectTemplate();

Python

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

def delete_inspect_template(project, template_id):
    """Deletes a Data Loss Prevention API template.
    Args:
        project: The id of the Google Cloud project which owns the template.
        template_id: The id of the template to delete.
    Returns:
        None; the response from the API is printed to the terminal.
    """

    # Import the client library
    import google.cloud.dlp

    # Instantiate a client.
    dlp = google.cloud.dlp_v2.DlpServiceClient()

    # Convert the project id into a full resource id.
    parent = f"projects/{project}"

    # Combine the template id with the parent id.
    template_resource = "{}/inspectTemplates/{}".format(parent, template_id)

    # Call the API.
    dlp.delete_inspect_template(request={"name": template_resource})

    print("Template {} successfully deleted.".format(template_resource))

Go

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

import (
	"context"
	"fmt"
	"io"

	dlp "cloud.google.com/go/dlp/apiv2"
	dlppb "google.golang.org/genproto/googleapis/privacy/dlp/v2"
)

// deleteInspectTemplate deletes the given template.
func deleteInspectTemplate(w io.Writer, templateID string) error {
	// projectID := "my-project-id"
	// templateID := "my-template"

	ctx := context.Background()

	client, err := dlp.NewClient(ctx)
	if err != nil {
		return fmt.Errorf("dlp.NewClient: %v", err)
	}

	req := &dlppb.DeleteInspectTemplateRequest{
		Name: templateID,
	}

	if err := client.DeleteInspectTemplate(ctx, req); err != nil {
		return fmt.Errorf("DeleteInspectTemplate: %v", err)
	}
	fmt.Fprintf(w, "Successfully deleted inspect template %v", templateID)
	return nil
}

PHP

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

/**
 * Delete a DLP inspection configuration template.
 */
use Google\Cloud\Dlp\V2\DlpServiceClient;

/** Uncomment and populate these variables in your code */
// $callingProjectId = 'The project ID to run the API call under';
// $templateId = 'The name of the template to delete';

// Instantiate a client.
$dlp = new DlpServiceClient();

// Run template deletion request
$templateName = "projects/$callingProjectId/locations/global/inspectTemplates/$templateId";
$dlp->deleteInspectTemplate($templateName);

// Print results
printf('Successfully deleted template %s' . PHP_EOL, $templateName);

C#

To learn how to install and use the client library for Cloud DLP, see the Cloud DLP Client Libraries.

View on GitHub Feedback

using Google.Cloud.Dlp.V2;
using System;

public class InspectTemplateDelete
{
    public static object Delete(string projectId, string templateName)
    {
        var client = DlpServiceClient.Create();

        var request = new DeleteInspectTemplateRequest
        {
            Name = templateName
        };

        client.DeleteInspectTemplate(request);
        Console.WriteLine($"Successfully deleted template {templateName}.");

        return templateName;
    }
}