Creating Cloud DLP de-identification templates

You can use templates to create and persist configuration information to use with Cloud Data Loss Prevention (DLP). Templates are useful for decoupling configuration such as what you inspect for and how you de-identify it from the implementation of your requests. Templates provide a robust way to re-use configuration and enable consistency across users and data sets.

The Cloud DLP supports two types of templates:

  • De-identification templates: Templates for saving configuration information for de-identification jobs, including both infoType and record (structured dataset) transformations.
  • Inspection templates: Templates for persisting configuration information for inspection scan jobs, including what predefined or custom detectors to use. For more information about inspection templates, see Creating Cloud DLP inspection templates.

For conceptual information about templates in Cloud DLP, see Templates.

The remainder of this topic shows you how to create de-identification templates for use with the Cloud DLP.

Create a de-identification template

It’s useful to note that a de-identification template is simply a reusable de-identification configuration plus some metadata. In API terms, the DeidentifyTemplate object is effectively a DeidentifyConfig object that includes a few more fields of metadata, such as a display name and a description. Therefore, to create a new de-identification template, the basic steps are:

  1. Start with a DeidentifyConfig object.
  2. Call or POST the create method of either the projects.deidentifyTemplates or organizations.deidentifyTemplates resource, including in your request a DeidentifyTemplate object that contains a display name, a description, and that DeidentifyConfig object.

The returned DeidentifyTemplate will be usable immediately. You can reference it in other calls or jobs by its name. You can list the existing templates by calling the *.deidentifyTemplates.list method. To view a specific template, call the *.deidentifyTemplates.get method. Note that the limit for the number of templates you can create is 1000.

If you've already had some experience de-identifying sensitive data from text, images, or structured content using Cloud DLP, you've already created a DeidentifyConfig object. One additional step turns it into a DeidentifyTemplate object.

REST example

The following JSON is an example of what you could send to the projects.deidentifyTemplates.create method. (Try it for yourself in APIs Explorer.) This JSON creates a new template with the given display name and description, and scans for matches on the infoTypes EMAIL_ADDRESS and LAST_NAME. When it finds content matching those infoTypes, it will mask the first three characters with an asterisk (*) character:

JSON Input:

POST https://dlp.googleapis.com/v2/projects/[PROJECT_ID]/deidentifyTemplates?key={YOUR_API_KEY}

{
  "deidentifyTemplate":{
    "displayName":"Email last name mask",
    "description":"De-identifies emails and last names with a series of asterisks.",
    "deidentifyConfig":{
      "infoTypeTransformations":{
        "transformations":[
          {
            "infoTypes":[
              {
                "name":"LAST_NAME"
              }
            ],
            "primitiveTransformation":{
              "replaceWithInfoTypeConfig":{

              }
            }
          },
          {
            "infoTypes":[
              {
                "name":"EMAIL_ADDRESS"
              }
            ],
            "primitiveTransformation":{
              "characterMaskConfig":{
                "charactersToIgnore":[
                  {
                    "charactersToSkip":"@"
                  }
                ],
                "maskingCharacter":"*"
              }
            }
          }
        ]
      }
    }
  }
}

JSON Output:

{
  "name":"projects/[PROJECT_ID]/deidentifyTemplates/[JOB-ID]",
  "displayName":"Email last name mask",
  "description":"De-identifies emails and last names with a series of asterisks.",
  "createTime":"2018-11-30T07:17:59.536022Z",
  "updateTime":"2018-11-30T07:17:59.536022Z",
  "deidentifyConfig":{
    "infoTypeTransformations":{
      "transformations":[
        {
          "infoTypes":[
            {
              "name":"LAST_NAME"
            }
          ],
          "primitiveTransformation":{
            "replaceWithInfoTypeConfig":{

            }
          }
        },
        {
          "infoTypes":[
            {
              "name":"EMAIL_ADDRESS"
            }
          ],
          "primitiveTransformation":{
            "characterMaskConfig":{
              "maskingCharacter":"*",
              "charactersToIgnore":[
                {
                  "charactersToSkip":"@"
                }
              ]
            }
          }
        }
      ]
    }
  }
}

Using de-identification templates

Once you’ve created a new template, you can use its name identifier anywhere deidentifyTemplateName is accepted, such as:

  • projects.content.deidentify: De-identifies potentially sensitive data in content using the template as its configuration. Be aware that this method can use either an inspection template or a de-identification template.

Listing de-identification templates

To list all de-identification templates that have been created, use one of the *.*.list methods:

Code examples

Following is sample code in several languages that demonstrates how to use Cloud DLP to list inspection templates. The process is identical to the one for listing de-identification templates.

Java

/**
 * List DLP inspection templates created in a given project
 *
 * @param projectId Google Cloud Project ID
 */
private static void listInspectTemplates(String projectId) {

  try (DlpServiceClient dlpServiceClient = DlpServiceClient.create()) {

    ListInspectTemplatesRequest request =
        ListInspectTemplatesRequest.newBuilder()
            .setParent(ProjectName.of(projectId).toString())
            .setPageSize(1)
            .build();

    ListInspectTemplatesPagedResponse response = dlpServiceClient.listInspectTemplates(request);
    ListInspectTemplatesPage page = response.getPage();
    ListInspectTemplatesResponse templatesResponse = page.getResponse();

    for (InspectTemplate template : templatesResponse.getInspectTemplatesList()) {
      System.out.printf("Template name: %s\n", template.getName());
      if (template.getDisplayName() != null) {
        System.out.printf("\tDisplay name: %s \n", template.getDisplayName());
        System.out.printf("\tCreate time: %s \n", template.getCreateTime());
        System.out.printf("\tUpdate time: %s \n", template.getUpdateTime());

        // print inspection config
        InspectConfig inspectConfig = template.getInspectConfig();
        for (InfoType infoType : inspectConfig.getInfoTypesList()) {
          System.out.printf("\tInfoType: %s\n", infoType.getName());
        }
        System.out.printf("\tMin likelihood: %s\n", inspectConfig.getMinLikelihood());
        System.out.printf("\tLimits: %s\n", inspectConfig.getLimits().getMaxFindingsPerRequest());
      }
    }
  } catch (Exception e) {
    System.out.printf("Error creating template: %s", e.getMessage());
  }
}

Node.js

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The project ID to run the API call under
// const callingProjectId = process.env.GCLOUD_PROJECT;

// Helper function to pretty-print dates
const formatDate = date => {
  const msSinceEpoch = parseInt(date.seconds, 10) * 1000;
  return new Date(msSinceEpoch).toLocaleString('en-US');
};

// Construct template-listing request
const request = {
  parent: dlp.projectPath(callingProjectId),
};

try {
  // Run template-deletion request
  const [templates] = await dlp.listInspectTemplates(request);

  templates.forEach(template => {
    console.log(`Template ${template.name}`);
    if (template.displayName) {
      console.log(`  Display name: ${template.displayName}`);
    }

    console.log(`  Created: ${formatDate(template.createTime)}`);
    console.log(`  Updated: ${formatDate(template.updateTime)}`);

    const inspectConfig = template.inspectConfig;
    const infoTypes = inspectConfig.infoTypes.map(x => x.name);
    console.log(`  InfoTypes:`, infoTypes.join(' '));
    console.log(`  Minimum likelihood:`, inspectConfig.minLikelihood);
    console.log(`  Include quotes:`, inspectConfig.includeQuote);

    const limits = inspectConfig.limits;
    console.log(`  Max findings per request:`, limits.maxFindingsPerRequest);
  });
} catch (err) {
  console.log(`Error in listInspectTemplates: ${err.message || err}`);
}

Python

def list_inspect_templates(project):
    """Lists all Data Loss Prevention API inspect templates.
    Args:
        project: The Google Cloud project id to use as a parent resource.
    Returns:
        None; the response from the API is printed to the terminal.
    """

    # Import the client library
    import google.cloud.dlp

    # Instantiate a client.
    dlp = google.cloud.dlp.DlpServiceClient()

    # Convert the project id into a full resource id.
    parent = dlp.project_path(project)

    # Call the API.
    response = dlp.list_inspect_templates(parent)

    # Define a helper function to convert the API's "seconds since the epoch"
    # time format into a human-readable string.
    def human_readable_time(timestamp):
        return str(time.localtime(timestamp.seconds))

    for template in response:
        print('Template {}:'.format(template.name))
        if template.display_name:
            print('  Display Name: {}'.format(template.display_name))
        print('  Created: {}'.format(
            human_readable_time(template.create_time)))
        print('  Updated: {}'.format(
            human_readable_time(template.update_time)))

        config = template.inspect_config
        print('  InfoTypes: {}'.format(', '.join(
            [it.name for it in config.info_types]
        )))
        print('  Minimum likelihood: {}'.format(config.min_likelihood))
        print('  Include quotes: {}'.format(config.include_quote))
        print('  Max findings per request: {}'.format(
            config.limits.max_findings_per_request))

Go


// listInspectTemplates lists the inspect templates in the project.
func listInspectTemplates(w io.Writer, client *dlp.Client, project string) {
	// Create a configured request.
	req := &dlppb.ListInspectTemplatesRequest{
		Parent: "projects/" + project,
	}
	// Send the request and iterate over the results.
	it := client.ListInspectTemplates(context.Background(), req)
	for {
		t, err := it.Next()
		if err == iterator.Done {
			break
		}
		if err != nil {
			log.Fatalf("error getting inspect templates: %v", err)
		}
		c := t.GetCreateTime()
		u := t.GetUpdateTime()
		fmt.Fprintf(w, "Inspect template %v\n", t.GetName())
		fmt.Fprintf(w, "  Created: %v\n", time.Unix(c.GetSeconds(), int64(c.GetNanos())).Format(time.RFC1123))
		fmt.Fprintf(w, "  Updated: %v\n", time.Unix(u.GetSeconds(), int64(u.GetNanos())).Format(time.RFC1123))
		fmt.Fprintf(w, "  Display Name: %q\n", t.GetDisplayName())
		fmt.Fprintf(w, "  Description: %q\n", t.GetDescription())
	}
}

PHP

/**
 * List DLP inspection configuration templates.
 */
use Google\Cloud\Dlp\V2\DlpServiceClient;

/** Uncomment and populate these variables in your code */
// $callingProjectId = 'The project ID to run the API call under';

// Instantiate a client.
$dlp = new DlpServiceClient();

$parent = $dlp->projectName($callingProjectId);

// Run request
$response = $dlp->listInspectTemplates($parent);

// Print results
$templates = $response->iterateAllElements();

foreach ($templates as $template) {
    printf('Template %s' . PHP_EOL, $template->getName());
    printf('  Created: %s' . PHP_EOL, $template->getCreateTime()->getSeconds());
    printf('  Updated: %s' . PHP_EOL, $template->getUpdateTime()->getSeconds());
    printf('  Display Name: %s' . PHP_EOL, $template->getDisplayName());
    printf('  Description: %s' . PHP_EOL, $template->getDescription());

    $inspectConfig = $template->getInspectConfig();
    if ($inspectConfig === null) {
        print('  No inspect config.' . PHP_EOL);
    } else {
        printf('  Minimum likelihood: %s' . PHP_EOL, $inspectConfig->getMinLikelihood());
        printf('  Include quotes: %s' . PHP_EOL, $inspectConfig->getIncludeQuote());
        $limits = $inspectConfig->getLimits();
        printf('  Max findings per request: %s' . PHP_EOL, $limits->getMaxFindingsPerRequest());
    }
}

C#

public static object ListInspectTemplate(string projectId)
{
    DlpServiceClient client = DlpServiceClient.Create();

    var response = client.ListInspectTemplates(
        new ListInspectTemplatesRequest
        {
            Parent = $"projects/{projectId}",
        }
    );
    foreach (var template in response)
    {
        Console.WriteLine($"Template {template.Name}:");
        Console.WriteLine($"\tDisplay Name: {template.DisplayName}");
        Console.WriteLine($"\tDescription: {template.Description}");
        Console.WriteLine($"\tCreated: {template.CreateTime}");
        Console.WriteLine($"\tUpdated: {template.UpdateTime}");
        Console.WriteLine("Configuration:");
        Console.WriteLine($"\tMin Likelihood: {template.InspectConfig?.MinLikelihood}");
        Console.WriteLine($"\tInclude quotes: {template.InspectConfig?.IncludeQuote}");
        Console.WriteLine($"\tMax findings per request: {template.InspectConfig?.Limits.MaxFindingsPerRequest}");
    }

    return null;
}

Deleting de-identification templates

To delete a specific de-identification template, use one of the *.*.delete methods:

With each *.*.delete method, you include the resource name of the template to be deleted.

Code examples

Following is sample code in several languages that demonstrates how to use Cloud DLP to delete an inspection template. The process is identical to the one for listing de-identification templates.

Java

/**
 * Delete the DLP inspection configuration template with the specified name.
 *
 * @param projectId Google Cloud Project ID
 * @param templateId Template ID to be deleted
 */
private static void deleteInspectTemplate(String projectId, String templateId) {

  // construct the template name to be deleted
  String templateName = String.format("projects/%s/inspectTemplates/%s", projectId, templateId);
  // instantiate the client
  try (DlpServiceClient dlpServiceClient = DlpServiceClient.create()) {
    // create delete template request
    DeleteInspectTemplateRequest request =
        DeleteInspectTemplateRequest.newBuilder().setName(templateName).build();

    dlpServiceClient.deleteInspectTemplate(request);
    System.out.printf("Deleted template: %s\n", templateName);
  } catch (Exception e) {
    System.err.printf("Error deleting template: %s\n", templateName);
  }
}

Node.js

// Imports the Google Cloud Data Loss Prevention library
const DLP = require('@google-cloud/dlp');

// Instantiates a client
const dlp = new DLP.DlpServiceClient();

// The name of the template to delete
// Parent project ID is automatically extracted from this parameter
// const templateName = 'projects/YOUR_PROJECT_ID/inspectTemplates/#####'

// Construct template-deletion request
const request = {
  name: templateName,
};

try {
  // Run template-deletion request
  await dlp.deleteInspectTemplate(request);
  console.log(`Successfully deleted template ${templateName}.`);
} catch (err) {
  console.log(`Error in deleteInspectTemplate: ${err.message || err}`);
}

Python

def delete_inspect_template(project, template_id):
    """Deletes a Data Loss Prevention API template.
    Args:
        project: The id of the Google Cloud project which owns the template.
        template_id: The id of the template to delete.
    Returns:
        None; the response from the API is printed to the terminal.
    """

    # Import the client library
    import google.cloud.dlp

    # Instantiate a client.
    dlp = google.cloud.dlp.DlpServiceClient()

    # Convert the project id into a full resource id.
    parent = dlp.project_path(project)

    # Combine the template id with the parent id.
    template_resource = '{}/inspectTemplates/{}'.format(parent, template_id)

    # Call the API.
    dlp.delete_inspect_template(template_resource)

    print('Template {} successfully deleted.'.format(template_resource))

Go


// deleteInspectTemplate deletes the given template.
func deleteInspectTemplate(w io.Writer, client *dlp.Client, templateID string) {
	req := &dlppb.DeleteInspectTemplateRequest{
		Name: templateID,
	}
	err := client.DeleteInspectTemplate(context.Background(), req)
	if err != nil {
		log.Fatalf("error deleting inspect template: %v", err)
	}
	fmt.Fprintf(w, "Successfully deleted inspect template %v", templateID)
}

PHP

/**
 * Delete a DLP inspection configuration template.
 */
use Google\Cloud\Dlp\V2\DlpServiceClient;

/** Uncomment and populate these variables in your code */
// $callingProjectId = 'The project ID to run the API call under';
// $templateId = 'The name of the template to delete';

// Instantiate a client.
$dlp = new DlpServiceClient();

// Run template deletion request
$templateName = $dlp->projectInspectTemplateName($callingProjectId, $templateId);
$dlp->deleteInspectTemplate($templateName);

// Print results
printf('Successfully deleted template %s' . PHP_EOL, $templateName);

C#

public static object DeleteInspectTemplate(string projectId, string templateName)
{
    DlpServiceClient client = DlpServiceClient.Create();

    var request = new DeleteInspectTemplateRequest
    {
        Name = templateName
    };

    client.DeleteInspectTemplate(request);
    Console.WriteLine($"Successfully deleted template {templateName}.");

    return templateName;
}

Was this page helpful? Let us know how we did:

Send feedback about...

Cloud Data Loss Prevention