This topic describes in detail how to create a new inspection template. For a quick walkthrough of how to create a new inspection template using the Google Cloud console, see Quickstart: Creating a Sensitive Data Protection inspection template.
About templates
You can use templates to create and persist configuration information to use with Sensitive Data Protection. Templates are useful for decoupling configuration information—such as what you inspect for and how you de-identify it—from the implementation of your requests. Templates provide a way to re-use configuration and enable consistency across users and datasets. In addition, whenever you update a template, it's updated for any job trigger that uses it.
Sensitive Data Protection supports inspection templates, which are discussed in this topic, and de-identification templates, which are discussed in Creating Sensitive Data Protection de-identification templates.
For conceptual information about templates in Sensitive Data Protection, see Templates.
Create a new inspection template
Console
In the Google Cloud console, go to the Create template page.
The Create template page contains the following sections:
Define template
Under Define template, enter an identifier for the inspection template. This is how you'll refer to the template when you run a job, create a job trigger, and so on. You can use letters, numbers, and hyphens. If you want, you can also enter a more human-friendly display name, as well as a description to better remember what the template does.
In the Resource location field, select the region where the data to be inspected is stored. The inspection template you create is also stored in this region. If you want to be able to use the new inspection template in any region, select Global (any region).
Configure detection
Next, you configure what Sensitive Data Protection detects in your content by choosing an infoType and other options.
InfoType detectors find sensitive data of a certain type. For example, the
Sensitive Data Protection US_SOCIAL_SECURITY_NUMBER
infoType detector finds
US Social Security numbers. In addition to the built-in infoType detectors, you
can create your own custom infoType detectors.
In the InfoTypes section, choose the infoType detector that corresponds to a data type you want to scan for. We don't recommend leaving this section blank. Doing so causes Sensitive Data Protection to scan your data with a default set of infoTypes, which might include infoTypes that you don't need. More information about each detector is provided in InfoType detector reference.
For more information about how to manage built-in and custom infoTypes in this section, see Manage infoTypes through the Google Cloud console.
Inspection rulesets
Inspection rulesets allow you to customize both built-in and custom infoType detectors using context rules. The two types of inspection rules are:
- Exclusion rules, which help exclude false or unwanted findings.
- Hotword rules, which help detect additional findings.
To add a new ruleset, first specify one or more built-in or custom infoType detectors in the InfoTypes section. These are the infoType detectors that your rulesets will be modifying. Then, do the following:
- Click in the Choose infoTypes field. The infoType or infoTypes you specified previously appear below the field in a menu, as shown here:
- Choose an infoType from the menu, and then click Add rule. A menu appears with the two options Hotword rule and Exclusion rule.
For hotword rules, choose Hotword rules. Then, do the following:
- In the Hotword field, enter a regular expression that Sensitive Data Protection should look for.
- From the Hotword proximity menu, choose whether the hotword you entered is found before or after the chosen infoType.
- In Hotword distance from infoType, enter the approximate number of characters between the hotword and the chosen infoType.
- In Confidence level adjustment, choose whether to assign matches a fixed likelihood level, or to increase or decrease the default likelihood level by a certain amount.
For exclusion rules, choose Exclusion rules. Then, do the following:
- In the Exclude field, enter a regular expression (regex) that Sensitive Data Protection should look for.
- From the Matching type menu, choose one of the following:
- Full match: The finding must completely match the regex.
- Partial match: A substring of the finding can match the regex.
- Inverse match: The finding doesn't match the regex.
You can add additional hotword or exclusion rules and rulesets to further refine your scan results.
Confidence threshold
Every time Sensitive Data Protection detects a potential match for sensitive data, it assigns it a likelihood value on a scale from "Very unlikely" to "Very likely." When you set a likelihood value here, you are instructing Sensitive Data Protection to only match on data that corresponds to that likelihood value or higher.
The default value of "Possible" is sufficient for most purposes. If you routinely get matches that are too broad, move the slider up. If you get too few matches, move the slider down.
When you're done, click Create to create the template. The template's summary information page appears.
To return to the main Sensitive Data Protection page, click the Back arrow in the Google Cloud console.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Node.js
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
PHP
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Python
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
REST
An inspection template is a reusable inspection configuration plus some
metadata. In API terms, the
InspectTemplate
object is effectively an
InspectConfig
object that includes a few more fields of metadata, such as a display name and
a description. Therefore, to create a new inspection template, the basic steps
are:
- Start with an
InspectConfig
object. - Call or POST the
create
method of either theprojects.inspectTemplates
ororganizations.inspectTemplates
resource, including in your request anInspectTemplate
object that contains a display name, a description, and thatInspectConfig
object.
The returned InspectTemplate
will be ready for use immediately. You can
reference it in other calls or jobs by its name
. You can list the existing
templates by calling the *.inspectTemplates.list
method. To view a specific
template, call the *.inspectTemplates.get
method. Note that the limit for the
number of templates you can create is 1000.
If you've already had some experience inspecting text, images, or structured
content for sensitive content using Sensitive Data Protection, you've already
created an
InspectConfig
object. One additional step turns it into an
InspectTemplate
object.
The following JSON is an example of what you can send to the
projects.inspectTemplates.create
method. This JSON creates a new template with the given display name and
description, and scans for matches on the infoTypes PHONE_NUMBER
and
US_TOLLFREE_PHONE_NUMBER
. It will include in its findings up to 100 matches
whose likelihoods are at least POSSIBLE
, and will include a snippet of
context for each.
JSON input:
POST https://dlp.googleapis.com/v2/projects/[PROJECT_ID]/inspectTemplates?key={YOUR_API_KEY}
{
"inspectTemplate":{
"displayName":"Phone number inspection",
"description":"Scans for phone numbers",
"inspectConfig":{
"infoTypes":[
{
"name":"PHONE_NUMBER"
},
{
"name":"US_TOLLFREE_PHONE_NUMBER"
}
],
"minLikelihood":"POSSIBLE",
"limits":{
"maxFindingsPerRequest":100
},
"includeQuote":true
}
}
}
JSON output:
The response JSON looks like the following:
{
"name":"projects/[PROJECT_ID]/inspectTemplates/[JOB_ID]",
"displayName":"Phone number inspection",
"description":"Scans for phone numbers",
"createTime":"2018-11-30T07:26:28.164136Z",
"updateTime":"2018-11-30T07:26:28.164136Z",
"inspectConfig":{
"infoTypes":[
{
"name":"PHONE_NUMBER"
},
{
"name":"US_TOLLFREE_PHONE_NUMBER"
}
],
"minLikelihood":"POSSIBLE",
"limits":{
"maxFindingsPerRequest":100
},
"includeQuote":true
}
}
To quickly try this out, you can use the APIs Explorer that's embedded below. For general information about using JSON to send requests to the DLP API, see the JSON quickstart.
Use inspection templates
After you create a new inspection template, you can use it when creating a new inspection job or job trigger. Whenever you update that template, it's updated in any job trigger that uses it. For more information, including code samples, see:
- Inspecting storage and databases for sensitive data
- Creating and scheduling Sensitive Data Protection inspection jobs
Console
To get started quickly using your new template, follow the instructions provided in Quickstart creating a Sensitive Data Protection inspection template with the following change:
- In the Configure detection > Templates section, click in the Template name field and select the template you just created.
For a more in-depth walkthrough of how to scan your content, see Creating and scheduling Sensitive Data Protection inspection jobs, paying particular attention to the "Configure detection" section.
REST
You can use the template identifier you specified when creating the template
anywhere inspectTemplateName
is accepted, such as:
projects.content.inspect
: Finds potentially sensitive data in content using the template as its configuration.projects.content.deidentify
: Finds and de-identifies potentially sensitive data in content using the template as its configuration. Be aware that this method uses both an inspection template and a de-identification template.projects.dlpJobs.create
, in theInspectJobConfig
object: Creates an inspection job that includes the template as its configuration.
List inspection templates
To list all inspection templates that have been created in the current project or organization:
Console
In the Google Cloud console, go to the Configuration page of Sensitive Data Protection.
Select the project that contains the inspection templates.
Click the Templates tab.
The console displays a list of all inspection templates for the current project.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Node.js
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
PHP
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Python
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
REST
Use one of the *.*.list
methods:
Copy an inspection template to the global
region
In the Google Cloud console, go to the Sensitive Data Protection Configuration page.
On the toolbar, click the project selector and select the project that contains the inspection template that you want to use.
Click the Templates tab, and then click the Inspect subtab.
Click the ID of the template that you want to use.
On the Inspection template details page, click Copy.
On the Create template page, in the Resource location list, select Global (any region).
Click Create.
The template is copied to the global
region.
Copy an inspection template to another project
In the Google Cloud console, go to the Sensitive Data Protection Configuration page.
On the toolbar, click the project selector and select the project that contains the inspection template that you want to use.
Click the Templates tab, and then click the Inspect subtab.
Click the ID of the template that you want to use.
On the Inspection template details page, click Copy.
Select the project that you want to copy the inspection template to.
The Create template page reloads in the project that you selected.
Click Create.
The template is created in the project that you selected.
Delete inspection templates
To delete an inspection template:
Console
In the Google Cloud console, go to the Configuration page of Sensitive Data Protection.
Select the project that contains the inspection template that you want to delete.
Click the Templates tab. The console displays a list of all templates for the current project.
In the Actions column for the template you want to delete, click the more actions menu (displayed as three dots arranged vertically)
, and then click Delete.
Alternatively, from the list of templates, click the name of the template you want to delete. On the template's detail page, click Delete.
C#
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Go
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Java
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Node.js
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
PHP
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
Python
To learn how to install and use the client library for Sensitive Data Protection, see Sensitive Data Protection client libraries.
To authenticate to Sensitive Data Protection, set up Application Default Credentials. For more information, see Set up authentication for a local development environment.
REST
Use one of the *.*.delete
methods:
With each *.*.delete
method, you include the resource name of the template to
be deleted.