InfoTypes and infoType detectors

Cloud Data Loss Prevention (DLP) uses information types—or infoTypes—to define what it scans for. An infoType is a type of sensitive data, such as a name, email address, telephone number, identification number, credit card number, and so on. An infoType detector is the corresponding detection mechanism that matches on an infoType's matching criteria.

How to use infoTypes

Cloud DLP uses infoType detectors in the configuration for its scans to determine what to inspect for and how to transform findings. InfoType names are also used when displaying or reporting scan results.

For example, if you wanted to look for email addresses in a block of text, you would specify the EMAIL_ADDRESS infoType detector in the inspection configuration. If you wanted to redact email addresses from the text block, you would specify EMAIL_ADDRESS in both the inspection configuration and the de-identification configuration to indicate how to redact or transform that type.

Further, you could use a combination of built-in and custom infoType detectors to exclude a subset of email addresses from scan findings. First, create a custom infoType called INTERNAL_EMAIL_ADDRESS and configure it to exclude internal test email addresses. Then, you can set up your scan to include findings for EMAIL_ADDRESS, but include an exclusion rule that excludes any findings that match INTERNAL_EMAIL_ADDRESS. For more information about exclusion rules and other features of custom infoType detectors, see Creating custom infoType detectors.

Cloud DLP provides a set of built-in infoType detectors that you specify by name, each of which is listed in InfoType detector reference. These detectors use a variety of techniques to discover and classify each type. For example, some types will require a pattern match, some may have mathematical checksums, some have special digit restrictions, and others may have specific prefixes or context around the findings.

Example JSON

When you set up Cloud DLP to scan your content, you include the infoType detectors to use in the scan configuration.

For example, the following JSON demonstrates a simple scan request to the Cloud DLP API. Notice that the PHONE_NUMBER detector is specified in inspectConfig, which instructs Cloud DLP to scan the given string for a phone number.

JSON input:

POST https://dlp.googleapis.com/v2/projects/[PROJECT-ID]/content:inspect?key={YOUR_API_KEY}

{
  "item":{
    "value":"My phone number is (415) 555-0890"
  },
  "inspectConfig":{
    "includeQuote":true,
    "minLikelihood":"POSSIBLE",
    "infoTypes":{
      "name":"PHONE_NUMBER"
    }
  }
}

When you send the preceding request the specified endpoint, Cloud DLP returns the following:

JSON output:

{
  "result":{
    "findings":[
      {
        "quote":"(415) 555-0890",
        "infoType":{
          "name":"PHONE_NUMBER"
        },
        "likelihood":"VERY_LIKELY",
        "location":{
          "byteRange":{
            "start":"19",
            "end":"33"
          },
          "codepointRange":{
            "start":"19",
            "end":"33"
          }
        },
        "createTime":"2018-10-29T23:46:34.535Z"
      }
    ]
  }
}

You should always specify an infoType in your scan configuration. If you don't specify an infoType in your scan configuration, Cloud DLP defaults to the ALL_BASIC infoType detector, or the Most common option in the Cloud DLP UI in the GCP Console.

Depending on the amount of content to scan, scanning for ALL_BASIC or Most common can be prohibitively time-consuming or expensive.

For more information on how to use infoType detectors to scan your content, see one of the how-to topics about inspecting, redacting, or de-identifying.

Certainty and testing

Findings are reported with a certainty score called likelihood. The likelihood score indicates how likely a finding matches the corresponding type. For example, a type may return a lower likelihood if it only matches the pattern and return a higher likelihood if it matches the pattern and has positive context around it. For this reason, you may notice that a single finding could match several types at lower likelihood. Also, a finding may not appear or might have lower certainty if it doesn't match properly, or if it has negative context around it. For example, a finding might not reported if it matches the structure for the specified infoType but fails the infoType's checksum. Or a finding could match more than one infoType but have context that boosts one of them, and thus only get reported for that type.

If you are testing various detectors, you may notice that fake or sample data does not get reported because that fake or sample data is not passing enough checks to report.

Kinds of infoType detectors

Cloud DLP includes several kinds of infoType detectors, all of which are summarized here:

  • Built-in infoType detectors are built into Cloud DLP. They include detectors for country- or region-specific sensitive data types as well as globally applicable data types.
  • Custom infoType detectors are detectors that you create yourself. There are three kinds of custom infoType detectors:
    • Small custom dictionary detectors are simple word lists that Cloud DLP matches on. Use small custom dictionary detectors when you have a list of up to several tens of thousands of words or phrases. Small custom dictionary detectors are preferred if you don't anticipate your word list changing significantly.
    • Large custom dictionary detectors are generated by Cloud DLP using large lists of words or phrases stored in either Cloud Storage or BigQuery. Use large custom dictionary detectors when you have a large list of words or phrases—up to tens of millions.
    • Regular expressions (regex) detectors enable Cloud DLP to detect matches based on a regular expression pattern.

In addition, Cloud DLP includes the concept of inspection rules, which enable you to fine-tune scan results using the following:

  • Exclusion rules enable you to decrease the number of findings returned by adding rules to a built-in or custom infoType detector.
  • Hotword rules enable you to increase the quantity or change the likelihood value of findings returned by adding rules to a built-in or custom infoType detector.

Built-in infoType detectors

Built-in infoType detectors are built into Cloud DLP, and include detectors for country- or region-specific sensitive data types such as the French Numéro d'Inscription au Répertoire (NIR) (FRANCE_NIR), UK driver's license number (UK_DRIVERS_LICENSE_NUMBER), and US Social Security number (US_SOCIAL_SECURITY_NUMBER). They also include globally applicable data types such as a person name (PERSON_NAME), telephone numbers (PHONE_NUMBER), email addresses (EMAIL_ADDRESS), and credit card numbers (CREDIT_CARD_NUMBER). To detect content that corresponds to infoTypes, Cloud DLP leverages various techniques including pattern matching, checksums, machine-learning, context analysis, and others.

The list of built-in infoType detectors is always being updated. For a complete list of currently supported built-in infoType detectors, see InfoType detector reference.

You can also view a complete list of all built-in infoType detectors by calling Cloud DLP's infoTypes.list method.

Built-in infoType detectors are not a 100% accurate detection method. For example, they can't guarantee compliance with regulatory requirements. You must decide what data is sensitive and how to best protect it. Google recommends that you test your settings to make sure your configuration meets your requirements.

Custom infoType detectors

There are three kinds of custom infoType detectors:

In addition, Cloud DLP includes inspection rules, which enable you to fine-tune scan results by adding the following to existing detectors:

Small custom dictionary detectors

Use small custom dictionary detectors (also referred to as "regular custom dictionary detectors") to match a short (up to several tens of thousands) list of words or phrases. A small custom dictionary can act as its own unique detector.

Custom dictionary detectors are useful when you want to scan for a list of words or phrases that are not easily matched by a regular expression or a built-in detector. For example, suppose you want to scan for conference rooms that are commonly referred to by their assigned room names rather than their room numbers, such as state or region names, landmarks, fictional characters, and so on. You can make a small custom dictionary detector that contains a list of these room names. Cloud DLP can scan your content for each of the room names and return a match when it encounters one of them in context. Learn more about how Cloud DLP matches dictionary words and phrases in the "Dictionary matching specifics" section of Creating a regular custom dictionary detector.

For more details about how small dictionary custom infoType detectors work, as well as examples in action, see Creating a regular custom dictionary detector.

Large custom dictionary detectors

Use large custom dictionary detectors (also referred to as "stored custom dictionary detectors") when you have more than a few words or phrases to scan for, or if your list of words or phrases changes frequently. Large custom dictionary detectors can match on up to tens of millions of words or phrases.

Large custom dictionary detectors are created differently from both regular expression custom detectors and small custom dictionary detectors. Each large custom dictionary has two components:

  • A list of phrases that you create and define. The list is stored as either a text file within Cloud Storage or a column in a BigQuery table.
  • The generated dictionary files, which are built by Cloud DLP based on your phrase list. The dictionary files are stored in Cloud Storage, and are comprised of a copy of the source phrase data plus bloom filters, which aid in searching and matching. You can't edit these files directly.

Once you've created a word list and then used Cloud DLP to generate a custom dictionary, you initiate or schedule a scan using a large custom dictionary detector in a similar way as other infoType detectors.

For more details about how large custom dictionary detectors work, as well as examples in action, see Creating a stored custom dictionary detector.

Regular expressions

A regular expression (regex) custom infoType detector allows you to create your own infoType detectors that enable Cloud DLP to detect matches based on a regex pattern. For example, suppose that you had medical record numbers in the form ###-#-#####. You could define a regex pattern such as the following:

[1-9]{3}-[1-9]{1}-[1-9]{5}

The Cloud DLP would then match items like this:

123-4-56789

You can also specify a likelihood to assign to each custom infoType match. That is, when Cloud DLP matches the sequence you specify, it will assign the likelihood that you have indicated. This is useful because if your custom regex defines a sequence that is common enough it could easily match some other random sequence, you would not want Cloud DLP to label every match as VERY_LIKELY. Doing so would erode confidence in scan results and potentially cause the wrong information to be matched or de-identified.

For more information about regular expression custom infoType detectors, and to see them in action, see Creating a custom regex detector.

Inspection rules

You use inspection rules to refine the results returned by existing infoType detectors—either built-in or custom. Inspection rules can be useful for times when the results that Cloud DLP returns need to be augmented in some way, either by adding to and excluding from the existing infoType detector.

The two types of inspection rules are:

  • Exclusion rules
  • Hotword rules

For more information about inspection rules, see Modifying infoType detectors to refine scan results.

Exclusion rules

Exclusion rules enable you to decrease the quantity or precision of findings returned by adding rules to a built-in or custom infoType detector. Exclusion rules can help you reduce noise or other unwanted findings from being returned by an infoType detector.

For example, if you scan a database for email addresses, you can add an exclusion rule in the form of a custom regex that instructs Cloud DLP to exclude any findings ending in "@example.com."

For more information about exclusion rules, see Modifying infoType detectors to refine scan results.

Hotword rules

Hotword rules enable you to increase the quantity or accuracy of findings returned by adding rules to a built-in or custom infoType detector. Hotword rules can effectively help you loosen an existing infoType detector's rules.

For example, suppose you want to scan a medical database for patient names. You can use Cloud DLP's built-in PERSON_NAME infoType detector, but that will cause Cloud DLP to match on all names of people, not just names of patients. To fix this, you can include a hotword rule in the form of a regex custom infoType that looks for the word "patient" within a certain character proximity from the first character of potential matches. You can then assign findings matching this pattern a likelihood of "very likely," since they correspond to your special criteria.

For more information about hotword rules, see Modifying infoType detectors to refine scan results.

Examples

To get a better idea of how infoTypes match on findings, look at the following examples of matching on a series of digits to determine whether they constitute a US Social Security number or a US Individual Taxpayer Identification Number. Keep in mind that these examples are for built-in infoType detectors. When you create a custom infoType detector, you specify the criteria that determine the likelihood of a scan match.

Example 1

"SSN 222-22-2222"

Reports a high likelihood score of VERY_LIKELY for a US_SOCIAL_SECURITY_NUMBER because:

  • It is in the standard Social Security number format, which raises the certainty.
  • It has context nearby ("SSN") that boosts towards US_SOCIAL_SECURITY_NUMBER.

Example 2

"999-99-9999"

Reports a low likelihood score of VERY_UNLIKELY for a US_SOCIAL_SECURITY_NUMBER because:

  • It is in the standard format, which raises the certainty.
  • It starts with a 9, which is not allowed in Social Security numbers, thus lowers the certainty.
  • It lacks context, which lowers the certainty.

Example 3

"999-98-9999"

Reports a likelihood score of POSSIBLE for a US_INDIVIDUAL_TAXPAYER_IDENTIFICATION_NUMBER and VERY_UNLIKELY for US_SOCIAL_SECURITY_NUMBER because:

  • It has the standard format for both US_SOCIAL_SECURITY_NUMBER and US_INDIVIDUAL_TAXPAYER_IDENTIFICATION_NUMBER.
  • It starts with a 9 and has another digit check, which boosts certainty for US_INDIVIDUAL_TAXPAYER_IDENTIFICATION_NUMBER.
  • It lacks any context, which lowers the certainty for both.

What's next

The Cloud DLP team releases new infoType detectors and groups periodically. To learn how to get the latest list of built-in infoTypes, see [Listing built-in infoType detectors].

หน้านี้มีประโยชน์ไหม โปรดแสดงความคิดเห็น