REST Resource: organizations.deidentifyTemplates

Resource: DeidentifyTemplate

The DeidentifyTemplates contains instructions on how to deidentify content. See https://cloud.google.com/dlp/docs/concepts-templates to learn more.

JSON representation
{
  "name": string,
  "displayName": string,
  "description": string,
  "createTime": string,
  "updateTime": string,
  "deidentifyConfig": {
    object(DeidentifyConfig)
  }
}
Fields
name

string

The template name. Output only.

The template will have one of the following formats: projects/PROJECT_ID/deidentifyTemplates/TEMPLATE_ID OR organizations/ORGANIZATION_ID/deidentifyTemplates/TEMPLATE_ID

displayName

string

Display name (max 256 chars).

description

string

Short description (max 256 chars).

createTime

string (Timestamp format)

The creation timestamp of a inspectTemplate, output only field.

A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2014-10-02T15:01:23.045123456Z".

updateTime

string (Timestamp format)

The last update timestamp of a inspectTemplate, output only field.

A timestamp in RFC3339 UTC "Zulu" format, accurate to nanoseconds. Example: "2014-10-02T15:01:23.045123456Z".

deidentifyConfig

object(DeidentifyConfig)

///////////// // The core content of the template // ///////////////

DeidentifyConfig

The configuration that controls how the data will change.

JSON representation
{

  // Union field transformation can be only one of the following:
  "infoTypeTransformations": {
    object(InfoTypeTransformations)
  },
  "recordTransformations": {
    object(RecordTransformations)
  }
  // End of list of possible types for union field transformation.
}
Fields

Union field transformation.

transformation can be only one of the following:

infoTypeTransformations

object(InfoTypeTransformations)

Treat the dataset as free-form text and apply the same free text transformation everywhere.

recordTransformations

object(RecordTransformations)

Treat the dataset as structured. Transformations can be applied to specific locations within structured datasets, such as transforming a column within a table.

InfoTypeTransformations

A type of transformation that will scan unstructured text and apply various PrimitiveTransformations to each finding, where the transformation is applied to only values that were identified as a specific infoType.

JSON representation
{
  "transformations": [
    {
      object(InfoTypeTransformation)
    }
  ]
}
Fields
transformations[]

object(InfoTypeTransformation)

Transformation for each infoType. Cannot specify more than one for a given infoType. [required]

InfoTypeTransformation

A transformation to apply to text that is identified as a specific infoType.

JSON representation
{
  "infoTypes": [
    {
      object(InfoType)
    }
  ],
  "primitiveTransformation": {
    object(PrimitiveTransformation)
  }
}
Fields
infoTypes[]

object(InfoType)

InfoTypes to apply the transformation to. An empty list will cause this transformation to apply to all findings that correspond to infoTypes that were requested in InspectConfig.

primitiveTransformation

object(PrimitiveTransformation)

Primitive transformation to apply to the infoType. [required]

PrimitiveTransformation

A rule for transforming a value.

JSON representation
{

  // Union field transformation can be only one of the following:
  "replaceConfig": {
    object(ReplaceValueConfig)
  },
  "redactConfig": {
    object(RedactConfig)
  },
  "characterMaskConfig": {
    object(CharacterMaskConfig)
  },
  "cryptoReplaceFfxFpeConfig": {
    object(CryptoReplaceFfxFpeConfig)
  },
  "fixedSizeBucketingConfig": {
    object(FixedSizeBucketingConfig)
  },
  "bucketingConfig": {
    object(BucketingConfig)
  },
  "replaceWithInfoTypeConfig": {
    object(ReplaceWithInfoTypeConfig)
  },
  "timePartConfig": {
    object(TimePartConfig)
  },
  "cryptoHashConfig": {
    object(CryptoHashConfig)
  },
  "dateShiftConfig": {
    object(DateShiftConfig)
  }
  // End of list of possible types for union field transformation.
}
Fields

Union field transformation.

transformation can be only one of the following:

replaceConfig

object(ReplaceValueConfig)

redactConfig

object(RedactConfig)

characterMaskConfig

object(CharacterMaskConfig)

cryptoReplaceFfxFpeConfig

object(CryptoReplaceFfxFpeConfig)

fixedSizeBucketingConfig

object(FixedSizeBucketingConfig)

bucketingConfig

object(BucketingConfig)

replaceWithInfoTypeConfig

object(ReplaceWithInfoTypeConfig)

timePartConfig

object(TimePartConfig)

cryptoHashConfig

object(CryptoHashConfig)

dateShiftConfig

object(DateShiftConfig)

ReplaceValueConfig

Replace each input value with a given Value.

JSON representation
{
  "newValue": {
    object(Value)
  }
}
Fields
newValue

object(Value)

Value to replace it with.

RedactConfig

Redact a given value. For example, if used with an InfoTypeTransformation transforming PHONE_NUMBER, and input 'My phone number is 206-555-0123', the output would be 'My phone number is '.

CharacterMaskConfig

Partially mask a string by replacing a given number of characters with a fixed character. Masking can start from the beginning or end of the string. This can be used on data of any type (numbers, longs, and so on) and when de-identifying structured data we'll attempt to preserve the original data's type. (This allows you to take a long like 123 and modify it to a string like **3.

JSON representation
{
  "maskingCharacter": string,
  "numberToMask": number,
  "reverseOrder": boolean,
  "charactersToIgnore": [
    {
      object(CharsToIgnore)
    }
  ]
}
Fields
maskingCharacter

string

Character to mask the sensitive values—for example, "*" for an alphabetic string such as name, or "0" for a numeric string such as ZIP code or credit card number. String must have length 1. If not supplied, we will default to "*" for strings, 0 for digits.

numberToMask

number

Number of characters to mask. If not set, all matching chars will be masked. Skipped characters do not count towards this tally.

reverseOrder

boolean

Mask characters in reverse order. For example, if maskingCharacter is '0', numberToMask is 14, and reverseOrder is false, then 1234-5678-9012-3456 -> 00000000000000-3456 If maskingCharacter is '*', numberToMask is 3, and reverseOrder is true, then 12345 -> 12***

charactersToIgnore[]

object(CharsToIgnore)

When masking a string, items in this list will be skipped when replacing. For example, if your string is 555-555-5555 and you ask us to skip - and mask 5 chars with * we would produce ***-*55-5555.

CharsToIgnore

Characters to skip when doing deidentification of a value. These will be left alone and skipped.

JSON representation
{

  // Union field characters can be only one of the following:
  "charactersToSkip": string,
  "commonCharactersToIgnore": enum(CommonCharsToIgnore)
  // End of list of possible types for union field characters.
}
Fields

Union field characters.

characters can be only one of the following:

charactersToSkip

string

commonCharactersToIgnore

enum(CommonCharsToIgnore)

CommonCharsToIgnore

Enums
COMMON_CHARS_TO_IGNORE_UNSPECIFIED
NUMERIC 0-9
ALPHA_UPPER_CASE A-Z
ALPHA_LOWER_CASE a-z
PUNCTUATION US Punctuation, one of !"#$%&'()*+,-./:;<=>?@[]^_`{|}~
WHITESPACE Whitespace character, one of [ \t\n\x0B\f\r]

CryptoReplaceFfxFpeConfig

Replaces an identifier with a surrogate using FPE with the FFX mode of operation; however when used in the content.reidentify API method, it serves the opposite function by reversing the surrogate back into the original identifier. The identifier must be encoded as ASCII. For a given crypto key and context, the same identifier will be replaced with the same surrogate. Identifiers must be at least two characters long. In the case that the identifier is the empty string, it will be skipped. See https://cloud.google.com/dlp/docs/pseudonymization to learn more.

JSON representation
{
  "cryptoKey": {
    object(CryptoKey)
  },
  "context": {
    object(FieldId)
  },
  "surrogateInfoType": {
    object(InfoType)
  },

  // Union field alphabet can be only one of the following:
  "commonAlphabet": enum(FfxCommonNativeAlphabet),
  "customAlphabet": string,
  "radix": number
  // End of list of possible types for union field alphabet.
}
Fields
cryptoKey

object(CryptoKey)

The key used by the encryption algorithm. [required]

context

object(FieldId)

The 'tweak', a context may be used for higher security since the same identifier in two different contexts won't be given the same surrogate. If the context is not set, a default tweak will be used.

If the context is set but:

  1. there is no record present when transforming a given value or
  2. the field is not present when transforming a given value,

a default tweak will be used.

Note that case (1) is expected when an InfoTypeTransformation is applied to both structured and non-structured ContentItems. Currently, the referenced field may be of value type integer or string.

The tweak is constructed as a sequence of bytes in big endian byte order such that:

  • a 64 bit integer is encoded followed by a single byte of value 1
  • a string is encoded in UTF-8 format followed by a single byte of value 2

surrogateInfoType

object(InfoType)

The custom infoType to annotate the surrogate with. This annotation will be applied to the surrogate by prefixing it with the name of the custom infoType followed by the number of characters comprising the surrogate. The following scheme defines the format: info_type_name(surrogate_character_count):surrogate

For example, if the name of custom infoType is 'MY_TOKEN_INFO_TYPE' and the surrogate is 'abc', the full replacement value will be: 'MY_TOKEN_INFO_TYPE(3):abc'

This annotation identifies the surrogate when inspecting content using the custom infoType SurrogateType. This facilitates reversal of the surrogate when it occurs in free text.

In order for inspection to work properly, the name of this infoType must not occur naturally anywhere in your data; otherwise, inspection may find a surrogate that does not correspond to an actual identifier. Therefore, choose your custom infoType name carefully after considering what your data looks like. One way to select a name that has a high chance of yielding reliable detection is to include one or more unicode characters that are highly improbable to exist in your data. For example, assuming your data is entered from a regular ASCII keyboard, the symbol with the hex code point 29DD might be used like so: ⧝MY_TOKEN_TYPE

Union field alphabet.

alphabet can be only one of the following:

commonAlphabet

enum(FfxCommonNativeAlphabet)

customAlphabet

string

This is supported by mapping these to the alphanumeric characters that the FFX mode natively supports. This happens before/after encryption/decryption. Each character listed must appear only once. Number of characters must be in the range [2, 62]. This must be encoded as ASCII. The order of characters does not matter.

radix

number

The native way to select the alphabet. Must be in the range [2, 62].

CryptoKey

This is a data encryption key (DEK) (as opposed to a key encryption key (KEK) stored by KMS). When using KMS to wrap/unwrap DEKs, be sure to set an appropriate IAM policy on the KMS CryptoKey (KEK) to ensure an attacker cannot unwrap the data crypto key.

JSON representation
{

  // Union field source can be only one of the following:
  "transient": {
    object(TransientCryptoKey)
  },
  "unwrapped": {
    object(UnwrappedCryptoKey)
  },
  "kmsWrapped": {
    object(KmsWrappedCryptoKey)
  }
  // End of list of possible types for union field source.
}
Fields

Union field source.

source can be only one of the following:

transient

object(TransientCryptoKey)

unwrapped

object(UnwrappedCryptoKey)

kmsWrapped

object(KmsWrappedCryptoKey)

TransientCryptoKey

Use this to have a random data crypto key generated. It will be discarded after the request finishes.

JSON representation
{
  "name": string
}
Fields
name

string

Name of the key. [required] This is an arbitrary string used to differentiate different keys. A unique key is generated per name: two separate TransientCryptoKey protos share the same generated key if their names are the same. When the data crypto key is generated, this name is not used in any way (repeating the api call will result in a different key being generated).

UnwrappedCryptoKey

Using raw keys is prone to security risks due to accidentally leaking the key. Choose another type of key if possible.

JSON representation
{
  "key": string
}
Fields
key

string (bytes format)

The AES 128/192/256 bit key. [required]

A base64-encoded string.

KmsWrappedCryptoKey

Include to use an existing data crypto key wrapped by KMS. Authorization requires the following IAM permissions when sending a request to perform a crypto transformation using a kms-wrapped crypto key: dlp.kms.encrypt

JSON representation
{
  "wrappedKey": string,
  "cryptoKeyName": string
}
Fields
wrappedKey

string (bytes format)

The wrapped data crypto key. [required]

A base64-encoded string.

cryptoKeyName

string

The resource name of the KMS CryptoKey to use for unwrapping. [required]

FfxCommonNativeAlphabet

These are commonly used subsets of the alphabet that the FFX mode natively supports. In the algorithm, the alphabet is selected using the "radix". Therefore each corresponds to particular radix.

Enums
FFX_COMMON_NATIVE_ALPHABET_UNSPECIFIED
NUMERIC [0-9] (radix of 10)
HEXADECIMAL [0-9A-F] (radix of 16)
UPPER_CASE_ALPHA_NUMERIC [0-9A-Z] (radix of 36)
ALPHA_NUMERIC [0-9A-Za-z] (radix of 62)

FixedSizeBucketingConfig

Buckets values based on fixed size ranges. The Bucketing transformation can provide all of this functionality, but requires more configuration. This message is provided as a convenience to the user for simple bucketing strategies.

The transformed value will be a hyphenated string of -, i.e if lowerBound = 10 and upperBound = 20 all values that are within this bucket will be replaced with "10-20".

This can be used on data of type: double, long.

If the bound Value type differs from the type of data being transformed, we will first attempt converting the type of the data to be transformed to match the type of the bound before comparing.

See https://cloud.google.com/dlp/docs/concepts-bucketing to learn more.

JSON representation
{
  "lowerBound": {
    object(Value)
  },
  "upperBound": {
    object(Value)
  },
  "bucketSize": number
}
Fields
lowerBound

object(Value)

Lower bound value of buckets. All values less than lowerBound are grouped together into a single bucket; for example if lowerBound = 10, then all values less than 10 are replaced with the value “-10”. [Required].

upperBound

object(Value)

Upper bound value of buckets. All values greater than upperBound are grouped together into a single bucket; for example if upperBound = 89, then all values greater than 89 are replaced with the value “89+”. [Required].

bucketSize

number

Size of each bucket (except for minimum and maximum buckets). So if lowerBound = 10, upperBound = 89, and bucketSize = 10, then the following buckets would be used: -10, 10-20, 20-30, 30-40, 40-50, 50-60, 60-70, 70-80, 80-89, 89+. Precision up to 2 decimals works. [Required].

BucketingConfig

Generalization function that buckets values based on ranges. The ranges and replacement values are dynamically provided by the user for custom behavior, such as 1-30 -> LOW 31-65 -> MEDIUM 66-100 -> HIGH This can be used on data of type: number, long, string, timestamp. If the bound Value type differs from the type of data being transformed, we will first attempt converting the type of the data to be transformed to match the type of the bound before comparing. See https://cloud.google.com/dlp/docs/concepts-bucketing to learn more.

JSON representation
{
  "buckets": [
    {
      object(Bucket)
    }
  ]
}
Fields
buckets[]

object(Bucket)

Set of buckets. Ranges must be non-overlapping.

Bucket

Bucket is represented as a range, along with replacement values.

JSON representation
{
  "min": {
    object(Value)
  },
  "max": {
    object(Value)
  },
  "replacementValue": {
    object(Value)
  }
}
Fields
min

object(Value)

Lower bound of the range, inclusive. Type should be the same as max if used.

max

object(Value)

Upper bound of the range, exclusive; type must match min.

replacementValue

object(Value)

Replacement value for this bucket. If not provided the default behavior will be to hyphenate the min-max range.

ReplaceWithInfoTypeConfig

Replace each matching finding with the name of the infoType.

TimePartConfig

For use with Date, Timestamp, and TimeOfDay, extract or preserve a portion of the value.

JSON representation
{
  "partToExtract": enum(TimePart)
}
Fields
partToExtract

enum(TimePart)

TimePart

Enums
TIME_PART_UNSPECIFIED
YEAR [0-9999]
MONTH [1-12]
DAY_OF_MONTH [1-31]
DAY_OF_WEEK [1-7]
WEEK_OF_YEAR [1-52]
HOUR_OF_DAY [0-23]

CryptoHashConfig

Pseudonymization method that generates surrogates via cryptographic hashing. Uses SHA-256. The key size must be either 32 or 64 bytes. Outputs a 32 byte digest as an uppercase hex string (for example, 41D1567F7F99F1DC2A5FAB886DEE5BEE). Currently, only string and integer values can be hashed.

JSON representation
{
  "cryptoKey": {
    object(CryptoKey)
  }
}
Fields
cryptoKey

object(CryptoKey)

The key used by the hash function.

DateShiftConfig

Shifts dates by random number of days, with option to be consistent for the same context. See https://cloud.google.com/dlp/docs/concepts-date-shifting to learn more.

JSON representation
{
  "upperBoundDays": number,
  "lowerBoundDays": number,
  "context": {
    object(FieldId)
  },
  "cryptoKey": {
    object(CryptoKey)
  }
}
Fields
upperBoundDays

number

Range of shift in days. Actual shift will be selected at random within this range (inclusive ends). Negative means shift to earlier in time. Must not be more than 365250 days (1000 years) each direction.

For example, 3 means shift date to at most 3 days into the future. [Required]

lowerBoundDays

number

For example, -5 means shift date to at most 5 days back in the past. [Required]

context

object(FieldId)

Points to the field that contains the context, for example, an entity id. If set, must also set method. If set, shift will be consistent for the given context.

cryptoKey

object(CryptoKey)

Causes the shift to be computed based on this key and the context. This results in the same shift for the same context and cryptoKey.

RecordTransformations

A type of transformation that is applied over structured data such as a table.

JSON representation
{
  "fieldTransformations": [
    {
      object(FieldTransformation)
    }
  ],
  "recordSuppressions": [
    {
      object(RecordSuppression)
    }
  ]
}
Fields
fieldTransformations[]

object(FieldTransformation)

Transform the record by applying various field transformations.

recordSuppressions[]

object(RecordSuppression)

Configuration defining which records get suppressed entirely. Records that match any suppression rule are omitted from the output [optional].

FieldTransformation

The transformation to apply to the field.

JSON representation
{
  "fields": [
    {
      object(FieldId)
    }
  ],
  "condition": {
    object(RecordCondition)
  },

  // Union field transformation can be only one of the following:
  "primitiveTransformation": {
    object(PrimitiveTransformation)
  },
  "infoTypeTransformations": {
    object(InfoTypeTransformations)
  }
  // End of list of possible types for union field transformation.
}
Fields
fields[]

object(FieldId)

Input field(s) to apply the transformation to. [required]

condition

object(RecordCondition)

Only apply the transformation if the condition evaluates to true for the given RecordCondition. The conditions are allowed to reference fields that are not used in the actual transformation. [optional]

Example Use Cases:

  • Apply a different bucket transformation to an age column if the zip code column for the same record is within a specific range.
  • Redact a field if the date of birth field is greater than 85.

Union field transformation. Transformation to apply. [required] transformation can be only one of the following:
primitiveTransformation

object(PrimitiveTransformation)

Apply the transformation to the entire field.

infoTypeTransformations

object(InfoTypeTransformations)

Treat the contents of the field as free text, and selectively transform content that matches an InfoType.

RecordCondition

A condition for determining whether a transformation should be applied to a field.

JSON representation
{
  "expressions": {
    object(Expressions)
  }
}
Fields
expressions

object(Expressions)

An expression.

Expressions

An expression, consisting or an operator and conditions.

JSON representation
{
  "logicalOperator": enum(LogicalOperator),
  "conditions": {
    object(Conditions)
  }
}
Fields
logicalOperator

enum(LogicalOperator)

The operator to apply to the result of conditions. Default and currently only supported value is AND.

conditions

object(Conditions)

LogicalOperator

Enums
LOGICAL_OPERATOR_UNSPECIFIED
AND

Conditions

A collection of conditions.

JSON representation
{
  "conditions": [
    {
      object(Condition)
    }
  ]
}
Fields
conditions[]

object(Condition)

Condition

The field type of value and field do not need to match to be considered equal, but not all comparisons are possible.

A value of type:

  • string can be compared against all other types
  • boolean can only be compared against other booleans
  • integer can be compared against doubles or a string if the string value can be parsed as an integer.
  • double can be compared against integers or a string if the string can be parsed as a double.
  • Timestamp can be compared against strings in RFC 3339 date string format.
  • TimeOfDay can be compared against timestamps and strings in the format of 'HH:mm:ss'.

If we fail to compare do to type mismatch, a warning will be given and the condition will evaluate to false.

JSON representation
{
  "field": {
    object(FieldId)
  },
  "operator": enum(RelationalOperator),
  "value": {
    object(Value)
  }
}
Fields
field

object(FieldId)

Field within the record this condition is evaluated against. [required]

operator

enum(RelationalOperator)

Operator used to compare the field or infoType to the value. [required]

value

object(Value)

Value to compare against. [Required, except for EXISTS tests.]

RelationalOperator

Operators available for comparing the value of fields.

Enums
RELATIONAL_OPERATOR_UNSPECIFIED
EQUAL_TO Equal.
NOT_EQUAL_TO Not equal to.
GREATER_THAN Greater than.
LESS_THAN Less than.
GREATER_THAN_OR_EQUALS Greater than or equals.
LESS_THAN_OR_EQUALS Less than or equals.
EXISTS Exists

RecordSuppression

Configuration to suppress records whose suppression conditions evaluate to true.

JSON representation
{
  "condition": {
    object(RecordCondition)
  }
}
Fields
condition

object(RecordCondition)

A condition that when it evaluates to true will result in the record being evaluated to be suppressed from the transformed content.

Methods

create

Creates a DeidentifyTemplate for re-using frequently used configuration for de-identifying content, images, and storage.

delete

Deletes a DeidentifyTemplate.

get

Gets a DeidentifyTemplate.

list

Lists DeidentifyTemplates.

patch

Updates the DeidentifyTemplate.
Was this page helpful? Let us know how we did:

Send feedback about...

Data Loss Prevention API