DLPJobTrigger
| Property | Value |
|---|---|
| Google Cloud Service Name | Cloud DLP |
| Google Cloud Service Documentation | /dlp/docs/ |
| Google Cloud REST Resource Name | projects.jobTriggers |
| Google Cloud REST Resource Documentation | /dlp/docs/reference/rest/v2/projects.jobTriggers |
| Config Connector Resource Short Names | gcpdlpjobtrigger gcpdlpjobtriggers dlpjobtrigger |
| Config Connector Service Name | dlp.googleapis.com |
| Config Connector Resource Fully Qualified Name | dlpjobtriggers.dlp.cnrm.cloud.google.com |
| Can Be Referenced by IAMPolicy/IAMPolicyMember | No |
| Config Connector Default Average Reconcile Interval In Seconds | 600 |
Custom Resource Definition Properties
Annotations
| Fields | |
|---|---|
cnrm.cloud.google.com/state-into-spec |
|
Spec
Schema
description: string
displayName: string
inspectJob:
actions:
- jobNotificationEmails: {}
pubSub:
topicRef:
external: string
name: string
namespace: string
publishFindingsToCloudDataCatalog: {}
publishSummaryToCscc: {}
publishToStackdriver: {}
saveFindings:
outputConfig:
dlpStorage: {}
outputSchema: string
table:
datasetRef:
external: string
name: string
namespace: string
projectRef:
external: string
name: string
namespace: string
tableRef:
external: string
name: string
namespace: string
inspectConfig:
customInfoTypes:
- detectionRules:
- hotwordRule:
hotwordRegex:
groupIndexes:
- integer
pattern: string
likelihoodAdjustment:
fixedLikelihood: string
relativeLikelihood: integer
proximity:
windowAfter: integer
windowBefore: integer
dictionary:
cloudStoragePath:
path: string
wordList:
words:
- string
exclusionType: string
infoType:
name: string
version: string
likelihood: string
regex:
groupIndexes:
- integer
pattern: string
storedType:
createTime: string
nameRef:
external: string
name: string
namespace: string
surrogateType: {}
excludeInfoTypes: boolean
includeQuote: boolean
infoTypes:
- name: string
limits:
maxFindingsPerInfoType:
- infoType:
name: string
version: string
maxFindings: integer
maxFindingsPerItem: integer
maxFindingsPerRequest: integer
minLikelihood: string
ruleSet:
- infoTypes:
- name: string
version: string
rules:
- exclusionRule:
dictionary:
cloudStoragePath:
path: string
wordList:
words:
- string
excludeInfoTypes:
infoTypes:
- name: string
version: string
matchingType: string
regex:
groupIndexes:
- integer
pattern: string
hotwordRule:
hotwordRegex:
groupIndexes:
- integer
pattern: string
likelihoodAdjustment:
fixedLikelihood: string
relativeLikelihood: integer
proximity:
windowAfter: integer
windowBefore: integer
inspectTemplateRef:
external: string
name: string
namespace: string
storageConfig:
bigQueryOptions:
excludedFields:
- name: string
identifyingFields:
- name: string
includedFields:
- name: string
rowsLimit: integer
rowsLimitPercent: integer
sampleMethod: string
tableReference:
datasetRef:
external: string
name: string
namespace: string
projectRef:
external: string
name: string
namespace: string
tableRef:
external: string
name: string
namespace: string
cloudStorageOptions:
bytesLimitPerFile: integer
bytesLimitPerFilePercent: integer
fileSet:
regexFileSet:
bucketRef:
external: string
name: string
namespace: string
excludeRegex:
- string
includeRegex:
- string
url: string
fileTypes:
- string
filesLimitPercent: integer
sampleMethod: string
datastoreOptions:
kind:
name: string
partitionId:
namespaceId: string
projectRef:
external: string
name: string
namespace: string
hybridOptions:
description: string
labels:
string: string
requiredFindingLabelKeys:
- string
tableOptions:
identifyingFields:
- name: string
timespanConfig:
enableAutoPopulationOfTimespanConfig: boolean
endTime: string
startTime: string
timestampField:
name: string
location: string
projectRef:
external: string
name: string
namespace: string
resourceID: string
status: string
triggers:
- manual: {}
schedule:
recurrencePeriodDuration: string
| Fields | |
|---|---|
|
Optional |
User provided description (max 256 chars) |
|
Optional |
Display name (max 100 chars) |
|
Required |
For inspect jobs, a snapshot of the configuration. |
|
Optional |
Actions to execute at the completion of the job. |
|
Optional |
|
|
Optional |
Enable email notification for project owners and editors on job's completion/failure. |
|
Optional |
Publish a notification to a pubsub topic. |
|
Optional |
|
|
Optional |
Cloud Pub/Sub topic to send notifications to. The topic must have given publishing access rights to the DLP API service account executing the long running DlpJob sending the notifications. Format is projects/{project}/topics/{topic}. Allowed value: The Google Cloud resource name of a `PubSubTopic` resource (format: `projects/{{project}}/topics/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
Publish findings to Cloud Datahub. |
|
Optional |
Publish summary to Cloud Security Command Center (Alpha). |
|
Optional |
Enable Stackdriver metric dlp.googleapis.com/finding_count. |
|
Optional |
Save resulting findings in a provided location. |
|
Optional |
Location to store findings outside of DLP. |
|
Optional |
Store findings directly to DLP. If neither this or bigquery is chosen only summary stats of total infotype count will be stored. Quotes will not be stored to dlp findings. If quotes are needed, store to BigQuery. Currently only for inspect jobs. |
|
Optional |
Schema used for writing the findings for Inspect jobs. This field is only used for Inspect and must be unspecified for Risk jobs. Columns are derived from the `Finding` object. If appending to an existing table, any columns from the predefined schema that are missing will be added. No columns in the existing table will be deleted. If unspecified, then all available columns will be used for a new table or an (existing) table with no schema, and no changes will be made to an existing table that has a schema. Only for use with external storage. Possible values: OUTPUT_SCHEMA_UNSPECIFIED, BASIC_COLUMNS, GCS_COLUMNS, DATASTORE_COLUMNS, BIG_QUERY_COLUMNS, ALL_COLUMNS |
|
Optional |
Store findings in an existing table or a new table in an existing dataset. If table_id is not set a new one will be generated for you with the following format: dlp_googleapis_yyyy_mm_dd_[dlp_job_id]. Pacific timezone will be used for generating the date details. For Inspect, each column in an existing output table must have the same name, type, and mode of a field in the `Finding` object. For Risk, an existing output table should be the output of a previous Risk analysis job run on the same source table, with the same privacy metric and quasi-identifiers. Risk jobs that analyze the same table but compute a different privacy metric, or use different sets of quasi-identifiers, cannot store their results in the same table. |
|
Optional |
|
|
Optional |
Dataset ID of the table. Allowed value: The Google Cloud resource name of a `BigQueryDataset` resource (format: `projects/{{project}}/datasets/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
|
|
Optional |
The Google Cloud Platform project ID of the project containing the table. If omitted, project ID is inferred from the API call. Allowed value: The Google Cloud resource name of a `Project` resource (format: `projects/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
|
|
Optional |
Name of the table. Allowed value: The Google Cloud resource name of a `BigQueryTable` resource (format: `projects/{{project}}/datasets/{{dataset_id}}/tables/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
How and what to scan for. |
|
Optional |
CustomInfoTypes provided by the user. See https://cloud.google.com/dlp/docs/creating-custom-infotypes to learn more. |
|
Optional |
|
|
Optional |
Set of detection rules to apply to all findings of this CustomInfoType. Rules are applied in order that they are specified. Not supported for the `surrogate_type` CustomInfoType. |
|
Optional |
|
|
Optional |
Hotword-based detection rule. |
|
Optional |
Regular expression pattern defining what qualifies as a hotword. |
|
Optional |
The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included. |
|
Optional |
|
|
Optional |
Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub. |
|
Optional |
Likelihood adjustment to apply to all matching findings. |
|
Optional |
Set the likelihood of a finding to a fixed value. Possible values: LIKELIHOOD_UNSPECIFIED, VERY_UNLIKELY, UNLIKELY, POSSIBLE, LIKELY, VERY_LIKELY |
|
Optional |
Increase or decrease the likelihood by the specified number of levels. For example, if a finding would be `POSSIBLE` without the detection rule and `relative_likelihood` is 1, then it is upgraded to `LIKELY`, while a value of -1 would downgrade it to `UNLIKELY`. Likelihood may never drop below `VERY_UNLIKELY` or exceed `VERY_LIKELY`, so applying an adjustment of 1 followed by an adjustment of -1 when base likelihood is `VERY_LIKELY` will result in a final likelihood of `LIKELY`. |
|
Optional |
Proximity of the finding within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. Note that the finding itself will be included in the window, so that hotwords may be used to match substrings of the finding itself. For example, the certainty of a phone number regex "(d{3}) d{3}-d{4}" could be adjusted upwards if the area code is known to be the local area code of a company office using the hotword regex "(xxx)", where "xxx" is the area code in question. |
|
Optional |
Number of characters after the finding to consider. |
|
Optional |
Number of characters before the finding to consider. |
|
Optional |
A list of phrases to detect as a CustomInfoType. |
|
Optional |
Newline-delimited file of words in Cloud Storage. Only a single file is accepted. |
|
Optional |
A url representing a file or path (no wildcards) in Cloud Storage. Example: gs://[BUCKET_NAME]/dictionary.txt |
|
Optional |
List of words or phrases to search for. |
|
Optional |
Words or phrases defining the dictionary. The dictionary must contain at least one phrase and every phrase must contain at least 2 characters that are letters or digits. [required] |
|
Optional |
|
|
Optional |
If set to EXCLUSION_TYPE_EXCLUDE this infoType will not cause a finding to be returned. It still can be used for rules matching. Possible values: EXCLUSION_TYPE_UNSPECIFIED, EXCLUSION_TYPE_EXCLUDE |
|
Optional |
CustomInfoType can either be a new infoType, or an extension of built-in infoType, when the name matches one of existing infoTypes and that infoType is specified in `InspectContent.info_types` field. Specifying the latter adds findings to the one detected by the system. If built-in info type is not specified in `InspectContent.info_types` list then the name is treated as a custom info type. |
|
Optional |
Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/dlp/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$-_]{1,64}`. |
|
Optional |
Optional version name for this InfoType. |
|
Optional |
Likelihood to return for this CustomInfoType. This base value can be altered by a detection rule if the finding meets the criteria specified by the rule. Defaults to `VERY_LIKELY` if not specified. Possible values: LIKELIHOOD_UNSPECIFIED, VERY_UNLIKELY, UNLIKELY, POSSIBLE, LIKELY, VERY_LIKELY |
|
Optional |
Regular expression based CustomInfoType. |
|
Optional |
The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included. |
|
Optional |
|
|
Optional |
Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub. |
|
Optional |
Load an existing `StoredInfoType` resource for use in `InspectDataSource`. Not currently supported in `InspectContent`. |
|
Optional |
Timestamp indicating when the version of the `StoredInfoType` used for inspection was created. Output-only field, populated by the system. |
|
Optional |
|
|
Optional |
Resource name of the requested `StoredInfoType`, for example `organizations/433245324/storedInfoTypes/432452342` or `projects/project-id/storedInfoTypes/432452342`. Allowed value: The Google Cloud resource name of a `DLPStoredInfoType` resource (format: `{{parent}}/storedInfoTypes/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
Message for detecting output from deidentification transformations that support reversing. |
|
Optional |
When true, excludes type information of the findings. This is not used for data profiling. |
|
Optional |
When true, a contextual quote from the data that triggered a finding is included in the response; see Finding.quote. This is not used for data profiling. |
|
Optional |
Restricts what info_types to look for. The values must correspond to InfoType values returned by ListInfoTypes or listed at https://cloud.google.com/dlp/docs/infotypes-reference. When no InfoTypes or CustomInfoTypes are specified in a request, the system may automatically choose what detectors to run. By default this may be all types, but may change over time as detectors are updated. If you need precise control and predictability as to what detectors are run you should specify specific InfoTypes listed in the reference, otherwise a default list will be used, which may change over time. |
|
Optional |
|
|
Optional |
Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/dlp/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$-_]{1,64}`. |
|
Optional |
Configuration to control the number of findings returned. This is not used for data profiling. |
|
Optional |
Configuration of findings limit given for specified infoTypes. |
|
Optional |
|
|
Optional |
Type of information the findings limit applies to. Only one limit per info_type should be provided. If InfoTypeLimit does not have an info_type, the DLP API applies the limit against all info_types that are found but not specified in another InfoTypeLimit. |
|
Optional |
Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/dlp/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$-_]{1,64}`. |
|
Optional |
Optional version name for this InfoType. |
|
Optional |
Max findings limit for the given infoType. |
|
Optional |
Max number of findings that will be returned for each item scanned. When set within `InspectJobConfig`, the maximum returned is 2000 regardless if this is set higher. When set within `InspectContentRequest`, this field is ignored. |
|
Optional |
Max number of findings that will be returned per request/job. When set within `InspectContentRequest`, the maximum returned is 2000 regardless if this is set higher. |
|
Optional |
Only returns findings equal or above this threshold. The default is POSSIBLE. See https://cloud.google.com/dlp/docs/likelihood to learn more. Possible values: LIKELIHOOD_UNSPECIFIED, VERY_UNLIKELY, UNLIKELY, POSSIBLE, LIKELY, VERY_LIKELY |
|
Optional |
Set of rules to apply to the findings for this InspectConfig. Exclusion rules, contained in the set are executed in the end, other rules are executed in the order they are specified for each info type. |
|
Optional |
|
|
Optional |
List of infoTypes this rule set is applied to. |
|
Optional |
|
|
Optional |
Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/dlp/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$-_]{1,64}`. |
|
Optional |
Optional version name for this InfoType. |
|
Optional |
Set of rules to be applied to infoTypes. The rules are applied in order. |
|
Optional |
|
|
Optional |
Exclusion rule. |
|
Optional |
Dictionary which defines the rule. |
|
Optional |
Newline-delimited file of words in Cloud Storage. Only a single file is accepted. |
|
Optional |
A url representing a file or path (no wildcards) in Cloud Storage. Example: gs://[BUCKET_NAME]/dictionary.txt |
|
Optional |
List of words or phrases to search for. |
|
Optional |
Words or phrases defining the dictionary. The dictionary must contain at least one phrase and every phrase must contain at least 2 characters that are letters or digits. [required] |
|
Optional |
|
|
Optional |
Set of infoTypes for which findings would affect this rule. |
|
Optional |
InfoType list in ExclusionRule rule drops a finding when it overlaps or contained within with a finding of an infoType from this list. For example, for `InspectionRuleSet.info_types` containing "PHONE_NUMBER"` and `exclusion_rule` containing `exclude_info_types.info_types` with "EMAIL_ADDRESS" the phone number findings are dropped if they overlap with EMAIL_ADDRESS finding. That leads to "555-222-2222@example.org" to generate only a single finding, namely email address. |
|
Optional |
|
|
Optional |
Name of the information type. Either a name of your choosing when creating a CustomInfoType, or one of the names listed at https://cloud.google.com/dlp/docs/infotypes-reference when specifying a built-in type. When sending Cloud DLP results to Data Catalog, infoType names should conform to the pattern `[A-Za-z0-9$-_]{1,64}`. |
|
Optional |
Optional version name for this InfoType. |
|
Optional |
How the rule is applied, see MatchingType documentation for details. Possible values: MATCHING_TYPE_UNSPECIFIED, MATCHING_TYPE_FULL_MATCH, MATCHING_TYPE_PARTIAL_MATCH, MATCHING_TYPE_INVERSE_MATCH |
|
Optional |
Regular expression which defines the rule. |
|
Optional |
The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included. |
|
Optional |
|
|
Optional |
Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub. |
|
Optional |
|
|
Optional |
Regular expression pattern defining what qualifies as a hotword. |
|
Optional |
The index of the submatch to extract as findings. When not specified, the entire match is returned. No more than 3 may be included. |
|
Optional |
|
|
Optional |
Pattern defining the regular expression. Its syntax (https://github.com/google/re2/wiki/Syntax) can be found under the google/re2 repository on GitHub. |
|
Optional |
Likelihood adjustment to apply to all matching findings. |
|
Optional |
Set the likelihood of a finding to a fixed value. Possible values: LIKELIHOOD_UNSPECIFIED, VERY_UNLIKELY, UNLIKELY, POSSIBLE, LIKELY, VERY_LIKELY |
|
Optional |
Increase or decrease the likelihood by the specified number of levels. For example, if a finding would be `POSSIBLE` without the detection rule and `relative_likelihood` is 1, then it is upgraded to `LIKELY`, while a value of -1 would downgrade it to `UNLIKELY`. Likelihood may never drop below `VERY_UNLIKELY` or exceed `VERY_LIKELY`, so applying an adjustment of 1 followed by an adjustment of -1 when base likelihood is `VERY_LIKELY` will result in a final likelihood of `LIKELY`. |
|
Optional |
Proximity of the finding within which the entire hotword must reside. The total length of the window cannot exceed 1000 characters. Note that the finding itself will be included in the window, so that hotwords may be used to match substrings of the finding itself. For example, the certainty of a phone number regex "(d{3}) d{3}-d{4}" could be adjusted upwards if the area code is known to be the local area code of a company office using the hotword regex "(xxx)", where "xxx" is the area code in question. |
|
Optional |
Number of characters after the finding to consider. |
|
Optional |
Number of characters before the finding to consider. |
|
Optional |
|
|
Optional |
If provided, will be used as the default for all values in InspectConfig. `inspect_config` will be merged into the values persisted as part of the template. Allowed value: The Google Cloud resource name of a `DLPInspectTemplate` resource (format: `{{parent}}/inspectTemplates/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Required |
The data to scan. |
|
Optional |
BigQuery options. |
|
Optional |
References to fields excluded from scanning. This allows you to skip inspection of entire columns which you know have no findings. |
|
Optional |
|
|
Optional |
Name describing the field. |
|
Optional |
Table fields that may uniquely identify a row within the table. When `actions.saveFindings.outputConfig.table` is specified, the values of columns specified here are available in the output table under `location.content_locations.record_location.record_key.id_values`. Nested fields such as `person.birthdate.year` are allowed. |
|
Optional |
|
|
Optional |
Name describing the field. |
|
Optional |
Limit scanning only to these fields. |
|
Optional |
|
|
Optional |
Name describing the field. |
|
Optional |
Max number of rows to scan. If the table has more rows than this value, the rest of the rows are omitted. If not set, or if set to 0, all rows will be scanned. Only one of rows_limit and rows_limit_percent can be specified. Cannot be used in conjunction with TimespanConfig. |
|
Optional |
Max percentage of rows to scan. The rest are omitted. The number of rows scanned is rounded down. Must be between 0 and 100, inclusively. Both 0 and 100 means no limit. Defaults to 0. Only one of rows_limit and rows_limit_percent can be specified. Cannot be used in conjunction with TimespanConfig. |
|
Optional |
Possible values: SAMPLE_METHOD_UNSPECIFIED, TOP, RANDOM_START |
|
Required* |
Complete BigQuery table reference. |
|
Optional |
|
|
Optional |
Dataset ID of the table. Allowed value: The Google Cloud resource name of a `BigQueryDataset` resource (format: `projects/{{project}}/datasets/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
|
|
Optional |
The Google Cloud Platform project ID of the project containing the table. If omitted, project ID is inferred from the API call. Allowed value: The Google Cloud resource name of a `Project` resource (format: `projects/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
|
|
Optional |
Name of the table. Allowed value: The Google Cloud resource name of a `BigQueryTable` resource (format: `projects/{{project}}/datasets/{{dataset_id}}/tables/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
Google Cloud Storage options. |
|
Optional |
Max number of bytes to scan from a file. If a scanned file's size is bigger than this value then the rest of the bytes are omitted. Only one of bytes_limit_per_file and bytes_limit_per_file_percent can be specified. Cannot be set if de-identification is requested. |
|
Optional |
Max percentage of bytes to scan from a file. The rest are omitted. The number of bytes scanned is rounded down. Must be between 0 and 100, inclusively. Both 0 and 100 means no limit. Defaults to 0. Only one of bytes_limit_per_file and bytes_limit_per_file_percent can be specified. Cannot be set if de-identification is requested. |
|
Optional |
The set of one or more files to scan. |
|
Optional |
The regex-filtered set of files to scan. Exactly one of `url` or `regex_file_set` must be set. |
|
Required* |
|
|
Optional |
The name of a Cloud Storage bucket. Required. Allowed value: The Google Cloud resource name of a `StorageBucket` resource (format: `{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
A list of regular expressions matching file paths to exclude. All files in the bucket that match at least one of these regular expressions will be excluded from the scan. Regular expressions use RE2 [syntax](https://github.com/google/re2/wiki/Syntax); a guide can be found under the google/re2 repository on GitHub. |
|
Optional |
|
|
Optional |
A list of regular expressions matching file paths to include. All files in the bucket that match at least one of these regular expressions will be included in the set of files, except for those that also match an item in `exclude_regex`. Leaving this field empty will match all files by default (this is equivalent to including `.*` in the list). Regular expressions use RE2 [syntax](https://github.com/google/re2/wiki/Syntax); a guide can be found under the google/re2 repository on GitHub. |
|
Optional |
|
|
Optional |
The Cloud Storage url of the file(s) to scan, in the format `gs:///`. Trailing wildcard in the path is allowed. If the url ends in a trailing slash, the bucket or directory represented by the url will be scanned non-recursively (content in sub-directories will not be scanned). This means that `gs://mybucket/` is equivalent to `gs://mybucket/*`, and `gs://mybucket/directory/` is equivalent to `gs://mybucket/directory/*`. Exactly one of `url` or `regex_file_set` must be set. |
|
Optional |
List of file type groups to include in the scan. If empty, all files are scanned and available data format processors are applied. In addition, the binary content of the selected files is always scanned as well. Images are scanned only as binary if the specified region does not support image inspection and no file_types were specified. Image inspection is restricted to 'global', 'us', 'asia', and 'europe'. |
|
Optional |
|
|
Optional |
Limits the number of files to scan to this percentage of the input FileSet. Number of files scanned is rounded down. Must be between 0 and 100, inclusively. Both 0 and 100 means no limit. Defaults to 0. |
|
Optional |
Possible values: SAMPLE_METHOD_UNSPECIFIED, TOP, RANDOM_START |
|
Optional |
Google Cloud Datastore options. |
|
Optional |
The kind to process. |
|
Optional |
The name of the kind. |
|
Optional |
A partition ID identifies a grouping of entities. The grouping is always by project namespace ID may be empty. |
|
Optional |
If not empty, the ID of the namespace to which the entities belong. |
|
Optional |
|
|
Optional |
The ID of the project to which the entities belong. Allowed value: The Google Cloud resource name of a `Project` resource (format: `projects/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
Hybrid inspection options. |
|
Optional |
A short description of where the data is coming from. Will be stored once in the job. 256 max length. |
|
Optional |
To organize findings, these labels will be added to each finding. Label keys must be between 1 and 63 characters long and must conform to the following regular expression: `[a-z]([-a-z0-9]*[a-z0-9])?`. Label values must be between 0 and 63 characters long and must conform to the regular expression `([a-z]([-a-z0-9]*[a-z0-9])?)?`. No more than 10 labels can be associated with a given finding. Examples: * `"environment" : "production"` * `"pipeline" : "etl"` |
|
Optional |
These are labels that each inspection request must include within their 'finding_labels' map. Request may contain others, but any missing one of these will be rejected. Label keys must be between 1 and 63 characters long and must conform to the following regular expression: `[a-z]([-a-z0-9]*[a-z0-9])?`. No more than 10 keys can be required. |
|
Optional |
|
|
Optional |
If the container is a table, additional information to make findings meaningful such as the columns that are primary keys. |
|
Optional |
The columns that are the primary keys for table objects included in ContentItem. A copy of this cell's value will stored alongside alongside each finding so that the finding can be traced to the specific row it came from. No more than 3 may be provided. |
|
Optional |
|
|
Optional |
Name describing the field. |
|
Optional |
|
|
Optional |
When the job is started by a JobTrigger we will automatically figure out a valid start_time to avoid scanning files that have not been modified since the last time the JobTrigger executed. This will be based on the time of the execution of the last run of the JobTrigger. |
|
Optional |
Exclude files, tables, or rows newer than this value. If not set, no upper time limit is applied. |
|
Optional |
Exclude files, tables, or rows older than this value. If not set, no lower time limit is applied. |
|
Optional |
Specification of the field containing the timestamp of scanned items. Used for data sources like Datastore and BigQuery. For BigQuery: If this value is not specified and the table was modified between the given start and end times, the entire table will be scanned. If this value is specified, then rows are filtered based on the given start and end times. Rows with a `NULL` value in the provided BigQuery column are skipped. Valid data types of the provided BigQuery column are: `INTEGER`, `DATE`, `TIMESTAMP`, and `DATETIME`. For Datastore: If this value is specified, then entities are filtered based on the given start and end times. If an entity does not contain the provided timestamp property or contains empty or invalid values, then it is included. Valid data types of the provided timestamp property are: `TIMESTAMP`. |
|
Optional |
Name describing the field. |
|
Optional |
Immutable. The location of the resource |
|
Required |
Immutable. The Project that this resource belongs to. Only one of [projectRef] may be specified. |
|
Optional |
Allowed value: The Google Cloud resource name of a `Project` resource (format: `projects/{{name}}`). |
|
Optional |
Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names |
|
Optional |
Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/ |
|
Optional |
Immutable. Optional. The service-generated name of the resource. Used for acquisition only. Leave unset to create a new resource. |
|
Required |
Immutable. Required. A status for this trigger. Possible values: STATUS_UNSPECIFIED, HEALTHY, PAUSED, CANCELLED |
|
Required |
A list of triggers which will be OR'ed together. Only one in the list needs to trigger for a job to be started. The list may contain only a single Schedule trigger and must have at least one object. |
|
Required |
|
|
Optional |
For use with hybrid jobs. Jobs must be manually created and finished. |
|
Optional |
Create a job on a repeating basis based on the elapse of time. |
|
Optional |
With this option a job is started a regular periodic basis. For example: every day (86400 seconds). A scheduled start time will be skipped if the previous execution has not ended when its scheduled time occurs. This value must be set to a time duration greater than or equal to 1 day and can be no longer than 60 days. |
* Field is required when parent field is specified
Status
Schema
conditions:
- lastTransitionTime: string
message: string
reason: string
status: string
type: string
createTime: string
errors:
- details:
code: integer
details:
- typeUrl: string
value: string
message: string
timestamps:
- string
lastRunTime: string
locationId: string
observedGeneration: integer
updateTime: string
| Fields | |
|---|---|
conditions |
Conditions represent the latest available observation of the resource's current state. |
conditions[] |
|
conditions[].lastTransitionTime |
Last time the condition transitioned from one status to another. |
conditions[].message |
Human-readable message indicating details about last transition. |
conditions[].reason |
Unique, one-word, CamelCase reason for the condition's last transition. |
conditions[].status |
Status is the status of the condition. Can be True, False, Unknown. |
conditions[].type |
Type is the type of the condition. |
createTime |
Output only. The creation timestamp of a triggeredJob. |
errors |
Output only. A stream of errors encountered when the trigger was activated. Repeated errors may result in the JobTrigger automatically being paused. Will return the last 100 errors. Whenever the JobTrigger is modified this list will be cleared. |
errors[] |
|
errors[].details |
Detailed error codes and messages. |
errors[].details.code |
The status code, which should be an enum value of google.rpc.Code. |
errors[].details.details |
A list of messages that carry the error details. There is a common set of message types for APIs to use. |
errors[].details.details[] |
|
errors[].details.details[].typeUrl |
A URL/resource name that uniquely identifies the type of the serialized protocol buffer message. This string must contain at least one "/" character. The last segment of the URL's path must represent the fully qualified name of the type (as in `path/google.protobuf.Duration`). The name should be in a canonical form (e.g., leading "." is not accepted). In practice, teams usually precompile into the binary all types that they expect it to use in the context of Any. However, for URLs which use the scheme `http`, `https`, or no scheme, one can optionally set up a type server that maps type URLs to message definitions as follows: * If no scheme is provided, `https` is assumed. * An HTTP GET on the URL must yield a google.protobuf.Type value in binary format, or produce an error. * Applications are allowed to cache lookup results based on the URL, or have them precompiled into a binary to avoid any lookup. Therefore, binary compatibility needs to be preserved on changes to types. (Use versioned type names to manage breaking changes.) Note: this functionality is not currently available in the official protobuf release, and it is not used for type URLs beginning with type.googleapis.com. Schemes other than `http`, `https` (or the empty scheme) might be used with implementation specific semantics. |
errors[].details.details[].value |
Must be a valid serialized protocol buffer of the above specified type. |
errors[].details.message |
A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client. |
errors[].timestamps |
The times the error occurred. |
errors[].timestamps[] |
|
lastRunTime |
Output only. The timestamp of the last time this trigger executed. |
locationId |
Output only. The geographic location where this resource is stored. |
observedGeneration |
ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource. |
updateTime |
Output only. The last update timestamp of a triggeredJob. |
Sample YAML(s)
Big Query Job Trigger
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPJobTrigger
metadata:
name: dlpjobtrigger-sample-bigqueryjobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
triggers:
- schedule:
recurrencePeriodDuration: "86400s"
status: "HEALTHY"
inspectJob:
storageConfig:
bigQueryOptions:
tableReference:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
datasetRef:
name: "dlpjobtriggerdepbigqueryjobtrigger"
tableRef:
name: "dlpjobtriggerdepbigqueryjobtrigger"
identifyingFields:
- name: "sample-field"
rowsLimit: 1
sampleMethod: "TOP"
excludedFields:
- name: "excluded-field"
actions:
- saveFindings:
outputConfig:
outputSchema: "BASIC_COLUMNS"
table:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
datasetRef:
name: "dlpjobtriggerdepbigqueryjobtrigger"
tableRef:
name: "dlpjobtriggerdepbigqueryjobtrigger"
- pubSub:
topicRef:
name: "dlpjobtrigger-dep-bigqueryjobtrigger"
---
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryDataset
metadata:
name: dlpjobtriggerdepbigqueryjobtrigger
spec:
location: US
---
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryTable
metadata:
name: dlpjobtriggerdepbigqueryjobtrigger
spec:
datasetRef:
name: "dlpjobtriggerdepbigqueryjobtrigger"
schema: '[{"name": "sample_field", "type": "STRING"}]'
---
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
name: dlpjobtrigger-dep-bigqueryjobtrigger
Cloud Storage Job Trigger
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPJobTrigger
metadata:
name: dlpjobtrigger-sample-cloudstoragejobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
location: "us-west2"
description: "A sample job trigger using cloud storage"
displayName: "sample-trigger"
triggers:
- schedule:
recurrencePeriodDuration: "86400s"
status: "HEALTHY"
inspectJob:
storageConfig:
cloudStorageOptions:
fileSet:
# Replace "${DLP_TEST_BUCKET?}" with your storage bucket name
url: "gs://${DLP_TEST_BUCKET?}/*"
bytesLimitPerFile: 1
fileTypes:
- "BINARY_FILE"
- "TEXT_FILE"
sampleMethod: "TOP"
filesLimitPercent: 50
timespanConfig:
startTime: "2017-01-15T01:30:15.010Z"
endTime: "2018-01-15T01:30:15.010Z"
timestampField:
name: "sample-field"
enableAutoPopulationOfTimespanConfig: true
inspectConfig:
infoTypes:
- name: "AGE"
minLikelihood: "UNLIKELY"
limits:
maxFindingsPerItem: 3
maxFindingsPerRequest: 3
maxFindingsPerInfoType:
- infoType:
name: "AGE"
version: "1"
maxFindings: 3
includeQuote: true
excludeInfoTypes: true
customInfoTypes:
- infoType:
name: "PHONE_NUMBER"
version: "1"
likelihood: "LIKELY"
detectionRules:
- hotwordRule:
hotwordRegex:
pattern: "([1-3])([0-9]*)"
groupIndexes:
- 1
- 2
proximity:
windowBefore: 3
windowAfter: 3
likelihoodAdjustment:
fixedLikelihood: "VERY_LIKELY"
- hotwordRule:
likelihoodAdjustment:
relativeLikelihood: -1
exclusionType: "EXCLUSION_TYPE_EXCLUDE"
dictionary:
wordList:
words:
- "one"
- "two"
- dictionary:
cloudStoragePath:
# Replace "${DLP_TEST_BUCKET?}" with your storage bucket name
path: "gs://${DLP_TEST_BUCKET?}/dictionary-1"
- regex:
pattern: "([a-e]+)([f-z]*)"
groupIndexes:
- 1
- 2
- storedType:
nameRef:
name: "dlpjobtrigger-dep-cloudstoragejobtrigger"
ruleSet:
- infoTypes:
- name: "AGE"
version: "1"
rules:
- hotwordRule:
hotwordRegex:
pattern: "([1-4])([0-9]*)"
groupIndexes:
- 1
- 2
proximity:
windowBefore: 3
windowAfter: 3
likelihoodAdjustment:
fixedLikelihood: "VERY_LIKELY"
- hotwordRule:
likelihoodAdjustment:
relativeLikelihood: -1
- exclusionRule:
matchingType: "MATCHING_TYPE_FULL_MATCH"
dictionary:
wordList:
words:
- "one"
- "two"
- exclusionRule:
dictionary:
cloudStoragePath:
# Replace "${DLP_TEST_BUCKET?}" with your storage bucket name
path: "gs://${DLP_TEST_BUCKET?}/dictionary-2"
- exclusionRule:
regex:
pattern: "([+-])([0-9]+)"
groupIndexes:
- 1
- 2
- exclusionRule:
excludeInfoTypes:
infoTypes:
- name: "AGE"
version: "1"
inspectTemplateName: "fake"
---
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPStoredInfoType
metadata:
name: dlpjobtrigger-dep-cloudstoragejobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
loction: "us-west2"
regex:
pattern: ".*"
Datastore Job Trigger
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPJobTrigger
metadata:
name: dlpjobtrigger-sample-datastorejobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
location: "us-west2"
triggers:
- schedule:
recurrencePeriodDuration: "86400s"
status: "HEALTHY"
inspectJob:
storageConfig:
datastoreOptions:
partitionId:
projectRef:
name: "dlpjobtrigger-dep-dsjobtrigger"
namespaceId: "test-namespace"
kind:
name: "test-kind"
---
apiVersion: resourcemanager.cnrm.cloud.google.com/v1beta1
kind: Project
metadata:
name: dlpjobtrigger-dep-dsjobtrigger
spec:
organizationRef:
# Replace "${ORG_ID?}" with the numeric ID for your organization
external: "${ORG_ID?}"
name: "dlpjobtrigger-dep-dsjobtrigger"
Hybrid Job Trigger
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPJobTrigger
metadata:
name: dlpjobtrigger-sample-hybridjobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
triggers:
- manual: {}
status: "HEALTHY"
inspectJob:
storageConfig:
hybridOptions:
description: "A sample data source outside GCP"
requiredFindingLabelKeys:
- "label-one"
- "label-two"
labels:
label-one: "value-one"
tableOptions:
identifyingFields:
- name: "sample-field"
Regex File Set Job Trigger
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPJobTrigger
metadata:
name: dlpjobtrigger-sample-regexfilesetjobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
triggers:
- schedule:
recurrencePeriodDuration: "86400s"
status: "HEALTHY"
inspectJob:
storageConfig:
cloudStorageOptions:
fileSet:
regexFileSet:
bucketRef:
# Replace "${DLP_TEST_BUCKET?}" with your storage bucket name
external: "${DLP_TEST_BUCKET?}"
includeRegex:
- "[a-z-]+"
excludeRegex:
- "[A-Z-]+"
bytesLimitPerFilePercent: 50
Rows Limit Percent Job Trigger
# Copyright 2022 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
apiVersion: dlp.cnrm.cloud.google.com/v1beta1
kind: DLPJobTrigger
metadata:
name: dlpjobtrigger-sample-rowslimitpercentjobtrigger
spec:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
triggers:
- schedule:
recurrencePeriodDuration: "86400s"
status: "HEALTHY"
inspectJob:
storageConfig:
bigQueryOptions:
tableReference:
projectRef:
# Replace "${PROJECT_ID?}" with your project ID
external: "projects/${PROJECT_ID?}"
datasetRef:
name: "dlpjobtriggerdeprowslimitpercentjobtrigger"
tableRef:
name: "dlpjobtriggerdeprowslimitpercentjobtrigger"
rowsLimitPercent: 50
includedFields:
- name: "included-field"
---
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryDataset
metadata:
name: dlpjobtriggerdeprowslimitpercentjobtrigger
spec:
location: US
---
apiVersion: bigquery.cnrm.cloud.google.com/v1beta1
kind: BigQueryTable
metadata:
name: dlpjobtriggerdeprowslimitpercentjobtrigger
spec:
datasetRef:
name: "dlpjobtriggerdeprowslimitpercentjobtrigger"
schema: '[{"name": "sample_field", "type": "STRING"}]'