- NAME
-
- gcloud dataplex datascans create data-profile - create a Dataplex data profile scan job
- SYNOPSIS
-
-
gcloud dataplex datascans create data-profile
(DATASCAN
:--location
=LOCATION
) (--data-source-entity
=DATA_SOURCE_ENTITY
|--data-source-resource
=DATA_SOURCE_RESOURCE
) [--description
=DESCRIPTION
] [--display-name
=DISPLAY_NAME
] [--labels
=[KEY
=VALUE
,…]] [--async
|--validate-only
] [--data-profile-spec-file
=DATA_PROFILE_SPEC_FILE
|--exclude-field-names
=EXCLUDE_FIELD_NAMES
--export-results-table
=EXPORT_RESULTS_TABLE
--include-field-names
=INCLUDE_FIELD_NAMES
--row-filter
=ROW_FILTER
--sampling-percent
=SAMPLING_PERCENT
] [--incremental-field
=INCREMENTAL_FIELD
--on-demand
=ON_DEMAND
|--schedule
=SCHEDULE
] [GCLOUD_WIDE_FLAG …
]
-
- DESCRIPTION
- Represents a user-visible job which provides the insights for the related data source about the structure, content and relationships (such as null percent, cardinality, min/max/mean, etc).
- EXAMPLES
-
To create a data profile scan
data-profile-datascan
in projecttest-project
located inus-central1
on bigquery resource tabletest-table
in datasettest-dataset
, run:gcloud dataplex datascans create data-profile data-profile-datascan --project=test-project --location=us-central1 --data-source-resource="//bigquery.googleapis.com/projects/test-project/datasets/test-dataset/tables/test-table"
- POSITIONAL ARGUMENTS
-
-
Datascan resource - Arguments and flags that define the Dataplex datascan you
want to create a data profile scan for. The arguments in this group can be used
to specify the attributes of this resource. (NOTE) Some attributes are not given
arguments in this group but can be set in other ways.
To set the
project
attribute:-
provide the argument
datascan
on the command line with a fully specified name; -
provide the argument
--project
on the command line; -
set the property
core/project
.
This must be specified.
DATASCAN
-
ID of the datascan or fully qualified identifier for the datascan.
To set the
dataScans
attribute:-
provide the argument
datascan
on the command line.
This positional argument must be specified if any of the other arguments in this group are specified.
-
provide the argument
--location
=LOCATION
-
The location of the Dataplex resource.
To set the
location
attribute:-
provide the argument
datascan
on the command line with a fully specified name; -
provide the argument
--location
on the command line; -
set the property
dataplex/location
.
-
provide the argument
-
provide the argument
-
Datascan resource - Arguments and flags that define the Dataplex datascan you
want to create a data profile scan for. The arguments in this group can be used
to specify the attributes of this resource. (NOTE) Some attributes are not given
arguments in this group but can be set in other ways.
- REQUIRED FLAGS
-
-
Data source for the data profile scan.
Exactly one of these must be specified:
--data-source-entity
=DATA_SOURCE_ENTITY
-
Dataplex entity that contains the data for the data profile scan, of the form:
projects/{project_number}/locations/{location_id}/lakes/{lake_id}/zones/{zone_id}/entities/{entity_id}
. --data-source-resource
=DATA_SOURCE_RESOURCE
-
Fully-qualified service resource name of the cloud resource that contains the
data for the data profile scan, of the form:
//bigquery.googleapis.com/projects/{project_number}/datasets/{dataset_id}/tables/{table_id}
.
-
Data source for the data profile scan.
- OPTIONAL FLAGS
-
--description
=DESCRIPTION
- Description of the data profile scan.
--display-name
=DISPLAY_NAME
- Display name of the data profile scan.
--labels
=[KEY
=VALUE
,…]-
List of label KEY=VALUE pairs to add.
Keys must start with a lowercase character and contain only hyphens (
-
), underscores (_
), lowercase characters, and numbers. Values must contain only hyphens (-
), underscores (_
), lowercase characters, and numbers. -
At most one of --async | --validate-only can be specified.
At most one of these can be specified:
--async
- Return immediately, without waiting for the operation in progress to complete.
--validate-only
- Validate the create action, but don't actually perform it.
-
Data spec for the data profile scan.
At most one of these can be specified:
--data-profile-spec-file
=DATA_PROFILE_SPEC_FILE
- path to the JSON/YAML file containing the spec for the data profile scan. The JSON representation reference: https://cloud.google.com/dataplex/docs/reference/rest/v1/DataProfileSpec
-
Command line spec arguments for the data profile scan.
--exclude-field-names
=EXCLUDE_FIELD_NAMES
-
Names of the fields to exclude from data profile. If specified, the respective
fields will be excluded from data profile, regardless of the fields specified in
the
--include-field-names
flag. --export-results-table
=EXPORT_RESULTS_TABLE
-
path to the resource table to export data profile scan results, of the form:
//bigquery.googleapis.com/projects/{project_number}/datasets/{dataset_id}/tables/{table_id}
. The table will be created if not present. --include-field-names
=INCLUDE_FIELD_NAMES
-
Names of the fields to include in data profile. If not specified, all fields at
the time of profile scan job execution are included. The fields listed in the
--exclude-field-names
flag are excluded. --row-filter
=ROW_FILTER
- A filter applied to all rows in a single data profile scan job.
--sampling-percent
=SAMPLING_PERCENT
- The percentage of the records to be selected from the dataset for data profile scan.
-
Data profile scan execution settings.
--incremental-field
=INCREMENTAL_FIELD
- Field that contains values that monotonically increase over time (e.g. timestamp).
-
Data profile scan scheduling and trigger settings.
At most one of these can be specified:
--on-demand
=ON_DEMAND
- If set, the scan runs one-time shortly after data profile scan creation.
--schedule
=SCHEDULE
-
Cron schedule (https://en.wikipedia.org/wiki/Cron) for running scans
periodically. To explicitly set a timezone to the cron tab, apply a prefix in
the cron tab: "CRON_TZ=${IANA_TIME_ZONE}" or "TZ=${IANA_TIME_ZONE}". The
${IANA_TIME_ZONE} may only be a valid string from IANA time zone database. For
example,
CRON_TZ=America/New_York 1 * * * *
orTZ=America/New_York 1 * * * *
. This field is required for RECURRING scans.
- GCLOUD WIDE FLAGS
-
These flags are available to all commands:
--access-token-file
,--account
,--billing-project
,--configuration
,--flags-file
,--flatten
,--format
,--help
,--impersonate-service-account
,--log-http
,--project
,--quiet
,--trace-token
,--user-output-enabled
,--verbosity
.Run
$ gcloud help
for details. - NOTES
-
This variant is also available:
gcloud alpha dataplex datascans create data-profile
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-02-06 UTC.