DataQualityRule(mapping=None, *, ignore_unknown_fields=False, **kwargs)
A rule captures data quality intent about a data source.
This message has oneof
_ fields (mutually exclusive fields).
For each oneof, at most one member field can be set at the same time.
Setting any member of the oneof automatically clears all other
members.
.. _oneof: https://proto-plus-python.readthedocs.io/en/stable/fields.html#oneofs-mutually-exclusive-fields
Attributes
Name | Description |
range_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.RangeExpectation
ColumnMap rule which evaluates whether each column value lies between a specified range. This field is a member of oneof _ rule_type .
|
non_null_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.NonNullExpectation
ColumnMap rule which evaluates whether each column value is null. This field is a member of oneof _ rule_type .
|
set_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.SetExpectation
ColumnMap rule which evaluates whether each column value is contained by a specified set. This field is a member of oneof _ rule_type .
|
regex_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.RegexExpectation
ColumnMap rule which evaluates whether each column value matches a specified regex. This field is a member of oneof _ rule_type .
|
uniqueness_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.UniquenessExpectation
ColumnAggregate rule which evaluates whether the column has duplicates. This field is a member of oneof _ rule_type .
|
statistic_range_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.StatisticRangeExpectation
ColumnAggregate rule which evaluates whether the column aggregate statistic lies between a specified range. This field is a member of oneof _ rule_type .
|
row_condition_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.RowConditionExpectation
Table rule which evaluates whether each row passes the specified condition. This field is a member of oneof _ rule_type .
|
table_condition_expectation |
google.cloud.dataplex_v1.types.DataQualityRule.TableConditionExpectation
Table rule which evaluates whether the provided expression is true. This field is a member of oneof _ rule_type .
|
column |
str
Optional. The unnested column which this rule is evaluated against. |
ignore_null |
bool
Optional. Rows with null values will automatically fail a rule, unless ignore_null is true. In that case, such null rows are trivially considered passing. Only applicable to ColumnMap rules. |
dimension |
str
Required. The dimension a rule belongs to. Results are also aggregated at the dimension-level. Supported dimensions are ["COMPLETENESS", "ACCURACY", "CONSISTENCY", "VALIDITY", "UNIQUENESS", "INTEGRITY"] |
threshold |
float
Optional. The minimum ratio of passing_rows / total_rows required to pass this rule, with a range of [0.0, 1.0] 0 indicates default value (i.e. 1.0) |
Classes
NonNullExpectation
NonNullExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether each column value is null.
RangeExpectation
RangeExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether each column value lies between a specified range.
RegexExpectation
RegexExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether each column value matches a specified regex.
RowConditionExpectation
RowConditionExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether each row passes the specified condition. The SQL expression needs to use BigQuery standard SQL syntax and should produce a boolean per row as the result. Example: col1 >= 0 AND col2 < 10
SetExpectation
SetExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether each column value is contained by a specified set.
StatisticRangeExpectation
StatisticRangeExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether the column aggregate statistic lies between a specified range.
TableConditionExpectation
TableConditionExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether the provided expression is true. The SQL expression needs to use BigQuery standard SQL syntax and should produce a scalar boolean result. Example: MIN(col1) >= 0
UniquenessExpectation
UniquenessExpectation(mapping=None, *, ignore_unknown_fields=False, **kwargs)
Evaluates whether the column has duplicates.