- Resource: Instance
- Type
- NetworkConfig
- ConnectionType
- PrivateServiceConnectConfig
- State
- Version
- Type
- Accelerator
- AcceleratorType
- State
- CryptoKeyConfig
- DisabledReason
- EventPublishConfig
- MaintenancePolicy
- MaintenanceWindow
- RecurringTimeWindow
- TimeWindow
- Methods
Resource: Instance
Represents a Data Fusion instance.
JSON representation |
---|
{ "name": string, "description": string, "type": enum ( |
Fields | |
---|---|
name |
Output only. The name of this instance is in the form of projects/{project}/locations/{location}/instances/{instance}. |
description |
A description of this instance. |
type |
Required. Instance type. |
enableStackdriverLogging |
Option to enable Stackdriver Logging. |
enableStackdriverMonitoring |
Option to enable Stackdriver Monitoring. |
privateInstance |
Specifies whether the Data Fusion instance should be private. If set to true, all Data Fusion nodes will have private IP addresses and will not be able to access the public internet. |
networkConfig |
Network configuration options. These are required when a private Data Fusion instance is to be created. |
labels |
The resource labels for instance to use to annotate any related underlying resources such as Compute Engine VMs. The character '=' is not allowed to be used within the labels. An object containing a list of |
options |
Map of additional options used to configure the behavior of Data Fusion instance. An object containing a list of |
createTime |
Output only. The time the instance was created. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
updateTime |
Output only. The time the instance was last updated. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
state |
Output only. The current state of this Data Fusion instance. |
stateMessage |
Output only. Additional information about the current state of this Data Fusion instance if available. |
serviceEndpoint |
Output only. Endpoint on which the Data Fusion UI is accessible. |
zone |
Name of the zone in which the Data Fusion instance will be created. Only DEVELOPER instances use this field. |
version |
Current version of the Data Fusion. Only specifiable in Update. |
serviceAccount |
Output only. Deprecated. Use tenantProjectId instead to extract the tenant project ID. |
displayName |
Display name for an instance. |
availableVersion[] |
Output only. Available versions that the instance can be upgraded to using UpdateInstanceRequest. |
apiEndpoint |
Output only. Endpoint on which the REST APIs is accessible. |
gcsBucket |
Output only. Cloud Storage bucket generated by Data Fusion in the customer project. |
accelerators[] |
Output only. List of accelerators enabled for this CDF instance. |
p4ServiceAccount |
Output only. Service agent for the customer project. |
tenantProjectId |
Output only. The name of the tenant project. |
dataprocServiceAccount |
User-managed service account to set on Dataproc when Cloud Data Fusion creates Dataproc to run data processing pipelines. This allows users to have fine-grained access control on Dataproc's accesses to cloud resources. |
enableRbac |
Option to enable granular role-based access control. |
cryptoKeyConfig |
The crypto key configuration. This field is used by the Customer-Managed Encryption Keys (CMEK) feature. |
disabledReason[] |
Output only. If the instance state is DISABLED, the reason for disabling the instance. |
eventPublishConfig |
Option to enable and pass metadata for event publishing. |
enableZoneSeparation |
Option to enable granular zone separation. |
satisfiesPzs |
Output only. Reserved for future use. |
workforceIdentityServiceEndpoint |
Output only. Endpoint on which the Data Fusion UI is accessible to third-party users |
patchRevision |
Optional. Current patch revision of the Data Fusion. |
dataplexDataLineageIntegrationEnabled |
Optional. Option to enable the Dataplex Lineage Integration feature. |
maintenancePolicy |
Optional. Configure the maintenance policy for this instance. |
Type
Represents the type of Data Fusion instance. Each type is configured with the default settings for processing and memory.
Enums | |
---|---|
TYPE_UNSPECIFIED |
No type specified. The instance creation will fail. |
BASIC |
Basic Data Fusion instance. In Basic type, the user will be able to create data pipelines using point and click UI. However, there are certain limitations, such as fewer number of concurrent pipelines, no support for streaming pipelines, etc. |
ENTERPRISE |
Enterprise Data Fusion instance. In Enterprise type, the user will have all features available, such as support for streaming pipelines, higher number of concurrent pipelines, etc. |
DEVELOPER |
Developer Data Fusion instance. In Developer type, the user will have all features available but with restrictive capabilities. This is to help enterprises design and develop their data ingestion and integration pipelines at low cost. |
NetworkConfig
Network configuration for a Data Fusion instance. These configurations are used for peering with the customer network. Configurations are optional when a public Data Fusion instance is to be created. However, providing these configurations allows several benefits, such as reduced network latency while accessing the customer resources from managed Data Fusion instance nodes, as well as access to the customer on-prem resources.
JSON representation |
---|
{ "network": string, "ipAllocation": string, "connectionType": enum ( |
Fields | |
---|---|
network |
Optional. Name of the network in the customer project with which the Tenant Project will be peered for executing pipelines. In case of shared VPC where the network resides in another host project the network should specified in the form of projects/{host-project-id}/global/networks/{network}. This is only required for connectivity type VPC_PEERING. |
ipAllocation |
Optional. The IP range in CIDR notation to use for the managed Data Fusion instance nodes. This range must not overlap with any other ranges used in the Data Fusion instance network. This is required only when using connection type VPC_PEERING. Format: a.b.c.d/22 Example: 192.168.0.0/22 |
connectionType |
Optional. Type of connection for establishing private IP connectivity between the Data Fusion customer project VPC and the corresponding tenant project from a predefined list of available connection modes. If this field is unspecified for a private instance, VPC peering is used. |
privateServiceConnectConfig |
Optional. Configuration for Private Service Connect. This is required only when using connection type PRIVATE_SERVICE_CONNECT_INTERFACES. |
ConnectionType
Represents connection type between the consumer project VPC and the corresponding Data Fusion tenant project, from a predefined list of available connection modes.
Enums | |
---|---|
CONNECTION_TYPE_UNSPECIFIED |
No specific connection type was requested, the default value of VPC_PEERING is chosen. |
VPC_PEERING |
Requests the use of VPC peerings for connecting the consumer and tenant projects. |
PRIVATE_SERVICE_CONNECT_INTERFACES |
Requests the use of Private Service Connect Interfaces for connecting the consumer and tenant projects. |
PrivateServiceConnectConfig
Configuration for using Private Service Connect to establish connectivity between the Data Fusion consumer project and the corresponding tenant project.
JSON representation |
---|
{ "networkAttachment": string, "unreachableCidrBlock": string, "effectiveUnreachableCidrBlock": string } |
Fields | |
---|---|
networkAttachment |
Required. The reference to the network attachment used to establish private connectivity. It will be of the form projects/{project-id}/regions/{region}/networkAttachments/{network-attachment-id}. |
unreachableCidrBlock |
Optional. Input only. The CIDR block to which the CDF instance can't route traffic to in the consumer project VPC. The size of this block should be at least /25. This range should not overlap with the primary address range of any subnetwork used by the network attachment. This range can be used for other purposes in the consumer VPC as long as there is no requirement for CDF to reach destinations using these addresses. If this value is not provided, the server chooses a non RFC 1918 address range. The format of this field is governed by RFC 4632. Example: 192.168.0.0/25 |
effectiveUnreachableCidrBlock |
Output only. The CIDR block to which the CDF instance can't route traffic to in the consumer project VPC. The size of this block is /25. The format of this field is governed by RFC 4632. Example: 240.0.0.0/25 |
State
Represents the state of a Data Fusion instance
Enums | |
---|---|
STATE_UNSPECIFIED |
Instance does not have a state yet |
CREATING |
Instance is being created |
ACTIVE |
Instance is active and ready for requests. This corresponds to 'RUNNING' in datafusion.v1beta1. |
FAILED |
Instance creation failed |
DELETING |
Instance is being deleted |
UPGRADING |
Instance is being upgraded |
RESTARTING |
Instance is being restarted |
UPDATING |
Instance is being updated on customer request |
AUTO_UPDATING |
Instance is being auto-updated |
AUTO_UPGRADING |
Instance is being auto-upgraded |
DISABLED |
Instance is disabled |
Version
The Data Fusion version. This proto message stores information about certain Data Fusion version, which is used for Data Fusion version upgrade.
JSON representation |
---|
{
"versionNumber": string,
"defaultVersion": boolean,
"availableFeatures": [
string
],
"type": enum ( |
Fields | |
---|---|
versionNumber |
The version number of the Data Fusion instance, such as '6.0.1.0'. |
defaultVersion |
Whether this is currently the default version for Cloud Data Fusion |
availableFeatures[] |
Represents a list of available feature names for a given version. |
type |
Type represents the release availability of the version |
Type
Each type represents the release availability of a CDF version
Enums | |
---|---|
TYPE_UNSPECIFIED |
Version does not have availability yet |
TYPE_PREVIEW |
Version is under development and not considered stable |
TYPE_GENERAL_AVAILABILITY |
Version is available for public use |
Accelerator
Identifies Data Fusion accelerators for an instance.
JSON representation |
---|
{ "acceleratorType": enum ( |
Fields | |
---|---|
acceleratorType |
Optional. The type of an accelator for a Cloud Data Fusion instance. |
state |
Output only. The state of the accelerator. |
AcceleratorType
Each type represents an Accelerator (Add-On) supported by Cloud Data Fusion service.
Enums | |
---|---|
ACCELERATOR_TYPE_UNSPECIFIED |
Default value, if unspecified. |
CDC |
Change Data Capture accelerator for Cloud Data Fusion. |
HEALTHCARE |
Reserved for internal use. |
CCAI_INSIGHTS |
Contact Center AI Insights This accelerator is used to enable import and export pipelines custom built to streamline CCAI Insights processing. |
CLOUDSEARCH |
Reserved for internal use. |
State
Different values possible for the state of an accelerator.
Enums | |
---|---|
STATE_UNSPECIFIED |
Default value, do not use. |
ENABLED |
Indicates that the accelerator is enabled and available to use. |
DISABLED |
Indicates that the accelerator is disabled and not available to use. |
UNKNOWN |
Indicates that accelerator state is currently unknown. Requests for enable, disable could be retried while in this state. |
CryptoKeyConfig
The crypto key configuration. This field is used by the Customer-managed encryption keys (CMEK) feature.
JSON representation |
---|
{ "keyReference": string } |
Fields | |
---|---|
keyReference |
The name of the key which is used to encrypt/decrypt customer data. For key in Cloud KMS, the key should be in the format of |
DisabledReason
The reason for disabling the instance if the state is DISABLED.
Enums | |
---|---|
DISABLED_REASON_UNSPECIFIED |
This is an unknown reason for disabling. |
KMS_KEY_ISSUE |
The KMS key used by the instance is either revoked or denied access to |
EventPublishConfig
Confirguration of PubSubEventWriter.
JSON representation |
---|
{ "enabled": boolean, "topic": string } |
Fields | |
---|---|
enabled |
Required. Option to enable Event Publishing. |
topic |
Required. The resource name of the Pub/Sub topic. Format: projects/{projectId}/topics/{topic_id} |
MaintenancePolicy
Maintenance policy of the instance.
JSON representation |
---|
{
"maintenanceWindow": {
object ( |
Fields | |
---|---|
maintenanceWindow |
Optional. The maintenance window of the instance. |
MaintenanceWindow
Maintenance window of the instance.
JSON representation |
---|
{
"recurringTimeWindow": {
object ( |
Fields | |
---|---|
recurringTimeWindow |
Required. The recurring time window of the maintenance window. |
RecurringTimeWindow
Represents an arbitrary window of time that recurs.
JSON representation |
---|
{
"window": {
object ( |
Fields | |
---|---|
window |
Required. The window representing the start and end time of recurrences. This field ignores the date components of the provided timestamps. Only the time of day and duration between start and end time are relevant. |
recurrence |
Required. An RRULE with format RFC-5545 for how this window reccurs. They go on for the span of time between the start and end time. The only supported FREQ value is "WEEKLY". To have something repeat every weekday, use: "FREQ=WEEKLY;BYDAY=MO,TU,WE,TH,FR". This specifies how frequently the window starts. To have a 9 am - 5 pm UTC-4 window every weekday, use something like:
|
TimeWindow
Represents an arbitrary window of time.
JSON representation |
---|
{ "startTime": string, "endTime": string } |
Fields | |
---|---|
startTime |
Required. The start time of the time window provided in RFC 3339 format. Example: "2024-01-01T12:04:06-04:00" A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
endTime |
Required. The end time of the time window provided in RFC 3339 format. The end time should take place after the start time. Example: "2024-01-02T12:04:06-06:00" A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
Methods |
|
---|---|
|
Creates a new Data Fusion instance in the specified project and location. |
|
Deletes a single Date Fusion instance. |
|
Gets details of a single Data Fusion instance. |
|
Gets the access control policy for a resource. |
|
Lists Data Fusion instances in the specified project and location. |
|
Updates a single Data Fusion instance. |
|
Restart a single Data Fusion instance. |
|
Sets the access control policy on the specified resource. |
|
Returns permissions that a caller has on the specified resource. |