DataFusionInstance


Property Value
Google Cloud Service Name Data Fusion
Google Cloud Service Documentation /data-fusion/docs/
Google Cloud REST Resource Name v1beta1/projects.locations.instances
Google Cloud REST Resource Documentation /data-fusion/docs/reference/rest/v1beta1/projects.locations.instances
Config Connector Resource Short Names gcpdatafusioninstance
gcpdatafusioninstances
datafusioninstance
Config Connector Service Name datafusion.googleapis.com
Config Connector Resource Fully Qualified Name datafusioninstances.datafusion.cnrm.cloud.google.com
Can Be Referenced by IAMPolicy/IAMPolicyMember No
Config Connector Default Average Reconcile Interval In Seconds 600

Custom Resource Definition Properties

Annotations

Fields
cnrm.cloud.google.com/project-id

Spec

Schema

dataprocServiceAccountRef:
  external: string
  name: string
  namespace: string
description: string
displayName: string
enableStackdriverLogging: boolean
enableStackdriverMonitoring: boolean
location: string
networkConfig:
  ipAllocation: string
  networkRef:
    external: string
    name: string
    namespace: string
options:
  string: string
privateInstance: boolean
resourceID: string
type: string
version: string
zone: string
Fields

dataprocServiceAccountRef

Optional

object

dataprocServiceAccountRef.external

Optional

string

User-managed service account to set on Dataproc when Cloud Data Fusion creates Dataproc to run data processing pipelines. This allows users to have fine-grained access control on Dataproc's accesses to cloud resources. Allowed value: The `email` field of an `IAMServiceAccount` resource.

dataprocServiceAccountRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

dataprocServiceAccountRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

description

Optional

string

Immutable. A description of this instance.

displayName

Optional

string

Immutable. Display name for an instance.

enableStackdriverLogging

Optional

boolean

Option to enable Stackdriver Logging.

enableStackdriverMonitoring

Optional

boolean

Option to enable Stackdriver Monitoring.

location

Required

string

Immutable. The location for the resource

networkConfig

Optional

object

Immutable. Network configuration options. These are required when a private Data Fusion instance is to be created.

networkConfig.ipAllocation

Optional

string

Immutable. The IP range in CIDR notation to use for the managed Data Fusion instance nodes. This range must not overlap with any other ranges used in the customer network.

networkConfig.networkRef

Optional

object

Immutable.

networkConfig.networkRef.external

Optional

string

Name of the network in the customer project with which the Tenant Project will be peered for executing pipelines. In case of shared VPC where the network resides in another host project the network should specified in the form of projects/{host-project-id}/global/networks/{network} Allowed value: The Google Cloud resource name of a `ComputeNetwork` resource (format: `projects/{{project}}/global/networks/{{name}}`).

networkConfig.networkRef.name

Optional

string

Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names

networkConfig.networkRef.namespace

Optional

string

Namespace of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

options

Optional

map (key: string, value: string)

Immutable. Map of additional options used to configure the behavior of Data Fusion instance.

privateInstance

Optional

boolean

Immutable. Specifies whether the Data Fusion instance should be private. If set to true, all Data Fusion nodes will have private IP addresses and will not be able to access the public internet.

resourceID

Optional

string

Immutable. Optional. The name of the resource. Used for creation and acquisition. When unset, the value of `metadata.name` is used as the default.

type

Required

string

Immutable. Required. Instance type. Possible values: TYPE_UNSPECIFIED, BASIC, ENTERPRISE, DEVELOPER

version

Optional

string

Current version of the Data Fusion.

zone

Optional

string

Immutable. Name of the zone in which the Data Fusion instance will be created. Only DEVELOPER instances use this field.

Status

Schema

apiEndpoint: string
availableVersion:
- availableFeatures:
  - string
  defaultVersion: boolean
  versionNumber: string
conditions:
- lastTransitionTime: string
  message: string
  reason: string
  status: string
  type: string
createTime: string
gcsBucket: string
observedGeneration: integer
p4ServiceAccount: string
serviceEndpoint: string
state: string
stateMessage: string
tenantProjectId: string
updateTime: string
Fields
apiEndpoint

string

Output only. Endpoint on which the REST APIs is accessible.

availableVersion

list (object)

Available versions that the instance can be upgraded to.

availableVersion[]

object

availableVersion[].availableFeatures

list (string)

Represents a list of available feature names for a given version.

availableVersion[].availableFeatures[]

string

availableVersion[].defaultVersion

boolean

Whether this is currently the default version for Cloud Data Fusion

availableVersion[].versionNumber

string

The version number of the Data Fusion instance, such as '6.0.1.0'.

conditions

list (object)

Conditions represent the latest available observation of the resource's current state.

conditions[]

object

conditions[].lastTransitionTime

string

Last time the condition transitioned from one status to another.

conditions[].message

string

Human-readable message indicating details about last transition.

conditions[].reason

string

Unique, one-word, CamelCase reason for the condition's last transition.

conditions[].status

string

Status is the status of the condition. Can be True, False, Unknown.

conditions[].type

string

Type is the type of the condition.

createTime

string

Output only. The time the instance was created.

gcsBucket

string

Output only. Cloud Storage bucket generated by Data Fusion in the customer project.

observedGeneration

integer

ObservedGeneration is the generation of the resource that was most recently observed by the Config Connector controller. If this is equal to metadata.generation, then that means that the current reported status reflects the most recent desired state of the resource.

p4ServiceAccount

string

Output only. P4 service account for the customer project.

serviceEndpoint

string

Output only. Endpoint on which the Data Fusion UI is accessible.

state

string

Output only. The current state of this Data Fusion instance. Possible values: STATE_UNSPECIFIED, ENABLED, DISABLED, UNKNOWN

stateMessage

string

Output only. Additional information about the current state of this Data Fusion instance if available.

tenantProjectId

string

Output only. The name of the tenant project.

updateTime

string

Output only. The time the instance was last updated.

Sample YAML(s)

Typical Use Case

# Copyright 2021 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

apiVersion: datafusion.cnrm.cloud.google.com/v1beta1
kind: DataFusionInstance
metadata:
  labels:
    label-one: value-one
  name: datafusioninstance-sample
spec:
  description: A sample DataFusion instance.
  displayName: Sample DataFusion Instance
  location: us-central1
  type: BASIC
  enableStackdriverMonitoring: true
  enableStackdriverLogging: true
  privateInstance: true
  networkConfig:
    networkRef:
      name: datafusioninstance-dep
    ipAllocation: 10.89.48.0/22
  dataprocServiceAccountRef:
    name: datafusioninstance-dep
---
apiVersion: compute.cnrm.cloud.google.com/v1beta1
kind: ComputeNetwork
metadata:
  name: datafusioninstance-dep
spec:
  routingMode: GLOBAL
  autoCreateSubnetworks: false
---
apiVersion: iam.cnrm.cloud.google.com/v1beta1
kind: IAMServiceAccount
metadata:
  name: datafusioninstance-dep
spec:
  displayName: DataFusionInstance Service Account