Creating a Cloud Data Fusion instance

This page explains how to create a Cloud Data Fusion instance.

Before you begin

• Enable the Cloud Data Fusion API.
• The following permission is required to create Cloud Data Fusion instances: instances.create. For more information, see Access control.
• Cloud Data Fusion instances run as the Compute Engine default service account. For information about the types and roles available, see Service accounts.
• By default, Cloud Data Fusion executes pipelines using a Dataproc cluster in your project. Ensure that your project meets the Dataproc networking requirements.
• New projects start with a default network. The default network is pre-populated with a firewall rule, default-allow-ssh, that allows incoming connections on TCP port 22 from any source to any instance in the network. In versions below 6.2.0, if this rule to allow ingress on TCP port 22 rule doesn't exist in the network used by your Cloud Data Fusion instance, you must create this rule.

Creating an instance

Instance creation options: You can select the following options when you create an instance. After you create an instance, you can't select or remove these options:

• Cloud Logging and Cloud Monitoring
• Private IP

Console

If the API is enabled, the Cloud Data Fusion section in the Cloud Console shows an Instances page where you can manage your Cloud Data Fusion instances. When no instances exist, the page has a link to create an instance, along with some useful links to documentation and samples.

1. Click Create Instance.

   OPEN THE CREATE INSTANCE PAGE

2. Enter an Instance name.
3. Enter a Description for your instance.
4. Specify the Region in which to create the instance.
5. Specify the Cloud Data Fusion Version you prefer.
6. Select the Cloud Data Fusion Edition you prefer.
7. In Cloud Data Fusion version 6.2.3 and higher, specify the Dataproc service account to use for running your Cloud Data Fusion pipeline in Dataproc. The UI pre-selects the default Compute Engine account. Regardless of version, make sure that a service account has appropriate Identity and Access Management roles for your needs. For more information, see Granting service account user permission.

8. Specify any additional settings. If you do not specify anything for the additional settings, the following defaults are used:

   Category	Setting	Description	Default
   Edition	Developer, Basic or Enterprise	Instance and pipeline features	Basic
   Advanced Options
   Private IP	Enable private IP addresses	Create a Cloud Data Fusion instance that uses private IP addresses.	Disabled
   Logging and Monitoring	Enable Cloud Logging service	Option to enable Cloud Logging	Disabled
   	Enable Cloud Monitoring service	Option to enable Cloud Monitoring	Disabled
   Labels	<Key> <Value> pair(s)	The resource labels for the instance to use to annotate any related underlying resources, such as Compute Engine VMs. Label keys and label values can only contain letters, numbers, dashes, and underscores. Label keys must start with a letter or number.	None

9. Click Create. It takes up to 30 minutes for the instance creation process to complete.

Note: While Cloud Data Fusion creates your instance, a progress wheel displays next to the instance name on the Instances page. After completion, it turns into a green check mark and indicates that you can start using the instance.

API

Create an instance:

To create a Cloud Data Fusion instance with the Cloud Data Fusion REST API, construct an instances.create API request, filling in the Instance resource with your configuration information.

What's next

• Learn more about Cloud Data Fusion.
• Work through a tutorial.