This page explains how to create a Cloud Data Fusion instance.
Before you begin
- Enable the Cloud Data Fusion API.
- The following permission is required to create Cloud Data Fusion
instances.create. For more information, see Access control.
- Cloud Data Fusion instances run as the Compute Engine default service account. For information about the types and roles available, see Service accounts.
- By default, Cloud Data Fusion executes pipelines using a Cloud Dataproc cluster in your project. Ensure that your project meets the Cloud Dataproc networking requirements.
- New projects start with a default network. The default network is pre-populated with a firewall rule, default-allow-ssh, that allows ingress connections on TCP port 22 from any source to any instance in the network. If such a rule doesn't exist in the network used by your Cloud Data Fusion instance, you need to create such a rule.
Creating an instance
If the API is enabled, the Cloud Data Fusion section in the GCP Console shows an Instances page where you can manage your Cloud Data Fusion instances. When no instances exist, the page has a link to create an instance, along with some useful links to documentation and samples.
- To create a Cloud Data Fusion instance, click the Create Instance link at the top of the section.
- Enter a name and a description for your instance. The name must start with a lowercase letter followed by up to 63 lowercase letters, numbers, or underscores.
- Specify the zone in which to create the instance
- Select the Cloud Data Fusion edition. For information about editions, see the pricing page.
Specify any additional settings. If you do not specify a setting, the following defaults are used.
Category Setting Description Default Logging and Monitoring Enable Stackdriver logging service Option to enable Stackdriver Logging. Disabled Enable Stackdriver monitoring service Option to enable Stackdriver Monitoring. Disabled Advanced Labels The resource labels for the instance to use to annotate any related underlying resources, such as Compute Engine VMs. Label keys and label values can only contain letters, numbers, dashes, and underscores. Label keys must start with a letter or number. None
Create an instance:
Setting up permissions
After creating an instance you need one additional step to grant the service account associated with the instance permissions on your project.
- Navigate to the instance detail page by clicking the instance name.
- Copy the service account
- Navigate to the IAM Page of your project.
- On the IAM Permissions page, grant the service account the Cloud Data Fusion API Service Agent role, by clicking the Add button.
- Click Save
Once these steps are done, you can start using Cloud Data Fusion by clicking the View Instance link on the Cloud Data Fusion instances page, or the details page of an instance.