Create and manage connections

This page describes how to create and manage connections in Cloud Data Fusion version 6.5 and later.

A connection stores information, such as user credentials and host information, which is needed to connect to data sources. When you use a connection in a plugin, the plugin inherits all of the properties configured in the connection. After you create the connection, you can reuse it for other pipelines in the namespace. When you export a pipeline that has a plugin that uses a connection, the pipeline configuration JSON includes all of the properties.

You can manage connections for each namespace on the following pages in the Cloud Data Fusion Studio:

  • The Namespace Admin page lets you add, edit, delete, duplicate, export, and import connections for each namespace.

  • The Wrangler workspace page lets you add, edit, delete, duplicate, and export connections for each namespace.

  • The Studio page lets you add and import connections for supported plugins for each namespace.

Before you begin

  • Before you add a connection for a database source, you must upload the JDBC driver to Cloud Data Fusion. You can upload the driver from the Hub or the Namespace Admin page.

Supported plugins

The following plugins support reusable connections:

  • Amazon S3
  • BigQuery
  • Spanner
  • Cloud Storage
  • Cloud SQL for MySQL
  • Cloud SQL for PostgreSQL
  • Database
  • Kafka
  • Microsoft SQL Server
  • MySQL
  • Oracle
  • PostgreSQL
  • Salesforce
  • SAP Ariba
  • SAP BW Open Hub Batch Source
  • SAP ODP
  • SAP Table
  • Zendesk

Create a connection

To create a new connection, follow these steps:

  1. In the Cloud Data Fusion Studio, click Menu > Namespace Admin > Connections.
  2. Click Add connection.
  3. From the Add connection page, select the type of connection to create.

  4. Configure the connection properties.

  5. Optional: to make sure the connection works, click Test connection.

  6. Click Create.

The connection appears in the Connection type field and is available for all users in the namespace. The connection is available for use on the Wrangler workspace page and on the Studio page in the Wrangler plugin properties.

Use a connection in a plugin

When you use a connection in a plugin, the plugin inherits the connection properties that you configured in the previous step.

To use a connection in one of the supported plugins, follow these steps:

  1. In the Cloud Data Fusion Studio interface, open a pipeline on the Studio page.
  2. In the plugin pallet, expand the Source menu, and select a supported batch source plugin.
  3. Hold the pointer over the source node that appears and click Properties.
  4. Set Use connection to Yes.
  5. Click Browse connections. You can select an existing connection or create a new one. After you select the connection, all properties associated with the connection no longer appear. The connection properties are stored in the connection.
  6. Configure the rest of the required plugin properties.
  7. Optional: to test the connection, click Validate.
  8. Click Close.

What's next