Cloud Data Fusion Plugin Reference

Most popular

Quickly discover and access the most popular Cloud Data Fusion plugins.

BigQuery
This source reads the entire contents of a BigQuery table.
BigQuery Multi Table
This sink writes to multiple BigQuery tables. Data is first written to a temporary location on Cloud Storage, then loaded into BigQuery from there.
BigQuery Table
This sink writes to a BigQuery table. Data is first written to a temporary location on Cloud Storage, then loaded into BigQuery from there.
Cloud Storage
This source plugin reads objects from a path in a Cloud Storage bucket.
Cloud Storage
This sink plugin writes records to one or more files in a directory on Cloud Storage in various formats such as CSV, Avro, Parquet, and JSON.
Multi Database Table
Reads from multiple tables within a database. Used with multi sink plugins like Cloud Storage multi sink and BigQuery multi sink to load data from entire database in a single pipeline.
MySQL
Reads from a MySQL instance using a configurable SQL query. Outputs one record for each row returned by the query.
Oracle
Reads from an Oracle table using a configurable SQL query. Outputs one record for each row returned by the query.
Salesforce
This source reads sObjects such as opportunities, contacts, accounts, leads, any custom object, etc. from Salesforce in batch.
SAP ODP
Connects to an SAP source for batch-based and delta-based data extraction through Operational Data Provisioning (ODP).
SQL Server
Reads from a SQL Server using a configurable SQL query. Outputs one record for each row returned by the query.

Google Cloud

See all of the plugins available for Google Cloud products including Cloud Storage, BigQuery, Cloud Bigtable, Datastore, Pub/Sub, and Google Workspace.

Aggregate

Plugins to aggregate functions like as avg, count, count(*), first, last, max, min, sum, collect including table joins, etc.

Alerting

Plugins for alerting.

Apache

Plugins for Apache.

APIs

Plugins for various APIs.

Avro

Plugins for Avro.

AWS

Available plugins for AWS.

Cleansing

Plugins for cleansing.

Cloud Storage

Available plugins for cloud storage.

Code

Plugins to execute your own code.

Compression

Plugins for compression.

Conditional

Pipelines also offer control flow plugins in the form of conditions. Condition plugins allow you to branch your pipeline into two separate paths, depending on whether the specified condition predicate evaluates to true or false.

Control Flow

Plugins for control flow.

CRM

Plugins for CRM platforms like Salesforce.

CSV

Plugins for CSV.

Database

Plugins for your databases.

Data Warehouse

Plugins for your data warehouse.

DLP

Plugins for Cloud Data Loss Prevention (DLP).

Dynamic

Dynamic plugins.

Email

Plugins for email.

File

File plugins.

Filter

Filter plugins.

FTP

FTP plugins.

Google Analytics 360

Plugins for Google Analytics 360.

Google Workspace

Plugins for Google Workspace including Google Sheets and Google Drive.

Hadoop

Plugins for Hadoop.

HDFS

Plugins for HDFS.

HTTP

Plugins for HTTP.

Argument Setter
Performs an HTTP request to fetch arguments to set in the pipeline.

IBM

Plugins for IBM.

JDBC

Plugins for JDBC.

JSON

Plugins for JSON.

Kafka

Plugins for Kafka.

Logs

Plugins for logs.

Webserver Log Parser
Parses logs from any input source for relevant information such as URI, IP, browser, device, HTTP status code, and timestamp.

Mapping

Plugins for mapping.

Masking

Plugins for masking.

Microsoft

Plugins for Microsoft Excel and Windows.

Microsoft Azure

Plugins for Microsoft Azure data warehouse, databases, cloud storage, and more.

Nested

Plugins for nested data.

Flatten Record
Plugin to transform nested data into flat data.

NoSQL

Plugins for NoSQL.

Oracle

Plugins for Oracle.

Parquet

Plugins for Parquet.

Python

Plugins for Python.

S3

Plugins for S3.

Amazon S3
This source is used whenever you need to read from Amazon S3.

SAP

SaaS

Plugins for common SaaS products like Salesforce, ServiceNow, and Zendesk.

Search

Plugins for search.

Snowflake

Plugins for Snowflake.

Spark

Plugins for Spark.

SSH

Plugins for SSH.

Statistics

Plugins for statistics.

Streaming

Plugins for streaming products.

Testing

Plugins for testing.

Trash
Trash consumes all the records on the input and eats them all, which means no output is generated or no output is stored anywhere.

Cloud Data Fusion Plugins

Filters