Data Fusion SAP Connectors

Businesses today have a growing demand for data analysis and insight-based action. More often than not, the valuable data driving these actions is in mission critical operational systems. Among all the applications that are in the market today, SAP is the leading provider of ERP software and Google Cloud is introducing integration with SAP to help unlock the value of SAP data quickly and easily.

Google Cloud’s native data integration platform Cloud Data Fusion now offers an additional capability to seamlessly get data out of SAP Business Suite, SAP ERP and S/4HANA. Cloud Data Fusion is a fully managed, cloud-native data integration and ingestion service that helps ETL developers, data engineers and business analysts efficiently build and manage ETL/ELT pipelines.  These pipelines accelerate the creation of data warehouses, data marts, and data lakes on BigQuery or operational reporting systems on CloudSQL, Spanner or other systems. 

To simplify the unlocking of SAP data, today we’re announcing the launch of SAP ODP General Availability, SAP SLT and SAP ODATA in Preview.  This will allow customers to have the flexibility to unlock data from SAP ECC and S4HANA in batch or real-time in a very easy  and code free way. This blog describes 3 connectors: SAP ODP, SLT connector,  and SAP ODATA. 

SAP ODP (Open Data Provisioning) Connector ^GA 

Transfer Full or Delta data from SAP to BigQuery or other systems, using ERP Extractors (Datasources).

In Cloud Data Fusion’s Pipeline Studio, you can add an SAP source Datasource to a data pipeline. If needed, add a Transform option, like Wrangler. Then add a sink like BigQuery or GCS.

There are 2 SAP connection options: Direct and via Load Balancer. 

Data can be transferred in 2 modes: Full (All Data) and Sync (Automatic selection based on previous execution). 

Sync mode automatically determines whether full, delta (incremental), or recovery (recover data from last execution) mode should be run based on the previous execution type and status available in SAP. It extracts full data in the initial pipeline execution (ODP mode F) and changes data in subsequent pipeline executions (ODP modes D, R).

Clicking Validate will check the properties, connect to SAP and retrieve the schema. 

SLT (SAP Landscape Transformation) Connector ^{Public Preview} 

SAP SLT (Landscape Transformation Server) is an SAP solution dedicated to replicating data within the SAP ecosystem and to external systems. 

Replicate data from SAP to BigQuery, using SAP SLT

There are a few configuration prerequisites for SLT 

• deploying the SAP transport, 

• setting up the replication configuration and 

• the Google ABAP SDK. 

In CDF, the user needs to provide the path where the SAP JCo libraries were downloaded from SAP.com

Google Cloud Storage is used as the staging area for the data pushed out of SLT, so make sure to configure the capacity/quota appropriately.  

In the Replication studio, you can add an SAP SLT Replicator source to a replication job. Then add the Big Query target.

To configure the SLT plugin, provide the connection parameters (SAP host, user, password) and SLT GUID and Mass Transfer ID. Then select the tables to be replicated.

SAP ODATA ^{Public Preview} 

Transfer data from SAP to BigQuery or other systems, using SAP OData Services

In the Cloud Data Fusion’s Pipeline Studio, you can add an SAP source OData Service to a data pipeline. If needed add a Transform option, like Wrangler. Then add a sink like Big Query or GCS.

The connection to SAP is done via http/s. The authentication method is through SAP user and password. An optional X.509 Client certificate can be used to improve security.  

The user must specify an OData base URL, a service name and entity to extract. Clicking Validate will check the properties, connect to SAP and retrieve the schema.