This page gives you an overview of the various Mainframe Connector configurations, and the advantages of each approach.
You can run Mainframe Connector in the following configurations based on your requirements:
- Transcode mainframe data locally on the mainframe and then migrate it to Cloud Storage
- Transcode mainframe data on Google Cloud using Cloud Run
- Transfer mainframe data to Cloud Storage using a virtual tape library (VTL) and then transcode the data on Google Cloud
The following sections discuss these configurations in detail.
Move locally transcoded mainframe data to Google Cloud
You can transcode mainframe data locally on the mainframe to the Optimized Row Columnar (ORC) format, which is supported by BigQuery. In this configuration, Mainframe Connector helps you manage a complete extract, transform, and load (ETL) pipeline entirely from IBM z/OS, as shown in the following figure.
Transcode mainframe data remotely on Google Cloud using Cloud Run
Transcoding data locally on a mainframe is a CPU-intensive process that results in high million instructions per second (MIPS) consumption. To avoid this, you can delegate the transcoding of mainframe data to a Cloud Run service on Google Cloud, as shown in the following figure. This frees up your mainframe for business critical tasks and also reduces MIPS consumption.
Transcode mainframe data moved to Google Cloud using a virtual tape library
If you want to transfer very large volumes of data (around 500+ GB daily) to Google Cloud, and don't want to use your mainframe for this effort, you can deploy a hardware device in your data center to transfer data directly from the mainframe storage system to Cloud Storage using a VTL and 10G ethernet. As the hardware device receives the data directly from the mainframe storage system using a VTL, the data transfer process between the mainframe and Cloud Storage doesn't use the mainframe at all, thereby freeing it up for business critical tasks. Data transcoding is performed by a Cloud Run service on Google Cloud, as shown in the following figure.
