Cloud Composer

A fully managed workflow orchestration service built on Apache Airflow.

View documentation for this product.

Complex workflows, simplified

Cloud Composer is a fully managed workflow orchestration service that empowers you to author, schedule, and monitor pipelines that span across clouds and on-premises data centers. Built on the popular Apache Airflow open source project and operated using the Python programming language, Cloud Composer is free from lock-in and easy to use.

Making orchestration easy

Cloud Composer pipelines are configured as directed acyclic graphs (DAGs) using Python, making it easy for users of any experience level to author and schedule a workflow. One-click deployment yields instant access to a rich library of connectors and multiple graphical representations of your workflow in action, increasing pipeline reliability by making troubleshooting easy. Automatic synchronization of your directed acyclic graphs ensures your jobs stay on schedule.

End-to-end integration for GCP workloads

Cloud Composer is deeply integrated within the Google Cloud Platform, giving users the ability to orchestrate their full pipeline. Cloud Composer has robust, built-in integration with many products, including BigQuery, Cloud Dataflow, Cloud Dataproc, Cloud Datastore, Cloud Storage, Cloud Pub/Sub, and AI Platform.

Create a hybrid and multi-cloud environment

Cloud Composer gives you the ability to connect your pipeline through a single orchestration tool whether your workflow lives on-premises, in multiple clouds, or fully within GCP. The ability to author, schedule, and monitor your workflows in a unified manner means you can break down the silos in your environment and focus less on infrastructure.

Open source at its core

Cloud Composer is built on Apache Airflow, the popular open source orchestration tool. This open source project, which Google is contributing back into, provides freedom from lock-in for customers as well as integration with a broad number of platforms, which will only expand as the Airflow community grows.

Features

Multi-cloud

Create workflows that connect data, processing, and services across clouds, giving you a unified data environment.

Open source

Cloud Composer is built upon Apache Airflow, giving users freedom from lock-in and portability.

Hybrid

Ease your transition to the cloud or maintain a hybrid data environment by orchestrating workflows that cross between on-premises and the public cloud.

Integrated

Built-in integration with BigQuery, Cloud Dataflow, Cloud Dataproc, Cloud Datastore, Cloud Storage, Cloud Pub/Sub, AI Platform, and more, giving you the ability to orchestrate end-to-end GCP workloads.

Python programming language

Leverage existing Python skills to dynamically author and schedule workflows within Cloud Composer.

Reliability

Increase reliability of your workflows through easy-to-use charts for monitoring and troubleshooting the root cause of an issue.

Fully managed

Cloud Composer's managed nature allows you to focus on authoring, scheduling, and monitoring your workflows as opposed to provisioning resources.

At Blue Apron, our data workflows need to operate flawlessly to enable on-time delivery of perishable food. Cloud Composer helps us orchestrate our pipelines more reliably by allowing us to author, schedule, and monitor our workflows from one place using Python.

Michael Collis, Staff Software Engineer, Blue Apron

Technical resources

Pricing

Pricing for Cloud Composer is consumption based, so you pay for what you use, as measured by vCPU/hour, GB/month, and GB transferred/month. We have multiple pricing units because Cloud Composer uses several GCP products as building blocks.

Pricing is uniform across all levels of consumption and sustained usage.

Google Cloud

Get started

Learn and build

New customers get $300 in free credits to learn and build on Google Cloud for up to 12 months.

Need more help?

Our experts will help you build the right solution or find the right partner for your needs.

Wyślij opinię na temat...