Data handling and analysis overview | Documentation

Data analytics and pipelines overview

Load, transform, and analyze data to provide business intelligence insights.

Explore data analytics and pipelines in Google Cloud

Read documentation and Cloud Architecture Center articles about data analytics and pipeline products, capabilities, and procedures.

Introduction to loading data into BigQuery

Learn how to batch load or stream data into BigQuery.

Data and analytics resources

Plan your approach with Architecture Center resources across a variety of data and analytics topics.open_in_new

Jump start solution: analytics lakehouse

Plan how to design an analytics lakehouse to store, process, and activate data.open_in_new

Training, blog articles, and more

Go to training courses, blog articles, and other related resources.

Data analyst learning path

Study Google Cloud technologies essential to the data analyst role.open_in_new

BI and analytics with Looker learning path

Study how to explore data in Looker and set up self-serve analytics for your users.open_in_new

Decision tree for data & analytics workloads

View a decision tree to choose services for running data and analytics workloads.open_in_new

Data analytics and pipelines products by use case

Expand sections or use the filter to find products and guides for typical use cases.

Data analysis and pipelines platforms and product suites

Analyze your local and external data with machine learning and AI tools, then securely share and visualize insights - with zero infrastructure management.

BigQuery

Understand your data using a fully managed, highly scalable data warehouse with built-in ML.

BigLake in BigQuery

Query structured data in external data stores with access delegation.

BigQuery Storage

Use storage optimized for running analytic queries over large datasets and high-throughput streaming ingestion and high-throughput reads.

BigQuery Engine for Apache Flink

Managed cloud service that lets you set up, maintain, and administer open source Apache Flink in Google Cloud.

Introduction to migrating to BigQuery

Migrate your data warehouse to BigQuery using free-to-use tools that help you with each phase of migration.

Data analysis

Query massive datasets with SQL, geospatial analytics, and BI tools that support ad hoc and programmatic analysis as well as data sharing.

Analytics Hub

Share data and insights at scale across organizational boundaries with a robust security and privacy framework.

BigQuery Analytics

Maximize your data analysis investments when running analytic queries on large datasets.

BigQuery ML

Create and run machine learning (ML) models by using GoogleSQL queries, and access LLMs and Cloud AI APIs to perform artificial intelligence (AI) tasks like text generation or machine translation.

Dataproc

Perform batch processing, querying, and streaming using a managed Apache Spark and Hadoop service.

Earth Engine

Google Earth Engine is a geospatial processing service. With Earth Engine, you can perform geospatial processing at scale, powered by Google Cloud Platform.

Looker

Access, analyze, and act on an up-to-date, trusted version of your data

Looker Studio

Tell great data stories to support better business decisions.

Data governance

Control and manage quality throughout the lifecycle of your data as you share it across and outside your organization.

Data Catalog

Discover and understand your data using a fully managed and scalable data discovery and metadata management service.

Dataplex

Organize your data into lakes and zones, and automate data management and governance to power analytics at scale.

Sensitive Data Protection

Discover and redact sensitive data.

Introduction to data governance in BigQuery

Implement and enforce BigQuery data governance policies.

Data ingestion

Migrate, stream, and batch-load data into a serverless, high-throughput storage architecture.

BigQuery Data Transfer Service

Automates data movement into BigQuery on a scheduled, managed basis. Lay the foundation for a BigQuery data warehouse without writing code.

Cloud Data Fusion

Quickly build and manage data pipelines using fully managed, code-free data integration with a graphical interface.

Dataflow

Develop real-time batch and stream data processing pipelines.

Dataproc

Perform batch processing, querying, and streaming using a managed Apache Spark and Hadoop service.

Dataproc Metastore

Use a fully managed Apache Hive metastore (HMS) that runs on Google Cloud to manage your data lake and metadata.

Dataproc Serverless

Use Dataproc Serverless to run Spark batch workloads without provisioning and managing your own cluster.

Datastream

A serverless and easy-to-use change data capture (CDC) and replication service.

Google Cloud Managed Service for Apache Kafka

A managed cloud service that lets you ingest Kafka streams directly into Google Cloud.

Pub/Sub

Ingest event streams from anywhere, at any scale.

Storage Transfer Service

Transfer data between Cloud Storage services such as AWS S3 and Cloud Storage.

Transfer Appliance

Ship large volumes of data to Google Cloud using rackable storage.

Data orchestration

Organize and optimize your workload management chain with seamless connections across data sources and processes.

Cloud Composer

Create, schedule, monitor, and manage workflows using a fully managed orchestration service built on Apache Airflow.

Dataform

Dataform offers an end-to-end experience that helps data teams build, version control, and orchestrate SQL workflows in BigQuery.

Data processing

Transform your data with powerful ETL, SQL workflow, and DML tools.