Ingest events at any scale
Data ingestion is the foundation for analytics and machine learning, whether you are building stream, batch, or unified pipelines. Cloud Pub/Sub provides a simple and reliable staging location for your event data on its journey towards processing, storage, and analysis.
With Cloud Pub/Sub, data engineers can scale without provisioning, partitioning, or load isolation worries, and expand applications and pipelines to new regions simply with global topics.
Use Cloud Dataflow with Cloud Pub/Sub to enrich, deduplicate, order, aggregate, and land events. Mix real-time and batch processing via Cloud Pub/Sub’s durable storage.
Simplify development of event-driven microservices
Whether you’re just beginning your journey to event-driven asynchronous microservices or migrating an existing system, making your events accessible through messaging middleware is a critical early step. Application developers on GCP rely on Cloud Pub/Sub to reliably deliver each event to all the services that must react to it.
Upon event publication to Cloud Pub/Sub, push subscriptions deliver the event to serverless apps running in Cloud Functions, App Engine, or Cloud Run and pull subscriptions make it available to more complex stateful services running in Google Kubernetes Engine or Cloud Dataflow. Multi-region environments operate seamlessly because of Cloud Pub/Sub’s global nature.
Be production ready from day one
Cloud Pub/Sub is designed as a premium service that lets Google Cloud users focus on application logic, regardless of location or scale. The service is minimal and easy to start with but also eliminates the operational, scaling, compliance, and security surprises that inevitably reveal themselves in software projects.
This is why Cloud Pub/Sub includes end-to-end encryption, IAM, and audit logging, as well as NoOps, fully automated scaling and provisioning with virtually unlimited throughput. It also provides extreme data durability and availability with synchronous cross-zone replication, plus native client libraries in major languages and an open-service API.
Synchronous, cross-zone message replication and per-message receipt tracking ensures at-least-once delivery at any scale.
Open APIs and client libraries in seven languages support cross-cloud and hybrid deployments.
Cloud Dataflow supports reliable, expressive, exactly-once processing of Cloud Pub/Sub streams.
Global by default
Publish from anywhere in the world and consume from anywhere, with consistent latency. No replication necessary.
No provisioning, auto-everything
Cloud Pub/Sub does not have shards or partitions. Just set your quota, publish, and consume.
Compliance and security
Cloud Pub/Sub is a HIPAA-compliant service, offering fine-grained access controls and end-to-end encryption.
Take advantage of integrations with multiple services, such as Cloud Storage and Gmail update events and Cloud Functions for serverless event-driven computing.
Seek and replay
Rewind your backlog to any point in time or a snapshot, giving the ability to reprocess the messages. Fast forward to discard outdated data.
Cloud Pub/Sub makes it simple for us to decouple our background data and event processing from the code that handles user-facing requests. This lets us analyze data in real-time with lower code complexity and without impacting the performance of our site.Colin Fuller, Software Engineer, Khan Academy