What’s New and What’s Next with Google Cloud Databases
GM & VP of Engineering, Google Cloud Databases
If there’s one thing the past 18 months have taught us, it’s that the ability to adapt to change is paramount. In order to be successful in this digital, disruptive era, development teams need to innovate and iterate on the customer experience faster than ever before. Businesses need to be able to experiment rapidly, frequently, and economically—not only with products and services, but also with business models, processes, and strategies.
At Google Cloud, we’ve been focused on building options that meet you where you are, and that create a path for innovation and growth in the future. In the newly published Gartner® Solutions Scorecard for Google Cloud Platform Operational Databases, July 2021, Google Cloud met 100% of the required criteria outlined by Gartner and received an overall score of 90 out of 100.
Just a few months ago, we announced Spanner granular instance sizing, where customers can start using Spanner at 1/10th the cost of regular instances, equating to approximately $65 per month. This week, we announced in preview a PostgreSQL interface for Spanner. With this new PostgreSQL interface, enterprises can now use skills and tools from the popular PostgreSQL ecosystem to take advantage of Spanner’s unmatched global scale, 99.999% availability and strong consistency.
This interface supports Spanner’s rich feature set using the most popular PostgreSQL data types and SQL features, reducing the barrier of entry for building transformational applications with Spanner. Developer teams can use the tools and skills they are familiar with and have the assurance that the schemas and queries they build against the PostgreSQL interface can be easily ported to another PostgreSQL environment, if needed, giving them flexibility and peace of mind. (Complete this form to request access to the preview.)
At Wayfair, we are focused on delivering the best online shopping experience for all things home to our customers”, said Phil Portnoy, Associate Director at Wayfair. “To make this possible, we moved to managed database services in the cloud and are using PostgreSQL because it's what our developers know and love. We are excited about Cloud Spanner’s new PostgreSQL interface because it makes Spanner more accessible to our application teams that need its unmatched scalability, strong consistency and 99.999% availability without the huge extra investment in specialized skills.
Another big announcement we are making during NEXT is the unveiling of Google Distributed Cloud, which is a portfolio of fully managed hardware and software solutions that extend Google Cloud’s infrastructure and services to the edge and into a customer’s own data center. With Google Distributed Cloud, customers will be able to deploy managed databases, machine learning, data analytics, container management services from Google Cloud, as well as well known third-party services. Google Distributed Cloud will help customers reduce operational overhead, modernize applications and process data locally to harness real-time insights across deployments.
You now have a sense of the biggest announcements coming out of NEXT. We have also been focused on additional innovations, including; simplifying migrations to the cloud, removing barriers for our transformative cloud-native databases and meeting developers with the tools they love.
Simplifying migrations to the cloud with enterprise capabilities
Over the past year, we’ve been focused on simplifying migrations to managed services on Google Cloud. We’ve built enterprise capabilities and removed major blockers for Cloud SQL, our fully managed relational database service for MySQL, PostgreSQL and SQL Server workloads. Cloud SQL offers easy integration with existing apps and Google Cloud services like Google Kubernetes Engine and BigQuery. More than 650 thousand GKE pods securely connect to Cloud SQL and BigQuery users query more than 125 Petabytes of data in Cloud SQL on average in a month using our federated querying capabilities.
In addition, we’ve introduced Cloud SQL capabilities such as cross-region replicas, point-in-time recovery (PITR), customer-managed encryption keys (CMEK), VPC Service Controls, Cloud IAM and Active Directory support. Cloud SQL maintenance downtime is now on average 80% shorter than it was 12 months ago, and is shorter compared to close competitors. We've also been staying current with support for the latest release versions including PostgreSQL 13, and PostgreSQL 14 following shortly. In addition, we recently introduced Cloud SQL cost recommenders with Active Assist that empowers developers to better manage costs across their Cloud SQL databases.
We’ve made migrations to Cloud SQL easier and faster with the Database Migration Service. More than 85% of all migrations are underway in under an hour, with the majority of customers migrating their databases from other clouds. Database Migration Service currently supports MySQL and PostgreSQL databases, with support for SQL Server migrations coming soon.
Finally, we announced Datastream, a serverless change data capture (CDC) and replication service, which provides access to streaming, low-latency data from MySQL and Oracle databases, with destinations in Google Cloud such as Spanner and BigQuery. You can learn more about migrating to managed services on Google Cloud in this white paper.
Transformative databases for always-on applications
With global reach and unlimited scale, we offer best-in-class databases for always-on applications. Cloud Spanner, the globally distributed, highly-available relational database now processes over 1 billion requests per second at peak and has been battle-tested with hundreds of applications and petabytes of data around the world. Cloud Bigtable, the highly performant NoSQL database has more than 10 Exabytes of data under management and this number continues to grow. And Firestore, our scalable document database for mobile, web, and server development, has built an active and thriving developer community and supports more than 750M monthly active end-users using Firebase Auth.
We’ve been making investments in key features and enhancements including CMEK for Spanner and Bigtable, Data Access Audit Logging in Bigtable and Firestore, and 99.999% availability SLA for Bigtable, which joins Spanner and Firestore in meeting this level of service. All these features help customers meet stringent regulatory requirements, especially financial services organizations.
And with capabilities like the soon to be released Bigtable Auto Scaling, you will be able to automatically add or remove capacity in response to changing demands of their workloads, so you only pay for the capacity you need. This will reduce management overhead, as teams can spend more time on strategic work and less time managing infrastructure. Bigtable nodes now support twice as much storage capability, from 2.5T per node to a max of 5T per node for SSD and from 8T per node to a max of 16T per node for HDD. This is especially cost-effective for batch workloads that operate on large amounts of data.
Meet developers with the tools they love
Developers play a key role in building applications, so we've also been focused on improving their speed of innovation, and offering enhanced features and capabilities across the portfolio. Developers love Firestore because of its ability to serve both as a document database and as a backend-as-a-service. It’s therefore no surprise that Firestore has built a thriving developer community with more than 250K monthly active developers for rapid app development. We announced Key Visualizer for Bigtable, Spanner and Firestore allowing developers to quickly and visually identify performance issues. This feature generates visual reports that break down usage based on the row keys that are accessed in tables. We also launched Cloud SQL insights earlier this year, which brings industry-leading database observability to developers at no extra charge, helping them detect, diagnose, and prevent query performance problems for Cloud SQL databases.
As mentioned earlier, we announced Spanner granular instance sizing, and this week, we announced in preview a PostgreSQL interface for Spanner. Developer teams can now use the tools and skills they already have and can be assured that the schemas and queries they build against the PostgreSQL interface can be easily ported to another PostgreSQL environment, if needed, giving them flexibility and peace of mind.
And for modernizing Oracle database operations, we announced El Carro, an open source tool that implements the Kubernetes operator pattern to deliver automation for provisioning and ongoing operations like backups, patching, and high availability for databases running in hybrid and multicloud environments. And it does so using the same declarative syntax that DevOps teams are using to manage applications.
Finally, we announced Sqlcommenter, an open source object-relational mapping (ORM) auto-instrumentation library, earlier this year and donated it to OpenTelemetry working closely with APM vendors such as Datadog, Dynatrace and Splunk. This announcement extends the vision of OpenTelemetry to databases and enables application-focused database observability with open standards.
The pace of change has only accelerated, and databases play a critical role in empowering teams to build applications faster. Google Cloud databases provide a ground-breaking platform for innovation based on decades of our own first-hand experiences shaping the digital world.