What’s new with Google Cloud databases — June 2024 edition

Databases are a fundamental part of every company's innovation plan. However, many customers express that databases are the most challenging aspect of the technology stack to design, code, operate, and manage. At Google Cloud, we assist our customers with migrating, modernizing, and transforming their databases to build the next generation of AI-driven solutions.

In this blog, you’ll find a comprehensive list of major database services updates to Google Cloud's database services since our last installment in January. For more on what’s arguably our biggest database-related news of the year — our partnership with Oracle that enables customers to migrate and run mission-critical enterprise workloads seamlessly across Google Cloud and Oracle Cloud Infrastructure (OCI) — read the blog post. And stay tuned!

Here are some the recent key product updates across our databases portfolio:

Gen AI with databases

Vector search has emerged as a critical capability for building useful and accurate gen AI-powered applications. Using a nearest neighbor algorithm, vector search makes it easier for similarity search results with unstructured data such as text and images from a product catalog.

• Vector support is now available across the databases portfolio including AlloyDB, Spanner, Cloud SQL, Memorystore for Redis and Firestore, helping you build enterprise gen AI apps backed by the data in your favorite database.

• Integrations with LangChain: Native integration between Google databases and LangChain support complex gen AI apps with frameworks and orchestration in a managed environment.

Gemini in Databases

Gemini in Databases is an always-on AI assistant designed to simplify and supercharge your experience and productivity across the entire database journey. 

• Build applications faster: Use natural language to develop applications quickly.

• Manage from a single pane of glass: Oversee, optimize, and govern your entire database fleet in one place.

• Accelerate migrations: Streamline the process of moving your databases.

• Get started today: Gemini in Databases is now available in preview.

AlloyDB

AlloyDB is a highly performant postgreSQL compatible database ready for top-tier relational database workloads and able to run anywhere including your laptop and data center.

• AlloyDB AI is now available, delivering up to 10x faster vector queries compared to standard PostgreSQL for IVFFlat indexing.

• You can now use federated queries from BigQuery to AlloyDB. This feature is now available in preview.

• Private Service Connect is now generally available (GA). Private Service Connect lets you connect to an AlloyDB for PostgreSQL instance from multiple VPC networks that belong to different groups, teams, projects, or organizations.

• You can now set maintenance windows (GA) for your AlloyDB clusters. When enabled, AlloyDB schedules non-emergency maintenance events to begin only during a specified weekly period. You can also opt in to receive email notifications of upcoming maintenance events.

• AlloyDB now incorporates Google’s ScaNN (Scalable Nearest Neighbor) algorithm for efficient vector search and retrieval. This integration brings significant advantages to AlloyDB: faster vector queries, higher write throughput, smaller memory footprint, and faster index build times (blog).

Cloud SQL 

Cloud SQL is a fully managed database service for MySQL, PostgreSQL, and SQL Server with high availability and scalability features.

• At Google I/O, we announced a private preview of Firebase Data Connect (FDC), which is powered by Cloud SQL under the hood. With FDC, developers at both startups and large enterprises can enjoy faster app development with access to PostgreSQL. Additionally,Firebase Data Connect provides access to the fully managed capabilities of Cloud SQL, allowing developers to build innovative and scalable solutions (Video, YouTube Shorts, Blog).

• Cloud SQL Enterprise Plus edition is now available in six more regions, providing the benefits of 99.99% availability SLA inclusive of maintenance, near-zero-downtime upgrades, and up to 35-day point-in-time recovery capability.

• Cloud SQL Enterprise Plus now offers even higher availability: 

• Near-zero downtime for instance scale-up operations on primary instances is now available. With this, you can now scale up their instances and continue to serve requests uninterrupted. 

• Subsecond downtime planned maintenance: Maintenance on primary instances with HA enabled now happens with < 1 sec of downtime. 

• Advanced DR for Cloud SQL (MySQL): Along with switchover, we have introduced a feature called replica failover that automatically rebuilds instances in a primary region after an outage, reducing toil. 

Memorystore

Memorystore is a fully managed in-memory Redis and Memcached service that offers sub millisecond data access, scalability, and high availability for a wide range of applications.

• Vector Search and Redis version 7.2 for Memorystore for Redis is now in GA. Vector search provides blazing fast in-memory vector search. Each Memorystore for Redis instance can perform vector search at single-digit millisecond latency over tens of millions of vectors.

• New node types: 1.4 GB shared core, 6.5 GB, and 58 GB for Memorystore for Redis Cluster are now available in GA. 

• Persistence (AOF and RDB) and Dynamic configuration for Memorystore for Redis Cluster is now in preview. 

• New OSS LangChain integrations with Memorystore can speed up your RAG workflows, serve as an LLM cache, or in a variety of other gen AI use cases.

• Single-zone instances are now in public preview. With single zone instances, you can eliminate cross-zone traffic to bring down networking costs and reduce latency.

Bare Metal Solution for Oracle

Bare Metal Solution for Oracle allows you to bring your Oracle workloads closer to Google Cloud and jumpstart cloud journey with minimal risk.

• Bare Metal Solution now offers Performance Storage, an option for high-performance workloads that expands storage options from 56 MB/s per TiB to 256 MB/s per TiB.

• Bare Metal Solution SKUs are now public and reflected in the Pricing Calculator.

Spanner

Spanner is a fully managed, globally distributed relational database service offering horizontal scalability, strong consistency, and high availability.

• Vector search: Spanner now supports vector search over trillions of vectors for real-time RAG applications at scale without bolt-on tools or data movement. Spanner provides cosine distance, Euclidean distance, and dot product vector functions for K-nearest neighbors (KNN) similarity search, along with efficient types for storing vector embeddings.

• Change streams: Lots of new capabilities improve Spanner’s built-in change data capture mechanism. Change streams are used to efficiently publish insert, update, and delete events to consumers, like Pub/Sub and Kafka. A new capture type allows you to include the complete new row and just the values that were changed. New filters allow you to include or exclude published changes by type: insert, update, or delete, as well as identify deletes that were generated as part of built-in TTL. Finally, change streams are also available for local development and testing in the Spanner emulator.

• Protocol Buffers: Spanner now supports protocol buffer types. Protobufs are an efficient way to serialize data for storage or transport, for example to share in an RPC or microservices architecture. 

• New Regional Endpoints (REP) are an alternative to Spanner’s global front-end that ensures connections are routed only through a configured region, supporting residency requirements for data in transit as well as at rest. Leader-aware routing is another new performance optimization for multi-region instances that reduces write latency by as much as 40% on many types of read-write transactions.

• OpenTelemetry support for Spanner metrics and traces is now GA. OpenTelemetry is the modern standard for open-source observability and was formed as a result of a merge between OpenCensus and OpenTracing. 

Bigtable

Bigtable is an enterprise-grade, low-latency NoSQL database service for machine learning, operational analytics, and user-facing applications at scale.

• Bigtable now offers Data Boost, which allows fully-isolated serverless reads on Bigtable data for batch workloads like ETL jobs, analytical queries and ML model training. 

• You can now create daily backups of your Bigtable table by enabling automated backups.

• Bigtable authorized views feature is now GA. Authorized views allow secure data collaboration through fine-grained access control at the row and column level. 

• Bigtable Spark connector lets you read and write data from and to Bigtable using Spark SQL and PySpark DataFrames. 

• Bigtable now supports distributed counters for scalable operational analytics over event streams. For details, see working with new aggregate data type.

Firestore

Firestore is a serverless document database for building rich mobile, web and IoT applications.

• By using Gemini Code Assist, along with your favorite Integrated Development Environment (IDE), you can now use natural language to define your Firestore data models and write queries.

• Firestore now has built-in support for vector search using exact nearest neighbors, the ability to automatically generate vector embeddings using popular embedding models via a turn-key extension, and integrations with popular gen AI libraries such as  LangChain and Llamaindex to support building AI-enabled solutions such as a chatbot or recommendations engine.

• With the addition of the Customer-Managed Encryption Keys (CMEK) feature, Firestore provides enterprises with greater control over how their data is secured, by giving the customer control of the encryption keys. 

• With the recent launch of queries using range filters on multiple fields in preview, customers can easily and cost-efficiently perform express queries that filter on range conditions across multiple fields directly in Firestore. 

• Firestore customers can now use the Query Explain feature to retrieve the proposed query plan; this feature also provides additional support for troubleshooting queries.

Database Migration Service (DMS)

Database Migration Service is a fully managed service that simplifies and accelerates the migration of your MySQL, PostgreSQL, and Oracle databases to Cloud SQL or AlloyDB. It minimizes downtime with continuous data replication. 

• With DMS, we have made it even easier and more seamless to migrate Oracle databases to AlloyDB or Cloud SQL. You can now migrate from Oracle to AlloyDB or Cloud SQL along with Gemini-assisted code and schema conversion.

• DMS now includes support for homogeneous SQL Server migration to Cloud SQL (SQL Server) in public preview. You can Customers will now migrate self-managed as well as RDS SQL Server instances to Cloud SQL. 

• DMS now supports faster PostgreSQL migrations to both Cloud SQL for PostgreSQL and AlloyDB for PostgreSQL.

• DMS for homogeneous migrations to Cloud SQL for MySQL and homogeneous migrations to Cloud SQL for PostgreSQL now supports migrations to existing destination instances that have read replicas enabled.

• DMS now supports Percona Xtrabackup in Public Preview to migrate MySQL databases to Cloud SQL MySQL. 

Datastream

Datastream is a serverless change data capture (CDC) and replication service for data movement on Google Cloud.

• Datastream now supports append-only mode. With append-only mode, you can capture each INSERT, UPDATE, and DELETE event in BigQuery as a distinct row, enabling use cases such as audit trails, trend analysis, and more.

• SQL Server as a source is now in public preview. This lets you replicate SQL Server data to several Google Cloud services such as BigQuery, Cloud Storage, AlloyDB, and Spanner.

Final thoughts

Dive into our comprehensive blogs to stay updated on all the latest developments, and explore our detailed product documentation for an in-depth understanding of our managed database services.