What’s new with Google Cloud Networking at Next ’24
Muninder Sambi
VP, Cloud Networking
Cross-Cloud Network has transformed how organizations connect and secure workloads across hybrid and multi-cloud networks. It simplifies complexity, strengthens security posture, and helps deliver faster business outcomes. Built on Google Cloud’s planet-scale network, Cross-Cloud Network enables you to deliver rich experiences, streamline operational efficiency, and lower TCO.
AI adoption and growth are driving a major inflection point in networking, requiring higher performance, scale, intelligence, and security. Foundation models such as large language models (LLMs) fuel a step-function improvement in performing a number of network operations such as design, secure, optimize and troubleshoot. With Cross-Cloud Network, you can train and inference AI models anywhere, unlocking new opportunities for innovation and growth.
At Google Cloud Next, we’re announcing a series of enhancements designed to help you modernize, simplify, and secure your cloud environment. These include:
-
Planet-scale networking for AI/ML workloads
-
Any cloud to any service connectivity
-
Securing the workload, data, and users
-
Gemini-powered network operations
Many of our customers are already on the journey to leverage Cross-Cloud Network to transform this business, please see this video here.
Planet-scale networking for AI/ML workloads
Cross-Cloud Network enables high speed, reliable data path for gen AI training and inference. Google Cloud offers planet-scale, high performance, low latency network infrastructure across zones and regions to enable gen AI training at scale with GPU and TPU networking. Innovations such as Cross-Cloud Interconnect are helping customers to use the best in cloud managed SaaS services from Google Cloud by enabling large volumes of data to be closer to training workloads.
“Because of its scale and shared services model, cloud technology is best-suited for the delivery of gen AI-enabled applications at scale and the development of general-purpose foundation models,”1 said Sid Nag, vice president and analyst, Gartner. Additionally, Gartner advised, “a scalable AI infrastructure requires a combination of high-speed, AI-optimized networks of multiple types.”2
Gen AI workloads have unique traffic patterns, with large requests and responses. This can lead to variable processing times, resulting in suboptimal user response times. To address this, an intelligent network can distribute foundation model queries based on usage and availability of resources. We are introducing a new class of innovations that are custom-built to address these challenges and optimize performance for AI workloads.
Generally available today, Model as a Service Endpoint is a solution that allows model creators to own the model service endpoint to which application developers then connect. The solution consists of Private Service Connect for AI model producer and consumer connectivity, Cloud Load Balancing for optimal traffic distribution, and App Hub for service discoverability.
In addition, we are introducing the following Cloud Load Balancing enhancements for inference workloads coming later this year:
-
Cloud Load Balancing with custom metrics provides queue depth as a metric for load balancing AI workloads to deliver faster user response time to prompts while optimizing TPU and GPU utilization. We provide an overview with a simple configuration in this demo video of load balancing for AI inferencing.
-
Cloud Load Balancing for streaming inference uses metrics based on number of streams, bytes-in, and bytes-out, versus requests per second and CPU utilization to optimize performance.
-
Cloud Load Balancing with traffic management for AI models monitors the health of individual model service endpoints and routes requests to healthy endpoints, initiates cross-region failover when an outage is detected, and splits traffic across different models and model versions, helping you to manage rollouts.
“AppLovin operates one of the most successful platforms for app developers to grow their business, reaching over 1.4 billion daily active users (DAUs) worldwide. We are leveraging Google Cloud to advance our next gen AI platform with state-of-the-art hardware to power our training and inference workloads. Google Cloud’s global front-end solution with Load Balancer, Cloud Armor, and CDN not only protects our users but helps businesses reach, monetize, and grow their audiences.” - Omer Hasan, VP of Operations, AppLovin
Cross-Cloud Network simplifies service-centric networking
A service-centric Cross-Cloud Network delivers a consistent, secure experience from any cloud to any service. By leveraging Private Service Connect (PSC) everywhere, DevOps, NetOps, and SecOps teams operate across a single service-centric network, executing everything from publishing SaaS/managed services to ensuring that security is consistently applied across their services. DevOps teams can use PSC to publish services; NetOps teams can simplify network connectivity; and SecOps teams can ensure encryption and consistent security is applied between workloads and services. Through service-centric Cross-Cloud Network, you may lower TCO by up to 40% as a result of simplification and operational efficiency.
“As the leader in digital payments, PayPal is revolutionizing commerce globally in over 200 markets around the world. To best serve the hundreds of millions of customers and merchants who use PayPal services, our network must provide high reliability, security, and performance. With Google Cross-Cloud Network, we can focus on innovation and reduce the management burden to scale our network globally.” - Saikrishna Kotha, Head of Global Network Services, PayPal
Figure 1: Cross-Cloud Network connects SaaS/managed services on any cloud with Private Service Connect
We are introducing Private Service Connect transitivity over Network Connectivity Center, available in preview this quarter. It enables services in a spoke VPC to be transitively accessible from other spoke VPCs. You can set up a service VPC to create multiple PSC consumer endpoints that are accessible to other VPCs. In combination with VPC spokes for NCC, this simplifies cloud network designs.
"Scotiabank’s vision is to be our clients’ most trusted financial partner, to deliver sustainable, profitable growth and maximize total shareholder return. Through our work with Google Cloud, we are leveraging service centric Cross-Cloud Networking to simplify our network with Private Service Connect, Cloud Load Balancing and Cross-Cloud Interconnect. Additionally, we are planning to implement cloud NGFW a native distributed cloud firewall providing Zero Trust protection." - Payam Kohan, Senior Principal Cloud Architect, Scotiabank
Securing the workload and data with Cross-Cloud Network
Organizations need high security efficacy, simplicity, and strong network controls to protect their cloud environments with a Zero Trust approach. With Cross-Cloud Network, you can secure the workload, data, and user powered by best-in-class technologies leveraging AI/ML.
Figure 2: Cross-Cloud Network security
Today, we are introducing new security innovations in the Cross-Cloud Network:
-
Cloud NGFW Enterprise (formerly Cloud Firewall Plus), now GA, provides leading network threat protection powered by Palo Alto Networks technology, plus network security posture controls for org-wide perimeter and Zero Trust microsegmentation.
-
Identity-based authorization with mTLS integrates the Identity-Aware Proxy with our internal application Load Balancer to support Zero Trust network access, including client-side and soon back-end mutual TLS.
-
In-line network data-loss prevention (DLP), in preview soon, integrates Symantec DLP into our Load Balancers and Secure Web Proxy using Service Extensions. This will help safeguard sensitive data-in-transit from accidental and malicious exposure.
“Zero Trust is a top priority for organizations everywhere. Symantec DLP supports Zero Trust by combining device location (network status), user risk, data discovery, and consistent, unified policies that allow dynamic data access decisions to be taken. Integrating Symantec DLP with Google Cloud Service Extension will significantly strengthen the security posture by mitigating data breach and compliance risks.” - Jason Rolleston, Chief Product Officer, Enterprise Security Group at Broadcom
Service Extensions for an open programmable Cross-Cloud Network
Service Extensions open up the web data plane (load balancers, secure proxies) in the Cross-Cloud Network to easily allow adding customizations and services into workload data paths, helping to protect, accelerate, and optimize web experiences.
Figure 3: Service Extensions enable customized service insertion into data paths
This includes our own Apigee API Protection service, which helps discover and manage shadow APIs. Using Service Extensions, Apigee was able to simplify deployment with transparent inspection, fast performance and broad coverage of backend infrastructure.
Figure 4: Case study showing effective prevention of shadow APIs using Service Extensions
Service Extensions also opens up the Cross-Cloud Network to ecosystem partners to integrate their services, and to protect, accelerate, and optimize web experiences.
On the protection side, partners like Imperva, HUMAN Security, Palo Alto Networks and Traceable are integrating their advanced web protection services.
Plus partners like Cloudinary, Nagra, Queue-it, and Datadog are helping customers deliver amazing web experiences by integrating their services via Service Extensions.
Along with Google and partner integrations, we are introducing a library of code examples to customize origin selection, adjust headers, and more.
Gemini-powered network operations
LLMs are enabling a new class of innovations that can enhance productivity and simplify complex analytical tasks. Today we announced Gemini Cloud Assist, which provides AI-based assistance and insights across network design, operations, and optimization, including network resources.
Cloud administrators can ask Gemini Cloud Assist to solve a variety of tasks and recommendations such as generate configurations, recommend capacity, correlate changes with issues, identify vulnerabilities, and optimize performance. In preview, Gemini Cloud Assist expedites network provisioning and management, enabling organizations to deliver business results faster and more reliably.
“Enterprises, worldwide, are looking for AI-powered recommendations to simplify and cost-reduce the design, operations, and optimization of network resources. Gemini Cloud Assist offers generative-AI-based actionable insights for network administrators to timely and cost-efficiently handle their day 0 to 2+ network design and lifecycle operations. This is in particular for modern AI workloads that traverse multicloud networking and security environments.” - Vijay Bhagavath, Research Vice President of Cloud and Datacenter Networks at IDC
A network ready for the AI era
With Cross-Cloud Network, we’re empowering you to simplify, modernize and secure your hybrid and multicloud network. With the advent of AI, you can leverage Cross-Cloud Network to bring your data anywhere for training and inference on Google Cloud.
"Uber operates in 71 countries with more than 100 million users each month. We deliver low latency user experience with our global distributed services and leverage Google Cross-Cloud Network as a fabric for our hybrid and multicloud environment, connecting our back end, front end, and AI applications. It provides an agile solution for us to reach users globally, accelerate our service rollout, and reduce overall costs." - Harry Liu, Director of Engineering, Uber
For more information, please see this IDC report on Accelerating the Enterprise AI Journey with Cross-Cloud Network and join us at Cloud Next where our product experts and architects will provide architecture deep dives:
1. Gartner© Press Release, Gartner Predicts 70% of Enterprises Adopting GenAI will Cite Sustainability and Digital Sovereignty as Top Criteria for Selecting Between Different Public Cloud GenAI Services by 2027, February 2024. GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved.
2. Gartner, A Packaging Approach to Simplify GenAI Cloud Portfolio, Wataru Katsurashima, Arun Chandrasekaran, Sid Nag, March 2024
