Introducing AI Hub and Kubeflow Pipelines: Making AI simpler, faster, and more useful for businesses

Whether they’re revolutionizing the clothing manufacturing supply chain or accelerating e-commerce, businesses from every industry are increasingly turning to AI to advance what’s possible. Yet for many businesses, the complexities of fully embracing AI can seem daunting.

Our goal is to put AI in reach of all businesses. But doing that means lowering the barriers to entry. That’s why we build all our AI offerings with three ideas in mind: make them simple, so more enterprises can adopt them, make them useful to the widest range of organizations, and make them fast, so businesses can iterate and succeed more quickly.

Earlier this year, we announced AutoML to help businesses with limited ML knowledge and expertise build their own custom ML models. We’ve invested in specialized training and certifications to help grow the ML skill set more broadly. And we provide enterprises resources like the Advanced Solutions Lab that offer on-site collaboration with Google’s own ML engineers. All these things have helped grow AI adoption across enterprises. To date, we have more than 15,000 paying customers across many different industries using our AI services.

Another way we’re aiming to make AI faster, simpler and more useful is by helping data scientists be more effective. Although there are approximately 20 million developers worldwide, there are only 2 million data scientists. They need tools that can help them scale their efforts, and organizations need more ways to take advantage of their work and make it accessible to their developers and engineers. Today we’re announcing several new products to our AI portfolio that do exactly that.

Making AI simpler with the AI Hub

Putting AI in reach of more businesses means making it easier for them to discover, share, and reuse existing tools and work. But until recently, the scarcity of ML knowledge in the workforce made it challenging to build a comprehensive resource. Today we’re launching the AI Hub to address this need.

The AI Hub is a one-stop destination for plug-and-play ML content, including pipelines, Jupyter notebooks, TensorFlow modules, and more. It offers two significant benefits. The first is making high quality ML resources developed by Google Cloud AI, Google Research and other teams across Google publicly available to all businesses. The second is that it provides a private, secure hub where enterprises can upload and share ML resources within their own organizations. This makes it easy for businesses to reuse pipelines and deploy them to production in GCP—or on hybrid infrastructures using the Kubeflow Pipeline system—in just a few steps.

In alpha, the AI Hub will provide these Google-developed resources and private sharing controls, and its beta release will expand to include more asset types and a broader array of public content, including partner solutions.

Making AI more useful with Kubeflow Pipelines and API updates for video

It’s not enough to provide a place where organizations can discover, share and reuse ML resources, they also need a way to build and package them so that they’re as useful as possible to the broadest range of internal users. That’s why we’re introducing Kubeflow Pipelines.

Kubeflow Pipelines are a new component of Kubeflow, a popular open source project started by Google, that packages ML code just like building an app so that it’s reusable to other users across an organization. Kubeflow Pipelines provides a workbench to compose, deploy and manage reusable end-to-end machine learning workflows, making it a no lock-in hybrid solution from prototyping to production. It also enables rapid and reliable experimentation, so users can try many ML techniques to identify what works best for their application.

Fairness is one of our guiding AI principles and something we discuss with our cloud customers adopting ML in their own businesses. Kubeflow Pipelines can help them take advantage of Google’s TensorFlow Extended (TFX) open source libraries that address production ML issues such as model analysis, data validation, training-serving skew, data drift, and more. This improves the accuracy, relevance, and fairness of results for businesses. You can get started with Kubeflow Pipelines on GitHub.

We also continue to expand the capabilities of our AI building blocks to make them even more useful for enterprises, including the beta release of three features in our Cloud Video API that address common challenges for businesses that work extensively with video. Text Detection can now determine where and when text appears in a video, making that video more readily searchable, and it supports more than 50 languages. Object Tracking can both identify more than 500 classes of objects in a video. Speech Transcription for Video can transcribe audio, making it possible to easily create captioning and subtitles, as well as increasing the searchability of its contents. You can learn more about our AI building blocks on our website.

Making AI faster with Cloud TPU updates

We’re continually lowering the compute barriers to AI with our Tensor Processing Units (TPUs). These custom ASIC chips designed by Google for machine learning workloads dramatically accelerate ML tasks, and are easily accessed through the cloud.

In July we announced that our second-generation TPUs are generally available and within reach of every Cloud user, including free-tier users. In October, we announced the beta release of our third generation, liquid-cooled Cloud TPUs, and we made PyTorch available across Google Cloud, and will soon be available for use on TPUs. Today we also announced pricing for our V2 TPU Pods. All these updates aim to make compute-intensive machine learning faster and more accessible to businesses worldwide. You can learn more about TPUs on our website.

Looking forward

Over the past several months we’ve heard from many of our customers successfully using AI to solve their unique business challenges.

Meredith Corporation, a media company, uses machine learning to automate content classification, applying a custom universal taxonomy with Cloud AutoML and Natural Language. Machine learning helps them make content classification more repeatable and scalable, saving time and improving reader experiences.

“At Meredith Corporation, we’re focused on creating compelling content across platforms and life stages for brands such as PEOPLE, Better Homes & Gardens, Martha Stewart Living, Allrecipes, and Food & Wine,” says Alysia Borsa, Chief Marketing & Data Officer, Meredith Corporation. “By using Natural Language and AutoML services to apply our custom universal taxonomy to our content, we’re able to better identify and respond to emerging trends, enable robust detailed targeting and provide our audience with more relevant and engaging experiences.”

Emory University is combining clinical data, machine learning, and the scalable infrastructure of GCP to develop a sepsis prediction engine that uses real-time analytics in an effort to provide better care for at-risk patients while also controlling medical costs.

“With sepsis, early detection is key,” says Dr. Ashish Sharma, Assistant Professor, Department Department of Biomedical Informatics, Emory University’s School of Medicine. "By converting our TensorFlow-based sepsis prediction algorithm into an App and running it on the Google App Engine, we're able to provide information in the actionable window when physicians can make meaningful interventions for a patient. What matters most is improving medical outcomes for real patients in ICUs and machine learning is crucial to helping optimize patient care.”

Geotab uses BigQuery ML and BigQuery GIS to predict potential hazardous driving areas in Chicago and promote data driven decision making and enable smart city initiatives.

“Geotab provides data-driven insights on commercial fleet vehicles across every continent,” says Mike Branch, Vice President Data & Analytics, Geotab. “By leveraging machine learning and BigQuery, among other smart city insights, we have been able to develop a solution for our customers that predicts particularly hazardous driving areas in a city based on weather and traffic flow. We’re incredibly excited to be collaborating with Google Cloud’s machine learning technology to help create better solutions for our customers and the community.”

We’re also thrilled to see the continued growth of the Kubeflow community. Organizations like NVIDIA, Cisco, and Intel are among the contributors to this open source project and are collaborating with us closely to adopt Kubeflow pipelines. NVIDIA is already underway integrating RAPIDS, a new suite of open source data science libraries, into Kubeflow. The RAPIDS library leverages GPUs to provide an order of magnitude speed-up for data pre-processing and machine learning, thus perfectly complementing Kubeflow.

“Machine learning is fast emerging as an indispensable part of the digital transformation our customers are undertaking. Further, ML is increasingly gaining traction with enterprise IT and mainstream engineering teams as they seek to deploy architectures that serve data scientists in their lines of business. Realizing the potential of ML in enterprise environments requires dramatic simplification of the lifecycle of the entire solution,” said Kaustubh Das, vice president, data center product management at Cisco. “Cisco’s significant contributions to Kubeflow aims to simplify hybrid/multi cloud AI/ML life cycle management. Cisco is also delighted to see the emergence of Kubeflow Pipeline that promises a radical simplification of ML workflows which are critical for mainstream adoption. We look forward to bringing the benefits of this technology alongside our world class AI/ML product portfolio to our customers.”

"Intel and Google have had a long engineering collaboration on multiple cloud workloads and deep learning and artificial intelligence frameworks on Intel architecture. The latest example of our long history of technical collaboration is our work on Kubeflow Pipelines," says Raejeanne Skillern, Vice President of the Data Center Group, General Manager of the Cloud Service Provider Business at Intel. "Specifically, we are adding a few components in Kubeflow Pipelines automating model optimization, running the predictions and deployment of model serving servers with OpenVINO toolkit to bring state of the art inference optimization to your AI application on Intel hardware. We are looking forward to our continued partnership with Google on this project to enable enhanced performance, efficiency and ease of deployment at scale for end customers."

We’ll continue to work closely with our customers and partners to understand their business challenges and how AI can be applied to solve them. To learn more about AI and machine learning on Google Cloud, visit our website.