Confluent brings real-time capabilities to Google Cloud generative AI

In 2023, the spotlight was on generative AI (gen AI) and how it is paving the way for a new category of AI that can create and co-innovate with humans to produce new content, such as text, code, images, and music. Gen AI capabilities are not only promising but extremely powerful, given that large language models (LLMs) can be trained and tuned on vast amounts of data.

Still, the freshness of the data can limit a gen AI model’s out-of-box capabilities and potential. Gen AI models sometimes need to be extended to other systems, such as to access new information when the use case requires real-time context. As a result, many organizations find themselves looking to solve issues related to real-time data access, integrating data from multiple sources in different formats, and the complexity associated with training and leveraging models.

Data streaming platforms like Confluent, powered by Apache Kafka, can help overcome these data challenges by providing the latest data and information. With Confluent, businesses can easily connect, process, integrate, and scale the data needed to support their gen AI use cases, enabling them to solve highly-specific, contextual issues in real time.

For instance, a customer service team could use Confluent to stream real-time customer requests and responses to a chatbot built with Google Cloud technologies. Leveraging AI, the chatbot can access the information used in responses to create more personalized and relevant recommendations while considering real-time context, such as weather conditions, demographics, and purchase history.

Improving personalization is just one example of ways businesses can benefit from using real-time data and gen AI to deliver better customer experiences. Overall, it can also help boost sales with better recommendations, reduce churn with more satisfying customer experiences, and even reduce costs by helping to automate more support tasks with the help of AI chatbots.

Creating a central nervous system for data movement

Through its many years of research and product development, Google has become a recognized leader in the AI space. Already, Google’s LLMs are providing a strong foundation for a rich set of gen AI capabilities that customers and partners can leverage to build new innovations.

Now, Confluent is making these models even easier to use by helping integrate structured and unstructured data from various sources directly into Google Cloud. You can use Connectors/Clients to stream reliable, real-time data in Confluent to Google Cloud AI products and services at scale.

The diagram above illustrates common architecture patterns for using Confluent to stream real-time data that can support gen AI workflows.

• Knowledge workflows: Confluent gathers data from various sources across internal and external data systems, pre-processes it into a specific format, and stores it to an appropriate location, which can be used as a knowledge base to build context for gen AI.
• Inference workflows: Confluent streams data to the systems and tools that help create gen AI-powered interactions between human users and machines, such as text, voice, conversation, and more.
• Central nervous system: Confluent orchestrates the data exchange between processes and services seamlessly, abstracting events as data streams, processing them, and connecting them directly to models and neural networks. By using stream processing the results can provide communication with a human through various machine interfaces.

To demonstrate how this comes to life, Confluent built a gen AI-powered, personalized shopping assistant that leverages Confluent and Google Cloud generative AI. The application flow allows a customer to have a conversation with an AI chatbot, which connects to Vertex AI and interacts in real time. Here is an example of the dialog:

Behind the scenes, Confluent takes the request, sends it to Vertex AI, and then provides a response. With Confluent, Apache Kafka provides the framework for the business to quickly process the data and provide a generative AI response. This delivers an enriched customer experience and allows the customer to receive precise details on product availability and purchasing locations.

Confluent improves gen AI chatbots in a number of ways, including:

• Creating a unified view for gen AI models. Confluent can combine data from multiple sources, including cross-cloud and on-premises, to create a unified view for gen AI models. This view can be used for a variety of applications, such as generating targeted content based on the user, no matter where the data lives..
• Reducing the cost of training and deploying gen AI models. Confluent helps reduce the amount of data that needs to be processed by gen AI models, leading to lower training and deployment costs.
• Improving the accuracy and performance of gen AI models: By providing gen AI models with real-time access and the ability to process data from various sources, Confluent improves the accuracy and performance of gen AI models.
• Making gen AI models more accessible to everyone. Confluent is easy to use and manage, making it easier for developers and organizations to build and deploy gen AI applications that require real-time information.

In addition to these benefits, there are a number of ways that Confluent enables the real-time data movement to Google Cloud AI services that empowers organizations to build more sophisticated gen AI experiences with text, voice, images, and video. For example, streaming your most recent, up-to-date customer data to Vertex AI Search and Conversation can enable teams to harness your data to provide more personalized, relevant chatbot responses and improve personalized recommendations based on customer preferences.

Confluent and Google Cloud: Better gen AI, together

Overall, Confluent allows organizations to get more out of Google Cloud generative AI, helping them build, deploy, and scale gen AI applications faster without worrying about whether they have the data they need.

Confluent delivers real-time data streaming, data integration, scalability, and reliability in one industry-leading platform, allowing organizations from every industry to tap into gen AI to provide better experiences or solve customer problems. Confluent also recently launched Data Streaming for AI, an initiative leveraging Google gen AI partnerships to accelerate organizations’ development of real-time AI applications. And that’s just the beginning — Confluent continues to work on delivering top data streaming innovation to help companies meet real-time AI demands with trustworthy, relevant data served up in the moment.

Learn more about Google Cloud’s open and innovative generative AI partner ecosystem. To get started with Confluent, join the Data Streaming Startup Challenge and begin experimenting with Confluent Cloud on the Google Cloud Marketplace today!