How SIGNAL IDUNA supercharges customer service with gen AI

Today’s insurance customers expect more: simple digital services, instant access to service representatives when they want to discuss personal matters, and quick feedback on submitted invoices. Meeting these demands has become increasingly difficult for insurers due to rising inquiry volumes, a shortage of skilled workers, and the loss of expertise as employees retire. 

Recognizing the growing need for immediate and accurate responses, SIGNAL IDUNA, a leading German full-service insurer, particularly prominent in health insurance, introduced a cutting-edge AI knowledge assistant, powered by Google Cloud generative AI.

“We’ve pioneered to unlock the power of human-AI collaboration: To redefine process efficiency by bringing together technology and subject matter experts to deliver exceptional customer experiences,” said Johannes Rath, board member for Customer, Service, and Transformation at SIGNAL IDUNA.

SIGNAL IDUNA, in collaboration with Google Cloud, BCG and Deloitte, has developed an AI knowledge assistant that empowers service agents to quickly and accurately resolve complex customer inquiries. This innovative solution uses Google Cloud AI, including Google’s multimodal Gemini models, to help agents find relevant documents and provide comprehensive answers 30% faster — ultimately, enhancing customer satisfaction.

The Challenge: Meeting modern expectations

Like many organizations in the insurance sector, SIGNAL IDUNA faced significant operational burdens. The complexity of insurance products, along with the growing demand for immediate and accurate responses, often leads to bottlenecks that can impact service experiences.

For example, prior to introducing its AI knowledge assistant, service agents had to manually search thousands of internal documents for hundreds of different tariffs to find the information needed to answer questions or resolve customer issues — including, insurance conditions, tariff information, guidelines, and standard operating procedures. As a result, 27% of inquiries required further escalation to other departments or specialists, resulting in delayed resolutions, increased costs, and potential damage to reputation.

Though complex, SIGNAL IDUNA prioritized this process as one of its top gen AI use cases, developing an AI assistant to help agents provide quick and accurate answers to customer inquiries, particularly those about health insurance. The AI knowledge assistant is grounded in more than 2,000 internal documents for more than 600 different tariffs, allowing agents to ask questions in natural language and receive accurate answers, significantly reducing the time spent searching for relevant information.

A deep dive into SIGNAL IDUNA's gen AI system

Working with Google Cloud, BCG, and Deloitte, SIGNAL IDUNA built a sophisticated generative AI architecture using Google Cloud’s AI platform, Vertex AI, and utilized Gemini 1.5 Pro’s long-context capabilities to develop an AI knowledge assistant that can provide quick and accurate access to the right information within a vast collection of documents. The system employs multiple steps to aggregate and process extensive information from diverse sources, ensuring agents can access the complete context necessary to effectively address customer inquiries.

Here’s a breakdown of the key steps:

1. Data pre-processing and extraction
The knowledge base is built from various document types, which are typically in PDF format, including policy documents, operating procedures, and general terms and conditions.

SIGNAL IDUNA utilizes a hybrid approach that combines Layout Parser in Google Cloud Document AI and PDFPlumber to parse these PDFs and extract the text content. While the Layout Parser is responsible for extracting the text segments, SIGNAL IDUNA enhances the extraction of tables with PDFPlumber if the quality of the PDFs allows. The extracted texts are then cleaned, chunked by Google's Gecko multilingual embedding model, and enhanced with additional metadata, enabling the ability to process and analyze the information later effectively.

For storing the vectorized texts, Google Cloud SQL for PostgreSQL is used with the pgvector PostgreSQL extension, which provides a highly effective vector database solution for our needs. By storing vectorized text chunks in Cloud SQL, SIGNAL IDUNA benefits from its scalability, reliability, and seamless integration with other Google Cloud services, while pgvector empowers efficient similarity search functionality.

2. Query augmentation
Query augmentation generates multiple queries to improve the formulation of user questions for both document retrieval from the vector store and answer generation. The original question is reformulated into several variants, creating three versions in total: the original query, a rewritten query, and an imitation query. These are used then to retrieve relevant documents and generate the final answer.

For the rewritten query, the system uses Gemini Pro 1.5 to correct spelling errors in the original question. Additionally, the query is expanded by adding synonyms for predefined terms and tagging specific terms (e.g., "remedies," "assistive devices," "wahlleistung/selective benefits") with categories. The system also uses information about selected tariffs to enrich the query. For example, tariff attributes, such as brand or contract type, are extracted from a database and appended to the query in a structured format. These specific adjustments make it possible to handle special tariff codes and add further context based on tariff prefixes.

The imitation query uses Gemini Pro 1.5 to rephrase the question to mimic the language of technical insurance documents, improving the semantic similarity with the source material. It considers conversation history and handles age formatting.

3. Retrieval
First, the system checks the query cache, which stores previously answered questions and their corresponding correct answers. If the question, or one very similar to it, has already been successfully resolved, the cached answer is retrieved, helping to provide a rapid answer. This efficient approach ensures quick access to information and avoids redundant processing. 

The accuracy of the cache is maintained through a user feedback loop, which identifies correctly answered questions to be stored in the cache through upvotes. A downvote on a cached answer triggers an immediate cache invalidation, ensuring only relevant and helpful responses are served. This dynamic approach improves the efficiency and accuracy of the system over time. If no matching questions are found in the query cache, the retrieval process falls back on the vector store, ensuring that the system can answer novel questions.

After retrieving any relevant information chunks from the query cache or vector store, the system uses the Vertex AI ranking API to rerank them. This crucial process analyzes various signals to refine the results, prioritizing relevance and ensuring the most accurate and helpful information is presented.

Ensuring complete and accurate answers is paramount during retrieval, and SIGNAL IDUNA found that some queries required information beyond what was available in the source documents. To address this issue, the system uses keyword-based augmentations to supplement the final prompt, providing a more comprehensive context for generating responses.

4. Generation
The answer generation process involves three key components: the user's question with multiple queries, retrieved chunks of relevant information, and augmentations that add further context. These elements are combined to create the final response using a complex prompt template.

Delivering a near real-time experience is crucial for service agents, so SIGNAL IDUNA also streams the generated response. During development, minimizing latency based on the input posed a significant technical hurdle. To address this issue, SIGNAL IDUNA reduced processing times using asynchronous APIs to help stream data and handle multiple requests. Currently, the system has achieved an average response time of approximately 6 seconds, and SIGNAL IDUNA is experimenting with newer faster models to reduce this time even further.

5. Evaluation
Rigorous evaluation is essential for optimizing Retrieval Augmented Generation (RAG) systems. SIGNAL IDUNA uses the Gen AI evaluation service in Vertex AI to automate the assessment of both response quality and the performance of all process components, such as retrieval. A comprehensive question set, created with input from SIGNAL IDUNA’s service agents, forms the basis of these automated tests. 

The evaluation results flow seamlessly into Vertex AI Experiments and Google Cloud BigQuery. This enables SIGNAL IDUNA to visualize performance trends and gain actionable insights using dashboards with Looker on Google Cloud.

Here’s a closer look at how Looker helps evaluate the AI knowledge assistant:

• Chunk retrieval: First, SIGNAL IDUNA evaluates retrieval of relevant information chunks. Metrics at this stage help assess how effectively the model identifies and gathers the necessary information from the source data. This includes tracking gen AI metrics, such as recall, precision, and F1-score, to pinpoint areas for improvement in the retrieval process. This is crucial as retrieving the correct information is the foundation of a good generated response.

• Document reranking: Once the relevant chunks are retrieved, they’re reranked to prioritize the most pertinent information. The Looker dashboard allows monitoring the effectiveness of this reranking process.

• Generated vs. expected response comparison: The final stage involves comparing the generated response with the expected response. SIGNAL IDUNA evaluates the quality, accuracy, and completeness of the generated output, utilizing large language models (LLMs) to score the similarity between the generated response and the expected response.

• Explanation generation: To understand the reasoning behind an LLM’s evaluation, SIGNAL IDUNA generates explanations for its judgments. This provides valuable insights into the strengths and weaknesses of the generated responses, helping the developers identify specific areas for improvement.

This multi-stage evaluation approach provides SIGNAL IDUNA a holistic view of the model’s performance, enabling data-driven optimization at each stage. The Looker dashboard plays a vital role in visualizing these metrics, making it easier for the developers to identify areas where the model excels and where it needs improvement.

Real-world impact: AI-powered efficiency and productivity

To determine whether the AI assistant provided measurable added value for its workforce, SIGNAL IDUNA conducted an experiment with a total of 20 employees (internal and with external providers). During the experiment, customer requests were processed with and without the AI knowledge assistant to assess its impact. 

One of the key benefits observed was a reduction in processing time. Searching across numerous data sources used to be a time-consuming process. The experiment showed that using the AI knowledge assistant reduced the core processing time (information search and response formulation) by approximately 30% and increased the quality of the response based on expert evaluations. The time saved was particularly notable for employees with less than two years of experience in health insurance.

In addition, the AI knowledge assistant significantly increased the case closure rate. Health insurance is a very complex field, and the use of external service providers means that not every employee can always answer every customer question. With support from the AI knowledge assistant, SIGNAL IDUNA’s case closure rate increased by approximately 24 percentage points, rising from 73% to almost 98%.

Scaling for the Future

“Together with Google, we at SIGNAL IDUNA have successfully applied gen AI to one of our core business processes” Stefan Lemke, CIO at SIGNAL IDUNA, said. “Now, it's time to scale this powerful technology across our entire organization. We're not just scaling a tool, we're scaling innovation, learning, and the possibilities of what we can achieve.”

Gen AI offers enormous potential for optimizing processes and developing innovative solutions. With its innovative approach — business teams experimenting with the technology in a decentralized manner and developing customized applications — SIGNAL IDUNA is primed to pioneer the next generation of insurance solutions and services. 

At the same time, SIGNAL IDUNA is establishing central standards to scale insights gained across the company and tap into the combined power of its teams, resources, and lines of business. This strategic decision has helped create valuable resources like code libraries, infrastructure blueprints, and centrally offered services. 

By combining agility with established standards and best practices, SIGNAL IDUNA can now react quickly to new requirements, setting a new standard for efficiency and customer satisfaction.

^{This project was delivered by the following core team members, Max Tschochohei, Anant Nawalgaria, and Corinna Ludwig by Google, and Christopher Masch, Michelle Mäding from SIGNAL IDUNA}