At the heart of Vet-AI’s solution lies an AI-powered triage system that transforms veterinary data — including over 400,000 video consultations and 100,000 labeled images — into real-time insights for pet owners. With Cloud Run, BigQuery, and Pub/Sub, Vet-AI built a fast, scalable pipeline to process and structure this vast dataset. By integrating Google’s text embedding models with Gemini’s generative capabilities, the system instantly retrieves relevant clinical insights and past interactions. This means users can describe their pets’ symptoms in the Joii app and immediately receive accurate information, helping to guide their pet to the appropriate level of care. 

With a team of clinicians in the loop, pet owners can depend on the clinical accuracy of the system, while built-in classification models mean they’ll also be alerted if their pet needs immediate in-clinic care. This intelligent system improves the quality of information vets receive, reducing wait times, and cutting in-app consultation times by 8%. It also provides an affordable first step in pet care to ensure more animals receive timely support, leading to a 94% customer satisfaction score.

For Vet-AI, accuracy is paramount to ensure its triage system is safe for pets. The company's internal blind tests showed Gemini to be the best-performing model for generating responses that adhered to its store of specific, clinically-relevant data. Those tests showed that Vet-AI's LLM-based triage system, underpinned by Gemini, operates at 81% clinical accuracy, significantly surpassing other leading LLMs.

Just as important is the need to maintain pet owners’ data privacy. Vet-AI uses Gemini to remove all personally identifiable information during the data processing stage, preventing any personal data from being ingested into its retrieval system. Vet-AI hosts its models using Vertex AI endpoints, which, as with all Google Paid Services, never uses customer data to train the underlying Gemini models, ensuring Vet-AI keeps pet owners' information secure and confidential.

The success of Vet-AI’s triage system is underlined by an 8% increase in users converting into a consultation when triaging through the LLM. Vet-AI is now developing a range of AI specialists built on Google Cloud and trained extensively on specific clinical datasets. Starting with a gastro specialist, these will offer specialized, nuanced advice, helping Vet-AI to further differentiate its offering. 

Vet-AI has also used Google Cloud to build a cutting-edge visual-analysis capability for skin conditions. This will allow pet owners to upload photos of skin lesions for assessment. A custom skin model, trained using TensorFlow on over 60,000 images, analyzes these photos to identify the type and severity of the condition. Vet-AI uses Vertex AI endpoints to host its models, creating an endpoint where the app can send photos and instantly receive an assessment. “With Vertex AI, we can deploy the model and get an endpoint really quickly, which is very refreshing,” says Gray. This foundational visual technology also extends to video analysis, meaning pet owners are now able to send videos of their pets to the Mobility Checker tool for instant insights such as gait analysis.