The generative AI evaluation service gives you the ability to evaluate a model's performance across specific use cases. Vertex AI provides the generative AI evaluation service, which includes both online and pipeline evaluation. This page includes examples of using the online evaluation service.
Set up your Google Cloud project
Set up your Google Cloud project. See Get set up on Google Cloud for detailed steps.
Perform evaluation
We provide several evaluation example notebooks to help you start evaluating your generative language models.
To start using our rapid evaluation online evaluation service, follow the steps in the Getting Started with the Vertex AI Python SDK for rapid evaluation notebook, which runs through an end-to-end evaluation of a Gemini Pro model.
For more information on all of our evaluation offerings, reference our evaluation overview.
What's next
- Learn about generative AI evaluation.
- Learn about evaluation examples.
- Learn about online evaluation with rapid evaluation.
- Learn about model-based pairwise evaluation with AutoSxS pipeline.
- Learn about the computation-based evaluation pipeline.
- Learn how to tune a foundation model.