We offer a comprehensive set of notebooks that demonstrate how to use Vertex AI LLM Evaluation Services in conjunction with other Vertex AI services. Additionally, we have provided notebooks that delve into the theory behind evaluation metrics.
Theory notebooks:
- Metrics for Classification
- Metrics for Summarization
- Metrics for Text Generation
- Metrics for Q&A
To run the walkthrough and demonstration in the notebook you'll need access to a Google Cloud project with the Vertex AI API enabled.
If you have any questions or find any problems, please report through GitHub issues.