Evaluate generative AI applications

Microsoft via Microsoft Learn

Microsoft Learn based on 0 ratings

Likes

Overview

Module 1: Learn the fundamental concepts of evaluating generative AI applications.By the end of this module, you're able to: Apply best practices for choosing evaluation data Understood the purpose of and types of synthetic data for evaluation Comprehend the scope of the built-in metrics Choose the appropriate metrics based on your AI system use case Understand how to interpret evaluation results Module 2: Learn how to run evaluations and generate synthetic datasets with the Azure AI Evaluation SDK.By...

Syllabus

Module 1: Module 1: Evaluating generative AI applications
- Introduction
- Evaluate generative AI
- The role of data in evaluations
- Choose and utilize metrics
- Custom prompt-based evaluators
- Interpret evaluation results
- Common pitfalls in result interpretation
- Knowledge check
- Summary

Module 2: Module 2: Run evaluations and generate synthetic datasets
- Introduction
- Prepare
- Exercise - Performance and quality metrics
- Exercise - Risk and safety metrics
- Exercise - Track evaluation results in Azure AI Studio
- Exercise - Custom evaluator with Prompty
- Exercise - Evaluate an endpoint
- Exercise - Generate a synthetic dataset
- Knowledge check
- Summary