Merge pull request #53460 from ivorb/new-agents

model catalog evaluation module

Author: JillGrant615

learn-pr/paths/develop-generative-ai-apps/index.yml

@@ -0,0 +1,38 @@
+### YamlMime:LearningPath
+uid: learn.wwl.develop-generative-ai-apps
+hidden: true
+metadata:
+  title: Develop generative AI apps on Microsoft Foundry
+  description: Learn how to develop generative AI apps in Microsoft Foundry. (AI-3016)
+  ms.date: 02/17/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: learning-path
+  ms.collection: wwl-ai-copilot
+  ms.custom: [copilot-learning-hub]
+title: Develop generative AI apps in Azure
+prerequisites: |
+  Before starting this module, you should be familiar with fundamental AI concepts and services in Azure. You should also have programming experience.
+summary: |
+  Generative artificial intelligence (AI) is becoming more accessible through comprehensive development platforms like Microsoft Foundry. Learn how to build generative AI applications that use language models to interact with your users.
+iconUrl: /training/achievements/generic-badge.svg
+levels:
+- intermediate
+roles:
+- data-scientist
+- ai-engineer
+products:
+- ai-services
+- azure-ai-foundry
+- azure-ai-foundry-sdk
+subjects:
+- artificial-intelligence
+modules:
+- learn.wwl.prepare-azure-ai-development
+- learn.wwl.model-catalog-evaluate
+- learn.wwl.ai-foundry-sdk
+- learn.wwl.finetune-model-copilot-ai-studio
+- learn.wwl.responsible-ai-studio
+trophy:
+  uid: learn.wwl.develop-generative-ai-apps.trophy
+

learn-pr/wwl-data-ai/model-catalog-evaluate/1-introduction.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.introduction
+title: Introduction
+metadata:
+  title: Introduction
+  description: Learn why model selection and evaluation are essential for building effective generative AI applications.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 3
+content: |
+  [!include[](includes/1-introduction.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/2-explore-model-catalog.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.explore-model-catalog
+title: Explore the model catalog
+metadata:
+  title: Explore the model catalog
+  description: Learn how to browse and filter models in the Microsoft Foundry portal model catalog.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 7
+content: |
+  [!include[](includes/2-explore-model-catalog.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/3-select-models-benchmarks.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.select-models-benchmarks
+title: Select models using benchmarks
+metadata:
+  title: Select models using benchmarks
+  description: Understand how to use quality, safety, cost, and performance benchmarks to compare and select models.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 9
+content: |
+  [!include[](includes/3-select-models-benchmarks.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/4-deploy-models.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.deploy-models
+title: Deploy models to endpoints
+metadata:
+  title: Deploy models to endpoints
+  description: Learn how to deploy models to different endpoint types and test them in the playground.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 8
+content: |
+  [!include[](includes/4-deploy-models.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/5-evaluate-performance.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.evaluate-performance
+title: Evaluate model performance
+metadata:
+  title: Evaluate model performance
+  description: Discover how to evaluate models using manual testing, AI-assisted metrics, and NLP metrics.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 10
+content: |
+  [!include[](includes/5-evaluate-performance.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/6-exercise.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.exercise
+title: Exercise - Select, deploy, and evaluate models
+metadata:
+  title: Exercise - Select, deploy, and evaluate models
+  description: Practice selecting, deploying, and evaluating models in Microsoft Foundry portal.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 20
+content: |
+  [!include[](includes/6-exercise.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/7-knowledge-check.yml

@@ -0,0 +1,48 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.knowledge-check
+title: Knowledge check
+metadata:
+  title: Knowledge check
+  description: Check your understanding of model selection, deployment, and evaluation concepts.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 3
+quiz:
+  title: Check your knowledge
+  questions:
+  - content: "Which deployment type in Microsoft Foundry portal requires an Azure Marketplace subscription for models from partners and the community?"
+    choices:
+    - content: "Managed compute"
+      isCorrect: false
+      explanation: "Incorrect. Managed compute doesn't necessarily require Azure Marketplace subscriptions. It uses Azure virtual machines and is billed for hosting and inference costs."
+    - content: "Serverless API"
+      isCorrect: true
+      explanation: "Correct. Serverless API deployments for models from partners and community require Azure Marketplace subscriptions, while models sold directly by Azure don't require this subscription."
+    - content: "Provisioned"
+      isCorrect: false
+      explanation: "Incorrect. Provisioned deployments reserve dedicated capacity but don't specifically require Azure Marketplace subscriptions."
+  - content: "Which evaluation metric measures whether model responses are based on provided context rather than speculation?"
+    choices:
+    - content: "Fluency"
+      isCorrect: false
+      explanation: "Incorrect. Fluency evaluates linguistic correctness and natural language quality, not whether responses are grounded in context."
+    - content: "Groundedness"
+      isCorrect: true
+      explanation: "Correct. Groundedness determines whether responses are based on provided context rather than speculation or the model's general knowledge."
+    - content: "Coherence"
+      isCorrect: false
+      explanation: "Incorrect. Coherence assesses whether responses flow logically and maintain consistent ideas, not whether they're based on provided context."
+  - content: "What type of model should you select if your application needs to process both text and images?"
+    choices:
+    - content: "Small Language Model (SLM)"
+      isCorrect: false
+      explanation: "Incorrect. SLMs are efficient text-focused models but don't inherently process images alongside text."
+    - content: "Embedding model"
+      isCorrect: false
+      explanation: "Incorrect. Embedding models convert text into numerical representations for semantic search and similarity tasks, not for processing images."
+    - content: "Multimodal model"
+      isCorrect: true
+      explanation: "Correct. Multimodal models like GPT-4o and Phi-3-vision can process multiple data types including both text and images."

learn-pr/wwl-data-ai/model-catalog-evaluate/8-summary.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.model-catalog-evaluate.summary
+title: Summary
+metadata:
+  title: Summary
+  description: Review key concepts for selecting, deploying, and evaluating models in Microsoft Foundry portal.
+  ms.date: 02/13/2026
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+  ai-usage: ai-assisted
+durationInMinutes: 3
+content: |
+  [!include[](includes/8-summary.md)]

learn-pr/wwl-data-ai/model-catalog-evaluate/includes/1-introduction.md

@@ -0,0 +1,15 @@
+Building effective generative AI applications requires selecting the right foundation model for your specific use case. With thousands of models available, you need a structured approach to discover, compare, deploy, and validate that a model meets your requirements.
+
+Consider a scenario where you're building an AI-powered customer support chatbot for a retail company. You need to select a language model that can understand customer questions, provide accurate responses, and maintain appropriate tone and safety standards. But how do you choose from the vast catalog of available models? How do you know if a model performs well for your specific needs? And once deployed, how do you measure and improve its performance?
+
+The Microsoft Foundry portal provides a comprehensive platform for this entire workflow. You can explore over 1,900 models from providers like Microsoft, Anthropic, OpenAI, Meta, and Hugging Face. You can compare models using industry-standard benchmarks for quality, safety, cost, and performance. After selecting a model, you deploy it to an endpoint where your application can consume it. Finally, you evaluate the model's performance using both automated metrics and manual testing to ensure it meets your quality and safety requirements.
+
+In this module, you explore how to use the Microsoft Foundry portal to select, deploy, and evaluate models from the model catalog. You learn how to make informed decisions about model selection, understand different deployment options, and assess model performance using various evaluation approaches.
+
+By the end of this module, you'll be able to:
+
+- Explore and filter models in the model catalog
+- Compare models using benchmark metrics for quality, safety, cost, and performance
+- Deploy a model to an endpoint and test it in the playground
+- Evaluate model performance using manual and automated approaches
+- Understand different evaluation metrics and when to use them