You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: "Secure Azure OpenAI and Cognitive Services"
4
4
metadata:
5
5
title: "Secure Azure OpenAI Service and Cognitive Services"
6
6
description: "Learn how to secure Azure OpenAI Service and Cognitive Services with managed identities, private endpoints, and content filtering for compliance."
Copy file name to clipboardExpand all lines: learn-pr/wwl-azure/implement-secure-ai-ready-infrastructure-azure-services/includes/1-introduction.md
-3Lines changed: 0 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,6 +1,3 @@
1
-
2
-
## Learning objectives
3
-
4
1
Microsoft Foundry Hubs and Projects deliver the governance framework your security team requires. By combining Foundry's centralized policy enforcement with Azure OpenAI Service, Azure Cognitive Services, and Azure Container Registry, you build AI infrastructure that passes enterprise security reviews while maintaining developer agility. This approach eliminates public internet exposure through private endpoints, removes credential sprawl through managed identities, and provides visibility into container vulnerabilities before production deployment.
5
2
6
3
In this module, you configure a Foundry Hub to enforce security policies across multiple AI projects, integrate Azure OpenAI Service with network isolation and identity controls, and deploy Azure Container Registry with automated vulnerability scanning. By the end, you have production-ready AI infrastructure that satisfies your compliance team and accelerates safe AI adoption across your organization.
Copy file name to clipboardExpand all lines: learn-pr/wwl-azure/implement-secure-ai-ready-infrastructure-azure-services/includes/2-understand-microsoft-foundry-security-architecture.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ This becomes especially important when your security team needs to respond to ne
17
17
18
18
## Core security components
19
19
20
-
The hub integrates three foundational security services that protect your AI workloads. Microsoft Entra ID provides identity and access management through role-based access control (RBAC), allowing you to define who can create projects, deploy models, and access training data. Unlike service-specific authentication, Entra ID centralizes identity management—your existing user groups and conditional access policies apply automatically to AI resources. At the same time, managed identities eliminate credential storage by allowing applications and services to authenticate directly using Azure Active Directory tokens, removing API keys from your codebase entirely.
20
+
The hub integrates three foundational security services that protect your AI workloads. Microsoft Entra ID provides identity and access management through role-based access control (RBAC), allowing you to define who can create projects, deploy models, and access training data. Unlike service-specific authentication, Microsoft Entra ID centralizes identity management—your existing user groups and conditional access policies apply automatically to AI resources. At the same time, managed identities eliminate credential storage by allowing applications and services to authenticate directly using Microsoft Entra tokens, removing API keys from your codebase entirely.
21
21
22
22
Azure Virtual Network delivers network isolation by creating private connectivity between your hub and Azure services. When you deploy a private endpoint for Azure OpenAI Service, all API traffic flows through your virtual network rather than the public internet. This network boundary prevents external access attempts and ensures compliance with data residency requirements. For example, your fraud detection models processing customer transactions stay within EU Azure regions, satisfying data localization mandates.
23
23
@@ -29,7 +29,7 @@ Now that you understand the hub's role, let's examine how projects maintain both
29
29
30
30
Building on this concept, policy inheritance works like a security firewall with mandatory baseline rules. The hub enforces non-negotiable requirements—private endpoints, managed identities, encryption at rest—that projects can't disable. Within these boundaries, project administrators customize permissions for their team members and configure project-specific resources like storage containers or compute clusters. For instance, the fraud detection project might grant senior data scientists permission to deploy production models while restricting junior analysts to development experiments. These project-level permissions add flexibility without compromising hub-enforced security controls.
31
31
32
-
Consider what happens when a new data scientist joins your fraud detection team. The project administrator assigns them the "Data Scientist" role within the Foundry Project. This role inherits hub-level policies automatically—they can only access AI services through private endpoints and must authenticate with their Entra ID credentials. The project role then grants specific permissions to training datasets and development compute resources. This layered approach means your security team manages baseline controls once at the hub level, while project administrators handle day-to-day access management within their domains.
32
+
Consider what happens when a new data scientist joins your fraud detection team. The project administrator assigns them the "Data Scientist" role within the Foundry Project. This role inherits hub-level policies automatically—they can only access AI services through private endpoints and must authenticate with their Microsoft Entra ID credentials. The project role then grants specific permissions to training datasets and development compute resources. This layered approach means your security team manages baseline controls once at the hub level, while project administrators handle day-to-day access management within their domains.
33
33
34
34
:::image type="content" source="../media/architecture-centralized-security-controls.png" alt-text="Diagram showing the Microsoft Foundry Hub architecture with centralized security controls.":::
Copy file name to clipboardExpand all lines: learn-pr/wwl-azure/implement-secure-ai-ready-infrastructure-azure-services/includes/4-secure-ai-container-images-azure-container.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -21,7 +21,7 @@ Now that you understand vulnerability management, let's examine how ACR controls
21
21
22
22
Azure Container Registry provides two layers of access control that address these requirements. Azure RBAC roles (AcrPush, AcrPull, AcrDelete) grant registry-wide permissions suitable for administrators and CI/CD service principals that manage multiple repositories. Building on this concept, repository-scoped tokens deliver granular access control by allowing you to create tokens with permissions limited to specific repositories. For example, you create a token named "fraud-detection-ci" with push permission only to the "fraud-detection" repository. Your CI/CD pipeline uses this token to publish new model containers but can't modify other teams' repositories or pull production images from protected repositories.
23
23
24
-
At the same time, managed identities eliminate tokens entirely when integrating ACR with Azure services. Your Azure Machine Learning workspace uses its managed identity to pull model containers for inference deployments. You assign the AcrPull role to the workspace's managed identity at the repository scope, granting pull access only to approved model repositories. This approach provides audit trails through Entra ID access logs and eliminates token management overhead—no credentials to rotate, secure, or accidentally expose in configuration files.
24
+
At the same time, managed identities eliminate tokens entirely when integrating ACR with Azure services. Your Azure Machine Learning workspace uses its managed identity to pull model containers for inference deployments. You assign the AcrPull role to the workspace's managed identity at the repository scope, granting pull access only to approved model repositories. This approach provides audit trails through Microsoft Entra ID access logs and eliminates token management overhead—no credentials to rotate, secure, or accidentally expose in configuration files.
25
25
26
26
## Content trust: Verifying image integrity
27
27
@@ -45,7 +45,7 @@ ACR geo-replication solves both problems by maintaining synchronized registry re
45
45
46
46
Now that you understand the individual security features, let's see how ACR integrates with the AI platforms your applications actually run on. Azure Machine Learning workspaces pull model containers from ACR for batch scoring jobs and real-time inference endpoints. When you deploy a model, Machine Learning uses its managed identity to authenticate to ACR, pulls the specified container image through your private endpoint, and launches the inference service. Because ACR provides the container, vulnerability scanning, and access logging, your security team has complete visibility into which models are deployed and whether they contain known vulnerabilities.
47
47
48
-
Azure Kubernetes Service (AKS) extends this integration to more complex AI workloads. Your fraud detection system runs as microservices in an AKS cluster, with each service packaged as a container stored in ACR. AKS worker nodes use managed identities to pull images from ACR through private endpoints. When you update a fraud detection model, you push a new container to ACR, Defender scans it for vulnerabilities, and Kubernetes automatically pulls the updated image during the next deployment. This workflow combines ACRs security controls with AKS's orchestration capabilities, ensuring every deployed container has passed security validation before serving production traffic.
48
+
Azure Kubernetes Service (AKS) extends this integration to more complex AI workloads. Your fraud detection system runs as microservices in an AKS cluster, with each service packaged as a container stored in Azure Container Registry. AKS worker nodes use managed identities to pull images from Azure Container Registry through private endpoints. When you update a fraud detection model, you push a new container to ACR, Defender scans it for vulnerabilities, and Kubernetes automatically pulls the updated image during the next deployment. This workflow combines Azure Container Registry's security controls with AKS's orchestration capabilities, ensuring every deployed container has passed security validation before serving production traffic.
49
49
50
50
With this understanding of container security—from vulnerability scanning through access control to network isolation—you've now covered the complete security architecture for AI infrastructure. You're ready to implement these controls in a hands-on exercise that validates your production-ready configuration.
Copy file name to clipboardExpand all lines: learn-pr/wwl-azure/implement-secure-ai-ready-infrastructure-azure-services/includes/5-exercise-configure-secure-ai-infrastructure.md
+5-4Lines changed: 5 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -6,9 +6,10 @@
6
6
### Scenario
7
7
Your company is building a centralized AI platform on Microsoft Foundry to support enterprise knowledge management and decision support across multiple business units, with a strong emphasis on network isolation and compliance-driven security controls. The platform enables intelligent document retrieval, automated analysis of internal reports, and conversational insights for employees, while ensuring that all traffic remains within approved network boundaries. To meet these requirements, the team deploys a Standard agent environment with private networking, integrating the Foundry account and project into an existing customer-managed virtual network that will host agents and private endpoints connecting services associated with the Microsoft Foundry resources. These services include customer-owned Azure Storage, Azure Cosmos DB, and Azure AI Search instances to be used for storing documents, conversation history, and search indexes without public exposure. The environment is deployed by using template-based automation.
8
8
9
-
:::image type="content" source="../media/company-building-centralized-platform-foundry.png" alt-text="Diagram showing a company building a centralized AI platform on Microsoft Foundry.":::
9
+
:::image type="content" source="../media/company-building-centralized-ai-platform.png" alt-text="Diagram showing a company building a centralized AI platform on Microsoft Foundry.":::
10
10
11
-
**Important**: The exercise involves provisioning a virtual network with two subnets and reviewing the process of a template-based deployment of a Microsoft Foundry standard agent environment with private networking, without the actual implementation of such environment. This is intentional in order to minimize the duration and cost of the exercise. There are more considerations associated with the deprovisioning process, which would further increase the cost and complexity of the implementation tasks.
11
+
[!Important]
12
+
The exercise involves provisioning a virtual network with two subnets and reviewing the process of a template-based deployment of a Microsoft Foundry standard agent environment with private networking, without the actual implementation of such environment. This is intentional in order to minimize the duration and cost of the exercise. There are more considerations associated with the deprovisioning process, which would further increase the cost and complexity of the implementation tasks.
12
13
13
14
[:::image type="content" source="../media/launch-exercise-button.png" alt-text="Button to launch exercise.":::](https://go.microsoft.com/fwlink/?linkid=2349800)
14
15
@@ -24,7 +25,7 @@ Securing network connectivity for Microsoft Foundry hubs is essential for mainta
24
25
### Scenario
25
26
Your company operates in the financial services industry, where strict regulatory requirements and data protection policies govern how sensitive information is accessed and processed. The organization plans to build a centralized AI platform on Microsoft Foundry to support the development of machine learning models used for fraud detection, credit risk assessment, and transaction analysis. Because these workloads involve highly confidential customer and financial data, ensuring network isolation and controlled data movement is a top priority.
26
27
27
-
To meet internal security policies and industry compliance standards (such as ISO 27001 and SOC 2), the company plans to deploy the Microsoft Foundry hub within a managed virtual network. This managed network isolates all AI resources from the public internet and enforce strict outbound connectivity rules to prevent accidental data exposure. Private endpoints are used to connect securely to dependent Azure services, including Storage accounts, Key Vault, and the Azure Container Registry, while private DNS zones ensure reliable name resolution within the isolated environment.
28
+
To meet internal security policies and industry compliance standards (such as ISO 27001 and SOC 2), the company plans to deploy the Microsoft Foundry hub within a managed virtual network. This managed network isolates all AI resources from the public internet and enforces strict outbound connectivity rules to prevent accidental data exposure. Private endpoints are used to connect securely to dependent Azure services, including Storage accounts, Key Vault, and the Azure Container Registry, while private DNS zones ensure reliable name resolution within the isolated environment.
28
29
29
30
:::image type="content" source="../media/financial-services-industry-strict-regulatory.png" alt-text="Diagram showing a company operating in the financial services industry with strict regulatory requirements.":::
30
31
@@ -45,7 +46,7 @@ Your company is a financial services firm that handles highly sensitive customer
45
46
46
47
Your company plans to build a centralized AI platform on Microsoft Foundry to support these operations. To ensure full control over encryption, the organization intends to use customer-managed keys, allowing it to enforce key rotation policies and maintain auditability of key usage. System-assigned managed identities will be used for the Microsoft Foundry resources to securely access the Key Vault storing the CMKs, reducing administrative overhead and minimizing the risk of accidental key exposure. This approach allows the company to protect sensitive data while applying AI workloads without modifying application code.
47
48
48
-
:::image type="content" source="../media/company-building-centralized-ai-platform.png" alt-text="Diagram showing of a company building a centralized AI platform on Microsoft Foundry.":::
49
+
:::image type="content" source="../media/company-building-centralized-ai-platform.png" alt-text="Diagram showing a company building a centralized AI platform on Microsoft Foundry.":::
49
50
50
51
[:::image type="content" source="../media/launch-exercise-button.png" alt-text="Button to launch exercise.":::](https://go.microsoft.com/fwlink/?linkid=2349901)
Copy file name to clipboardExpand all lines: learn-pr/wwl-azure/implement-secure-ai-ready-infrastructure-azure-services/includes/7-summary.md
+1-4Lines changed: 1 addition & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -22,7 +22,4 @@ Expand your secure AI infrastructure by implementing these advanced capabilities
22
22
## More resources
23
23
24
24
-[Monitor Azure AI workloads](/azure/ai-services/openai/how-to/monitoring) - Implementation guide for Azure Monitor integration with AI services
25
-
-[Azure AI security baseline](/security/benchmark/azure/baselines/ai-services-security-baseline) - Comprehensive security recommendations for Azure AI deployments
26
-
-[Responsible AI practices](/azure/ai-services/responsible-use-of-ai-overview) - Framework for implementing ethical and compliant AI systems
27
-
28
-
25
+
-[Responsible AI practices](/azure/ai-services/responsible-use-of-ai-overview) - Framework for implementing ethical and compliant AI systems
title: "Implement Secure AI Infrastructure in Azure"
5
5
description: "This course explains how to design secure AI platforms using Microsoft Foundry, applying centralized governance, managed identities, private networking, Azure OpenAI security controls, and container image protection to ensure compliant, production‑ready AI workloads across enterprise environments."
0 commit comments