You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/api-management/api-management-howto-llm-logs.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -31,7 +31,7 @@ Learn more about:
31
31
32
32
## Prerequisites
33
33
- An Azure API Management instance.
34
-
- A managed LLM chat completions API integrated with Azure API Management. For example, [Import an Azure AI Foundry API](azure-ai-foundry-api.md).
34
+
- A managed LLM chat completions API integrated with Azure API Management. For example, [Import a Microsoft Foundry API](azure-ai-foundry-api.md).
35
35
- Access to an Azure Log Analytics workspace.
36
36
- Appropriate permissions to configure diagnostic settings and access logs in API Management.
37
37
@@ -106,17 +106,17 @@ ApiManagementGatewayLlmLog
106
106
107
107
:::image type="content" source="media/api-management-howto-llm-logs/llm-log-query-small.png" alt-text="Screenshot of query results for LLM logs in the portal." lightbox="media/api-management-howto-llm-logs/llm-log-query.png":::
108
108
109
-
## Upload data to Azure AI Foundry for model evaluation
109
+
## Upload data to Microsoft Foundry for model evaluation
110
110
111
-
You can export LLM logging data as a dataset for [model evaluation](/azure/ai-foundry/concepts/observability) in Azure AI Foundry. With model evaluation, you can assess the performance of your generative AI models and applications against a test model or dataset using built-in or custom evaluation metrics.
111
+
You can export LLM logging data as a dataset for [model evaluation](/azure/ai-foundry/concepts/observability) in Microsoft Foundry. With model evaluation, you can assess the performance of your generative AI models and applications against a test model or dataset using built-in or custom evaluation metrics.
112
112
113
113
To use LLM logs as a dataset for model evaluation:
114
114
115
115
1. Join LLM request and response messages into a single record for each interaction, as shown in the [previous section](#review-azure-monitor-logs-for-requests-and-responses). Include the fields you want to use for model evaluation.
116
-
1. Export the dataset to CSV format, which is compatible with Azure AI Foundry.
117
-
1. In the Azure AI Foundry portal, create a new evaluation to upload and evaluate the dataset.
116
+
1. Export the dataset to CSV format, which is compatible with Microsoft Foundry.
117
+
1. In the Microsoft Foundry portal, create a new evaluation to upload and evaluate the dataset.
118
118
119
-
For details to create and run a model evaluation in Azure AI Foundry, see [Evaluate generative AI models and applications by using Azure AI Foundry](/azure/ai-foundry/how-to/evaluate-generative-ai-app).
119
+
For details to create and run a model evaluation in Microsoft Foundry, see [Evaluate generative AI models and applications by using Microsoft Foundry](/azure/ai-foundry/how-to/evaluate-generative-ai-app).
You can import AI model endpoints deployed in Azure AI Foundry to your API Management instance as APIs. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
18
+
You can import AI model endpoints deployed in Microsoft Foundry to your API Management instance as APIs. Use AI gateway policies and other capabilities in API Management to simplify integration, improve observability, and enhance control over the model endpoints.
19
19
20
20
Learn more about managing AI APIs in API Management:
21
21
@@ -24,23 +24,23 @@ Learn more about managing AI APIs in API Management:
24
24
25
25
## Client compatibility options
26
26
27
-
API Management supports two client compatibility options for AI APIs from Azure AI Foundry. When you import the API using the wizard, choose the option suitable for your model deployment. The option determines how clients call the API and how the API Management instance routes requests to the AI service.
27
+
API Management supports two client compatibility options for AI APIs from Microsoft Foundry. When you import the API using the wizard, choose the option suitable for your model deployment. The option determines how clients call the API and how the API Management instance routes requests to the AI service.
28
28
29
-
***Azure OpenAI** - Manage Azure OpenAI in Azure AI Foundry model deployments.
29
+
***Azure OpenAI** - Manage Azure OpenAI in Microsoft Foundry model deployments.
30
30
31
31
Clients call the deployment at an `/openai` endpoint such as `/openai/deployments/my-deployment/chat/completions`. Deployment name is passed in the request path. Use this option if your AI service only includes Azure OpenAI model deployments.
32
32
33
-
***Azure AI** - Manage model endpoints in Azure AI Foundry that are exposed through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api).
33
+
***Azure AI** - Manage model endpoints in Microsoft Foundry that are exposed through the [Azure AI Model Inference API](/azure/ai-studio/reference/reference-model-inference-api).
34
34
35
35
Clients call the deployment at a `/models` endpoint such as `/my-model/models/chat/completions`. Deployment name is passed in the request body. Use this option if you want flexibility to switch between models exposed through the Azure AI Model Inference API and those deployed in Azure OpenAI in Foundry Models.
36
36
37
37
## Prerequisites
38
38
39
39
- An existing API Management instance. [Create one if you haven't already](get-started-create-service-instance.md).
40
40
41
-
- An Azure AI service in your subscription with one or more models deployed. Examples include models deployed in Azure AI Foundry or Azure OpenAI.
41
+
- An Azure AI service in your subscription with one or more models deployed. Examples include models deployed in Microsoft Foundry or Azure OpenAI.
42
42
43
-
## Import AI Foundry API using the portal
43
+
## Import Microsoft Foundry API using the portal
44
44
45
45
Use the following steps to import an AI API to API Management.
46
46
@@ -52,11 +52,11 @@ When you import the API, API Management automatically configures:
52
52
* Authentication to the backend using the instance's system-assigned managed identity.
53
53
* (optionally) Policies to help you monitor and manage the API.
54
54
55
-
To import an AI Foundry API to API Management:
55
+
To import an Microsoft Foundry API to API Management:
56
56
57
57
1. In the [Azure portal](https://portal.azure.com), navigate to your API Management instance.
58
58
1. In the left menu, under **APIs**, select **APIs** > **+ Add API**.
59
-
1. Under **Create from Azure resource**, select **Azure AI Foundry**.
59
+
1. Under **Create from Azure resource**, select **Microsoft Foundry**.
60
60
61
61
:::image type="content" source="media/azure-ai-foundry-api/ai-foundry-api.png" alt-text="Screenshot of creating an OpenAI-compatible API in the portal." :::
62
62
1. On the **Select AI service** tab:
@@ -69,11 +69,11 @@ To import an AI Foundry API to API Management:
69
69
1. In **Base path**, enter a path that your API Management instance uses to access the deployment endpoint.
70
70
1. Optionally select one or more **Products** to associate with the API.
71
71
1. In **Client compatibility**, select either of the following based on the types of client you intend to support. See [Client compatibility options](#client-compatibility-options) for more information.
72
-
***Azure OpenAI** - Select this option if your clients only need to access Azure OpenAI in AI Foundry model deployments.
73
-
***Azure AI** - Select this option if your clients need to access other models in Azure AI Foundry.
72
+
***Azure OpenAI** - Select this option if your clients only need to access Azure OpenAI in Microsoft Foundry model deployments.
73
+
***Azure AI** - Select this option if your clients need to access other models in Microsoft Foundry.
74
74
1. Select **Next**.
75
75
76
-
:::image type="content" source="media/azure-ai-foundry-api/client-compatibility.png" alt-text="Screenshot of AI Foundry API configuration in the portal.":::
76
+
:::image type="content" source="media/azure-ai-foundry-api/client-compatibility.png" alt-text="Screenshot of Microsoft Foundry API configuration in the portal.":::
77
77
78
78
1. On the **Manage token consumption** tab, optionally enter settings or accept defaults that define the following policies to help monitor and manage the API:
Enable semantic caching of responses to LLM API requests to reduce bandwidth and processing requirements imposed on the backend APIs and lower latency perceived by API consumers. With semantic caching, you can return cached responses for identical prompts and also for prompts that are similar in meaning, even if the text isn't identical. For background, see [Tutorial: Use Azure Managed Redis as a semantic cache](../redis/tutorial-semantic-cache.md).
20
20
21
21
> [!NOTE]
22
-
> The configuration steps in this article show how to enable semantic caching for APIs added to API Management from Azure OpenAI in Azure AI Foundry models. You can apply similar steps to enable semantic caching for corresponding large language model (LLM) APIs available through the [Azure AI Model Inference API](/rest/api/aifoundry/modelinference/) or with OpenAI-compatible models served through third-party inference providers.
22
+
> The configuration steps in this article show how to enable semantic caching for APIs added to API Management from Azure OpenAI in Microsoft Foundry models. You can apply similar steps to enable semantic caching for corresponding large language model (LLM) APIs available through the [Azure AI Model Inference API](/rest/api/aifoundry/modelinference/) or with OpenAI-compatible models served through third-party inference providers.
23
23
24
24
## Prerequisites
25
25
26
-
* Add one or more Azure OpenAI in Azure AI Foundry model deployments as APIs to your API Management instance. For more information, see [Add an Azure OpenAI API to Azure API Management](azure-openai-api-from-specification.md).
26
+
* Add one or more Azure OpenAI in Microsoft Foundry model deployments as APIs to your API Management instance. For more information, see [Add an Azure OpenAI API to Azure API Management](azure-openai-api-from-specification.md).
27
27
* Create deployments for the following APIs:
28
28
29
29
* Chat Completion API - Deployment used for API consumer calls
Copy file name to clipboardExpand all lines: articles/api-management/backends.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,7 +51,7 @@ Configure and manage backend entities in the Azure portal, or by using Azure API
51
51
You can create a backend in the Azure portal, or by using Azure APIs or tools.
52
52
53
53
> [!NOTE]
54
-
> When you import certain APIs, such as APIs from Azure AI Foundry or other AI services, API Management automatically configures a backend entity.
54
+
> When you import certain APIs, such as APIs from Microsoft Foundry or other AI services, API Management automatically configures a backend entity.
55
55
56
56
To create a backend in the portal:
57
57
@@ -309,7 +309,7 @@ With any of the preceding load balancing options, you can enable **session aware
309
309
310
310
When you use session awareness, the client must handle cookies appropriately. The client needs to store the `Set-Cookie` header value and send it with subsequent requests to maintain session state.
311
311
312
-
You can use API Management policies to help set cookies for session awareness. For example, for the case of the Assistants API (a feature of [Azure OpenAI in Azure AI Foundry Models](/azure/ai-services/openai/concepts/models)), the client needs to keep the session ID, extract the thread ID from the body, and keep the pair and send the right cookie for each call. Moreover, the client needs to know when to send a cookie or when not to send a cookie header. These requirements can be handled appropriately by defining the following example policies:
312
+
You can use API Management policies to help set cookies for session awareness. For example, for the case of the Assistants API (a feature of [Azure OpenAI in Microsoft Foundry Models](/azure/ai-services/openai/concepts/models)), the client needs to keep the session ID, extract the thread ID from the body, and keep the pair and send the right cookie for each call. Moreover, the client needs to know when to send a cookie or when not to send a cookie header. These requirements can be handled appropriately by defining the following example policies:
![learn-build-service-prod[bot]](https://avatars.githubusercontent.com/in/237442?v=4&size=40){kind=avatar}
0 commit comments