Skip to content

build-bot-based-agent.md

Latest commit

 

History

History
371 lines (239 loc) · 22.5 KB

build-bot-based-agent.md

File metadata and controls

371 lines (239 loc) · 22.5 KB
title Message Extension as Agent for Copilot
description Learn how to build a bot-based message extension agent for Microsoft 365 Copilot using Developer Portal for Teams and Microsoft 365 Agents Toolkit.
ms.localizationpriority medium
ms.topic overview
ms.author anclear
ms.date 09/16/2024
ms.collection ce-skilling-ai-copilot

Extend bot-based message extension as agent for Microsoft 365 Copilot

Note

  • Ensure that Microsoft 365 Copilot is available for your organization. You have two ways to get a developer environment for Microsoft 365 Copilot:
  • Bot-based search message extension agent is available in public developer preview.
  • Only bot-based message extensions with search commands can be extended as agents for Microsoft 365 Copilot.

Microsoft 365 Copilot, powered by an advanced processing and orchestration engine, integrates Microsoft 365 apps, Microsoft Graph, and Large Language Models (LLMs) to transform your words into a potent productivity tool. Although Microsoft 365 Copilot can utilize apps and data within the Microsoft 365 ecosystem, many users rely on various external tools and services for work management and collaboration. By extending your message extension as an agent in Microsoft 365 Copilot, you can enable users to interact with third-party tools and services, therefore empowering them to achieve more with Microsoft 365 Copilot. You can achieve this extension by developing an agent or connecting to an external data source.

:::image type="content" source="../assets/images/Copilot/ailib-copilot-diff.png" alt-text="Graphic shows the user interaction flow between the user, Microsoft Teams, and Microsoft 365 Copilot." lightbox="../assets/images/Copilot/ailib-copilot-diff.png":::

[!INCLUDE bot-based-me-note]

What is an agent?

An agent allows Microsoft 365 Copilot to interact directly with third-party data, apps, and services, enhance its capabilities, and broaden its range of capabilities. Agents allow Microsoft 365 Copilot to:

  • Fetch real-time information, such as the latest news coverage on a product launch.
  • Retrieve knowledge-based information, such as a team’s design files in Figma.

Descriptions enhance the usability and effectiveness of a message extension agent. The following description offer a clear and concise summary of the app’s features:

  • App description: App description helps improve your app discoverability in the Teams Store.
  • Command description: Command description maps user intent and utterance to search command inside an agent and must be built based on the analysis of the user intent and keywords.
  • Parameter description: Parameter description explains the requirements of the system in a natural language with output format.
  • Semantic description: Semantic description helps Microsoft 365 Copilot generate content by providing a conceptual understanding of the agent's capabilities and scenarios where it can help achieve user goals and match user’s intent with the agent's capabilities.

For more information, see guidelines to create and upgrade agents.

All bot-based search message extensions that are eligible for agent support must meet policy requirement and are subject to validation to ensure the agent meets quality, security, privacy, and usefulness expectations. You can create a bot-based search message extension using Microsoft 365 Agents Toolkit (previously known as Teams Toolkit) for Visual Studio Code, Visual Studio, Agents Toolkit command line interface (CLI) (previously known as Teams Toolkit CLI), or Developer Portal for Teams and extend the message extension to function as an agent in Microsoft 365 Copilot.

Prerequisites

Before you get started, ensure that you're familiar with the following standards and guidelines for building message extension agents for Microsoft 365 Copilot:

Create bot-based message extension

Important

Agents for Microsoft 365 Copilot are in preview and work only in Microsoft 365 Copilot in Teams.

Let's create a bot-based search message extension agent for Teams and Microsoft 365 Copilot and share results through the compose message area of the Microsoft Teams client.

Visual Studio Code

Before you get started, ensure that you install the following tools to build and deploy your message extension:

  • Install the latest Agents Toolkit prerelease version.
  • Ensure that the Develop Copilot Plugin feature flag is enabled. To enable the feature flag, follow these steps:
    1. Open Visual Studio Code.
    2. Go to Manage :::image type="icon" source="../assets/icons/gear-icon.png" border="false"::: > Settings.
    3. Enter Microsoft 365 Agents Toolkit in the Search settings search box.
    4. Under Extensions, select the Fx-extension: Develop Copilot Plugin checkbox.

To create a bot-based search message extension agent using Visual Studio Code, follow these steps:

  1. Open Visual Studio Code.

  2. From the left pane, select Microsoft 365 Agents Toolkit.

  3. Select Create a New Agent/App.

  4. Select Message Extension.

  5. Select Custom Search Results.

  6. Select Start with a Bot.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-start-with-bot.png" alt-text="Screenshot shows the Start with a Bot option to create a bot-based message extension in Visual Studio Code.":::

  7. Select a programming language.

  8. Select Default folder.

  9. Enter the name of your app and select Enter. Agents Toolkit scaffolds your app and creates a message extension.

To run your message extension in Teams, follow these steps:

  1. From the left pane, select Microsoft 365 Agents Toolkit.

  2. Under ACCOUNTS, perform the following steps:

    1. Select Sign in to Microsoft 365 and enter your Microsoft 365 credentials.

    2. Select Sign in to Azure and enter your Azure credentials.

      :::image type="content" source="../assets/images/Copilot/api-based-me-ttk-accounts.png" alt-text="Screenshot shows the Sign in to Microsoft 365 and Azure option under ACCOUNTS in Agents Toolkit for Visual Studio Code.":::

  3. From the left pane, select Run and Debug (Ctrl+Shift+D).

  4. Select Debug in Teams (Edge) or Debug in Teams (Chrome). Agents Toolkit launches your app in Teams using a web browser.

  5. Select Add. The app is installed on Teams.

  6. Go to a chat and select Actions and apps.

  7. From the message extension fly-out menu, enter the name of your message extension in the search box.

  8. Select your message extension and enter your search query.

  9. Select a product from the list. Teams unfurls the product as an Adaptive Card in the message compose area.

  10. Select Send.

Trigger message extension in Microsoft 365 Copilot

To trigger the message extension as agent in Microsoft 365 Copilot in Teams, follow these steps:

  1. Select Apps.

  2. Search for Copilot and open Microsoft 365 Copilot.

  3. Select Panel from the top-right corner.

  4. Select the required agent from the right pane.

    :::image type="content" source="../assets/images/messaging-extension/message-extension-agents.png" alt-text="Screenshot shows the list of agents in Microsoft 365." lightbox="../assets/images/messaging-extension/message-extension-agent.png":::

  5. From the message compose area, send a message to Microsoft 365 Copilot to search for the latest update from Teams and Microsoft 365 Copilot.

    :::image type="content" source="../assets/images/Copilot/bot-based-message-extension-agent.png" alt-text="Screenshot shows the agent prompt and the response from Microsoft 365 Copilot." lightbox="../assets/images/Copilot/bot-based-message-extension-agent.png":::

Note

This prompt might not always ensure a response from your message extension in Microsoft 365 Copilot. If this occurs, try different prompts or provide feedback by downvoting the Microsoft 365 Copilot response and leaving a message.

Visual Studio

Before you get started, ensure that you install the following tools to build and deploy your message extension:

  • Install the Visual Studio Enterprise 2022 Preview version 17.9.0 Preview 1.0 or later and install the Microsoft Teams development tools under ASP.NET and web development workload.
  • Ensure that the Copilot feature flag is enabled. To enable the feature flag, follow these steps:
    1. Open Visual Studio.
    2. Go to Tools > Options.
    3. Enter Microsoft 365 Agents Toolkit in the Search Settings search box.
    4. In the Preview Features section, select the Microsoft 365 Agents Toolkit: Develop Copilot Plugin checkbox.
  • Ensure that the multi-project launch profile is enabled. To enable the profile, follow these steps:
    1. Open Visual Studio.
    2. Go to Tools > Options.
    3. In the Preview Features section, select the Enable Multi-Project Launch Profiles checkbox.

To create a bot-based search message extension agent using Visual Studio, follow these steps:

  1. Open Visual Studio.

  2. Go to File > New > Project... or Create a new Project.

  3. Search for Teams and select Microsoft Teams App.

  4. Enter the project details. Ensure that the Place solution and project in the same directory checkbox isn't selected.

  5. Select Create.

  6. Select Custom Search Results.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-new-project.png" alt-text="Screenshot shows the custom search results option in Visual Studio.":::

  7. Select Create. The project is scaffolded.

  8. In the debug dropdown menu, select Dev Tunnels > Create a Tunnel.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-dev-tunnel.png" alt-text="Screenshot shows the create a tunnel option in Visual Studio.":::

    1. Select the account to create the tunnel. Azure, Microsoft Account (MSA), and GitHub are the account types that are supported.
    2. Name: Enter a name for the tunnel.
    3. Tunnel Type: Select Persistent or Temporary.
    4. Access: Select Public.
    5. Select OK. Visual Studio displays a confirmation message that a tunnel is created.

    The tunnel you created is listed under Dev Tunnels > (name of the tunnel).

  9. Go to Solution Explorer and select your project.

  10. Right-click the project menu and select Microsoft 365 Agents Toolkit > Select Microsoft 365 Account.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-teams-app-dependencies.png" alt-text="Screenshot shows the Select Microsoft 365 Account option under Agents Toolkit in Visual Studio app project.":::

    If prompted, sign in with a Microsoft 365 account.

  11. In the debug dropdown menu, select Microsoft Teams (browser). Visual Studio launches Teams web client.

  12. Select Add. The message extension is added to Teams.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-debug-teams-add-app.png" alt-text="Screenshot shows an example of the Add option to add Contoso agent local app to Teams.":::

  13. Go to a chat and select Actions and apps.

  14. From the message extension fly-out menu, enter the name of your message extension in the search box.

  15. Select your message extension and enter your search query.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-debug-teams-invoke.png" alt-text="Screenshot shows an example of Contoso search query entered in the Contoso agent local message extension and the message extension displays seven products results.":::

  16. Select a product from the list. Teams unfurls the product into an Adaptive Card in the message compose area.

  17. Select Send. The Adaptive Card is sent to the chat.

    :::image type="content" source="../assets/images/Copilot/bot-based-VS-debug-teams-send-card.png" alt-text="Screenshot shows Contoso. Base product sent as an Adaptive Card in Teams chat.":::

Trigger message extension in Microsoft 365 Copilot

To trigger the message extension as agent in Microsoft 365 Copilot, follow these steps:

  1. Go to Visual Studio.

  2. In the debug dropdown, select Copilot (browser). Visual Studio launches Teams web client.

    :::image type="content" source="../assets/images/Copilot/bot-based-VSC-debug-Copilot.png" alt-text="Screenshot shows the Copilot (Browser) debug option in Visual Studio.":::

  3. Go to Teams web client > Apps.

  4. Search for Copilot and open Microsoft 365 Copilot.

  5. Select Panel from the top-right corner.

  6. Select the required agent from the right pane.

    :::image type="content" source="../assets/images/messaging-extension/message-extension-agents.png" alt-text="Screenshot shows the list of agents in Microsoft 365." lightbox="../assets/images/messaging-extension/message-extension-agent.png":::

  7. From the message compose area, send a message to Microsoft 365 Copilot to search for the latest update from Teams and Microsoft 365 Copilot.

    :::image type="content" source="../assets/images/Copilot/bot-based-message-extension-agent.png" alt-text="Screenshot shows the agent prompt and the response from Microsoft 365 Copilot." lightbox="../assets/images/Copilot/bot-based-message-extension-agent.png":::

Note

This prompt might not always ensure a response from your message extension in Microsoft 365 Copilot. If this occurs, try different prompts or provide feedback by downvoting the Microsoft 365 Copilot response and leaving a message.

Toolkit CLI

Before you get started, ensure that you set the environment variable DEVELOP_COPILOT_PLUGIN to true. To set the environment variables, follow these steps:

  1. On your computer, select the Windows + X key.

  2. Select System.

  3. Under About > Device specifications, select Advanced system settings.

  4. Under System Properties, select Environment Variables...

  5. Under User variables, select New... A New User Variable dialog opens.

  6. Update the following fields:

    1. Variable name: DEVELOP_COPILOT_PLUGIN
    2. Variable value: true

  7. Select OK.

    :::image type="content" source="../assets/images/Copilot/bot-based-plugin-CLI-add-env-variable.png" alt-text="Screenshot shows the Variable name and Variable value fields in the New User Variable dialog.":::

To create a bot-based search message extension agent using Agents Toolkit CLI, follow these steps:

  1. Go to Command Prompt.

  2. Enter the following command:

    npm install -g @microsoft/atk-cli

  3. Enter atk new in the terminal.

  4. Select Message Extension. Use the arrow keys to switch between options.

    :::image type="content" source="../assets/images/Copilot/api-based-me-CLI-new-project-me.png" alt-text="Screenshot shows Teams capabilities as options in the CLI interface.":::

  5. Select Custom Search Results.

  6. Select Start with a Bot.

    :::image type="content" source="../assets/images/Copilot/bot-based-CLI-new-project.png" alt-text="Screenshot shows the message extension, custom search results, and start with a bot option selected in the CLI window.":::

  7. Enter the location for your project and select Enter.

  8. Enter the name of your application and select Enter.

    Agents Toolkit scaffolds and creates a project in the specified location.

  9. Go to the folder path where your project is created and enter the following command to provision your app in Azure:

     teamsapp provision --env dev

    Agents Toolkit CLI opens a browser window and requests you to sign in to your Microsoft Account.

  10. Sign in to your Microsoft account. Agents Toolkit CLI executes validation and provisions your app on Azure.

  11. From the list, select a subscription.

    :::image type="content" source="../assets/images/Copilot/bot-based-CLI-resource-group.png" alt-text="Screenshot shows the sign-in to Azure and select a subscription option in the CLI window.":::

  12. From the list, select a resource group.

  13. If you receive a message, which reads Cost may incur according to the usage. Do you want to provision resources in dev environment using accounts listed above?, enter Y.

    :::image type="content" source="../assets/images/Copilot/bot-based-CLI-provisoin-preview.png" alt-text="Screenshot shows the Cost may incur according to the usage. Do you want to provision resources in dev environment using accounts listed option in the CLI window.":::

    Agents Toolkit validates your app manifest and provisions your app on Azure.

  14. In the command prompt window, enter the following command to preview your app in Teams:

     teamsapp preview --env dev

    A new browser window with Teams web client opens. You can add your app to Teams.

  15. Select Add. The message extension is added to Teams.

  16. Go to a chat and select Actions and apps.

  17. From the message extension fly-out menu, enter the name of your message extension in the search box.

  18. Select your message extension and enter your search query.

Trigger message extension in Microsoft 365 Copilot

To trigger the message extension as agent in Microsoft 365 Copilot in Teams, follow these steps:

  1. Select Apps.

  2. Search for Copilot and open Microsoft 365 Copilot.

  3. Select Panel from the top-right corner.

  4. Select the required agent from the right pane.

    :::image type="content" source="../assets/images/messaging-extension/message-extension-agents.png" alt-text="Screenshot shows the list of agents in Microsoft 365." lightbox="../assets/images/messaging-extension/message-extension-agent.png":::

  5. From the message compose area, send a message to Microsoft 365 Copilot to search for the latest update from Teams and Microsoft 365 Copilot.

    :::image type="content" source="../assets/images/Copilot/bot-based-message-extension-agent.png" alt-text="Screenshot shows the agent prompt and the response from Microsoft 365 Copilot." lightbox="../assets/images/Copilot/bot-based-message-extension-agent.png":::

Note

This prompt might not always ensure a response from your message extension in Microsoft 365 Copilot. If this occurs, try different prompts or provide feedback by downvoting the Microsoft 365 Copilot response and leaving a message.

Developer Portal for Teams

To create a bot-based message extension using Developer Portal for Teams, follow these steps:

  1. Go to Developer Portal for Teams.

  2. Go to Apps.

  3. Select + New apps.

  4. In the Add apps page, update the name and select the manifest version as Public developer preview (devPreview).

    :::image type="content" source="../assets/images/Copilot/api-based-me-tdp-app-feature.png" alt-text="Screenshot shows the messaging extension option in Teams Developer Portal.":::

  5. Under Configure, select App features.

  6. Select Messaging extension.

  7. Under Message extension type, select Bot.

    1. If you get a disclaimer, which reads API Message extension is already in use by users. Would you like to change message extension type to bot?. Select Yes, change.

      :::image type="content" source="../assets/images/Copilot/bot-based-me-tdp-type.png" alt-text="Screenshot shows API Message extension is already in use disclaimer when a user switches from API to bot message extension type.":::

  8. If you have an existing bot, select Existing bot or if you have a bot ID, select Enter Bot ID.

    1. If you don't have a Bot ID, you can select Create a new bot, to create a new bot and enter the bot ID of the new bot that you've created.

      :::image type="content" source="../assets/images/Copilot/bot-based-tdp-select-bot.png" alt-text="Screenshot shows the Bot, Existing bot, and Enter Bot ID options for messaging extension app feature in Developer Portal for Teams.":::

  9. Select Save.

  10. Under Command, select + Add a command.

    Add a command page appears.

  11. In the Add a command page, select the Search as the type of command and update the following fields:

    • Command ID
    • Command title
    • Command description
    • Context in which the command works
    • Parameter name
    • Parameter title
    • Parameter description

  12. Select Save. The command is added to the message extension.

  13. Select Save.

A bot-based search message extension is created.

:::image type="content" source="../assets/images/Copilot/bot-based-tdp-message-extension-created.png" alt-text="Screenshot shows the messaging extension created and listed in the App features page in Developer Portal for Teams.":::

To test your bot-based message extension created in the Developer Portal for Teams, you can use the following methods:

  • Preview in Teams: In Developer Portal, open your message extension and select Preview in Teams in the upper-right corner. You're redirected to Teams, where you can add the app to Teams to preview the app.

  • Download app package: On the message extension page, select App package from the left pane and then, in the upper-left corner of the window, select Download app package. The app package is downloaded to your local machine in a .zip file. You can upload the app package to teams and test the message extension.

For more information on testing your agent in Microsoft 365 Copilot, see debugging agent selection.

See also