Merge pull request #49482 from MicrosoftDocs/NEW-agents-custom-tools

v-dirichards · web-flow · commit 7ceb3a238345 · 2025-03-12T09:48:39.000-05:00
New agents custom tools
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/1-introduction.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/1-introduction.yml
@@ -0,0 +1,13 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.introduction
+title: Introduction
+metadata:
+  title: Introduction
+  description: "Introduction to building an agent with custom tools."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 2
+content: |
+  [!include[](includes/1-introduction.md)]
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/2-why-use-custom-tools.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/2-why-use-custom-tools.yml
@@ -0,0 +1,13 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.why-use-custom-tools
+title: Why use custom tools
+metadata:
+  title: Why use custom tools
+  description: "Understand the benefits of using custom tools with your agent."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 3
+content: |
+  [!include[](includes/2-why-use-custom-tools.md)]
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/3-custom-tool-options.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/3-custom-tool-options.yml
@@ -0,0 +1,13 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.custom-tool-options
+title: Options for implementing custom tools
+metadata:
+  title: Options for implementing custom tools
+  description: "Explore the options for implementing custom tools in your agent."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 6
+content: |
+  [!include[](includes/3-custom-tool-options.md)]
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/4-how-use-custom-tools.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/4-how-use-custom-tools.yml
@@ -0,0 +1,13 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.how-use-custom-tools
+title: How to integrate custom tools
+metadata:
+  title: How to integrate custom tools
+  description: "Learn how to integrate custom tools into your agent. This unit covers function calling, Azure Function integration, and OpenAPI specification integration."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 7
+content: |
+  [!include[](includes/4-how-use-custom-tools.md)]
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/5-exercise.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/5-exercise.yml
@@ -0,0 +1,13 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.exercise
+title: Exercise - Build an agent with custom tools
+metadata:
+  title: Exercise - Build an agent with custom tools
+  description: "Get interactive experience with building an agent with custom tools."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 30
+content: |
+  [!include[](includes/5-exercise.md)]
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/6-knowledge-check.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/6-knowledge-check.yml
@@ -0,0 +1,48 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.knowledge-check
+title: Knowledge check
+metadata:
+  title: Knowledge check
+  description: "Check your knowledge of building an agent with custom tools."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 3
+content: |
+quiz:
+  questions:
+  - content: "What is one of the primary benefits of using custom tools in Azure AI Agent Service?"  
+    choices:  
+    - content: "They eliminate the need for any coding knowledge."  
+      isCorrect: false  
+      explanation: "Incorrect. While custom tools simplify many processes, some level of coding knowledge may still be required for advanced configurations."  
+    - content: "They enhance productivity by automating repetitive tasks."  
+      isCorrect: true  
+      explanation: "Correct. Custom tools in Azure AI services can automate repetitive tasks, saving time and resources, which enhance productivity."  
+    - content: "They provide free access to all Azure AI services."  
+      isCorrect: false  
+      explanation: "Incorrect. While some Azure AI services offer a free tier, not all services are free, and custom tools don't inherently provide free access."
+  - content: "A company wants to integrate its AI agents with existing patient management systems securely that they have a standard specification for already. Which tool should they use?"  
+    choices:  
+    - content: "Custom question answering"  
+      isCorrect: false  
+      explanation: "Incorrect. Custom question answering is designed for creating FAQ bots, not for integrating AI agents with existing systems."  
+    - content: "Azure AI Search tool"  
+      isCorrect: false  
+      explanation: "Incorrect. The Azure AI Search tool is used for performing searches on large datasets, not for system integration."  
+    - content: "OpenAPI specified tools"  
+      isCorrect: true  
+      explanation: "Correct. OpenAPI specified tools enable scalable interoperability with various applications, making them suitable for integrating AI agents with existing systems securely."  
+  - content: "What is a key advantage of using custom tools in Azure AI Agent Service?"  
+    choices:  
+    - content: "They provide standardized, automated, and scalable integrations."  
+      isCorrect: true  
+      explanation: "Correct. Custom tools in Azure AI services offer standardized, automated, and scalable integrations, improving the function calling experience and overall efficiency."  
+    - content: "They eliminate the need for any human intervention."  
+      isCorrect: false  
+      explanation: "Incorrect. While custom tools automate many processes, some level of human intervention may still be required for configuration and maintenance."  
+    - content: "They guarantee 100% accuracy in all predictions."  
+      isCorrect: false  
+      explanation: "Incorrect. No AI tool can guarantee 100% accuracy in all predictions; however, custom tools can significantly improve accuracy by fine-tuning models with specific data."  
+  
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/7-summary.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/7-summary.yml
@@ -0,0 +1,13 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.build-agent-with-custom-tools.summary
+title: Summary
+metadata:
+  title: Summary
+  description: "Review the key points of using custom tools with your agent."
+  ms.date: 3/11/2025
+  author: ivorb
+  ms.author: berryivor
+  ms.topic: unit
+durationInMinutes: 2
+content: |
+  [!include[](includes/7-summary.md)]
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/1-introduction.md b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/1-introduction.md
@@ -0,0 +1,13 @@
+Azure AI Agent Service offers a seamless way to build an agent without needing extensive AI or machine learning expertise. Through the use of tools, you can provide your agent additional functionality to execute actions on your behalf.
+
+The AI Agent Service provides built-in tools for gathering knowledge and interpreting code, which provide your agent with some powerful functionality. However, sometimes your agent needs to be able to complete specific tasks or actions that an AI model would struggle to handle on its own. To accomplish these actions, you can provide your agent a **custom tool**.
+
+Imagine you're working in the retail industry, and your company is struggling with managing customer inquiries efficiently. The customer support team is overwhelmed with repetitive questions, leading to delays in response times and decreased customer satisfaction. By leveraging Azure AI Agent Service with custom tools, you can create a custom FAQ agent that handles common inquiries. This agent can be provided a set of custom tools to look up customer orders, freeing up your support team to focus on more complex issues.
+
+In this module, you'll learn how to utilize custom tools in Azure AI Agent Service to enhance productivity, improve accuracy, and create tailored solutions for specific needs.
+
+After completing this module, you’ll be able to:
+
+- Describe the benefits of using custom tools in your agent.
+- Explore the different options for custom tools.
+- Build an agent with custom tools using the Azure AI Agent Service.
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/2-why-use-custom-tools.md b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/2-why-use-custom-tools.md
@@ -0,0 +1,20 @@
+Azure AI Agent Service offers a powerful platform for integrating custom tools to enhance productivity and provide tailored solutions for specific business needs. By leveraging these custom tools, businesses can achieve greater efficiency and effectiveness in their operations.
+
+## Benefits of using custom tools in Azure AI Agent Service
+
+Custom tools in Azure AI services can significantly enhance productivity by automating repetitive tasks and streamlining workflows that are specific to your use case. These tools improve accuracy by providing precise and consistent outputs, reducing the likelihood of human error. Additionally, custom tools offer tailored solutions that address specific business needs, enabling organizations to optimize their processes and achieve better outcomes.
+
+- **Enhanced productivity**: Automate repetitive tasks and streamline workflows.
+- **Improved accuracy**: Provide precise and consistent outputs, reducing human error.
+- **Tailored solutions**: Address specific business needs and optimize processes.
+
+Adding tools to achieve custom functionality are available for the agent to decide to use, depending on how it decides to respond to the user prompt.
+
+![Diagram showing the flow of logic through an agent using a custom tool.](../media/agent-tool-diagram.png)
+
+The diagram shows the process of an agent choosing to use the provided tool:
+
+1. A user asks an agent for a report, such as about recent snowfall in their local mountains.
+1. The agent determines the provided tool to retrieve snowfall for a specific location will be useful, and calls that tool.
+1. The agent may choose to use other tools to best accomplish the user's task, such as retrieve knowledge using a built-in tool.
+1. The agent then outputs the report and responds to the user.
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/3-custom-tool-options.md b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/3-custom-tool-options.md
@@ -0,0 +1,45 @@
+Azure AI Agent Service offers a variety of custom tools that enhance the capabilities and efficiency of your AI agents. These tools allow for scalable interoperability with various applications, making it easier to integrate with existing infrastructure or web services.
+
+## Custom tool options available in Azure AI services
+
+Azure AI services provide several custom tool options, including OpenAPI specified tools, Azure Functions, and function calling. These tools enable seamless integration with external APIs, event-driven applications, and custom functions.
+
+- **OpenAPI specified tools**: These tools allow you to connect your Azure AI Agent to an external API using an OpenAPI 3.0 specification. This provides standardized, automated, and scalable API integrations that enhance the capabilities of your agent. OpenAPI specifications describe HTTP APIs, enabling people to understand how an API works, generate client code, create tests, and apply design standards.
+- **Function calling**: Function calling allows you to describe the structure of functions to an agent and return the functions that need to be called along with their arguments. This feature is useful for integrating custom logic and workflows into your AI agents.
+- **Azure Functions**: Azure Functions enable you to create intelligent, event-driven applications with minimal overhead. They support triggers and bindings, which simplify how your AI Agents interact with external systems and services. Triggers determine when a function executes, while bindings facilitate streamlined connections to input or output data sources.
+
+## Example use of custom tools with the Azure AI Agent Service
+
+Custom tools within the Azure AI Agent Service enable users to extend the capabilities of AI agents, tailoring them to meet specific business needs. Some example use cases that illustrate the versatility and impact of custom tools include:
+
+### Customer support automation
+
+- **Scenario**: A retail company integrates a custom tool that connects the Azure AI Agent to their customer relationship management (CRM) system.
+- **Functionality**: The AI agent can retrieve customer order histories, process refunds, and provide real-time updates on shipping statuses.
+- **Outcome**: Faster resolution of customer queries, reduced workload for support teams, and improved customer satisfaction.
+
+### Inventory management
+
+- **Scenario**: A manufacturing company develops a custom tool to link the AI agent with their inventory management system.
+- **Functionality**: The AI agent can check stock levels, predict restocking needs using historical data, and place orders with suppliers automatically.
+- **Outcome**: Streamlined inventory processes and optimized supply chain operations.
+
+### Healthcare appointment scheduling
+
+- **Scenario**: A healthcare provider integrates a custom scheduling tool with the AI agent.
+- **Functionality**: The AI agent can access patient records, suggest available appointment slots, and send reminders to patients.
+- **Outcome**: Reduced administrative burden, improved patient experience, and better resource utilization.
+
+### IT Helpdesk support
+
+- **Scenario**: An IT department develops a custom tool to integrate the AI agent with their ticketing and knowledge base systems.
+- **Functionality**: The AI agent can troubleshoot common technical issues, escalate complex problems, and track ticket statuses.
+- **Outcome**: Faster issue resolution, reduced downtime, and improved employee productivity.
+
+### E-learning and training
+
+- **Scenario**: An educational institution creates a custom tool to connect the AI agent with their learning management system (LMS).
+- **Functionality**: The AI agent can recommend courses, track student progress, and answer questions about course content.
+- **Outcome**: Enhanced learning experiences, increased student engagement, and streamlined administrative tasks.
+
+These examples demonstrate how custom tools within the Azure AI Agent Service can be used across industries to address unique challenges, drive efficiency, and deliver value.
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/4-how-use-custom-tools.md b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/4-how-use-custom-tools.md
@@ -0,0 +1,166 @@
+Custom tools in an agent can be defined in a handful of ways, depending on what works best for your scenario. You may find that your company already has Azure Functions implemented for your agent to use, or a public OpenAPI specification gives your agent the functionality you're looking for.
+
+## Agents function calling
+
+Function calling allows agents to execute predefined functions dynamically based on user input. This feature is ideal for scenarios where agents need to perform specific tasks, such as retrieving data or processing user queries, and can be done in code from within the agent. Your function may call out to other APIs to get additional information or initiate a program.
+
+### Example: Defining and using a function
+
+Start by defining a function that the agent can call. For instance, here's a fake snowfall tracking function:
+
+```python
+import json
+
+def recent_snowfall(location: str) -> str:
+    """
+    Fetches recent snowfall totals for a given location.
+    :param location: The city name.
+    :return: Snowfall details as a JSON string.
+    """
+    mock_snow_data = {"Seattle": "0 inches", "Denver": "2 inches"}
+    snow = mock_snow_data.get(location, "Data not available.")
+    return json.dumps({"location": location, "snowfall": snow})
+
+user_functions: Set[Callable[..., Any]] = {
+    recent_snowfall,
+}
+```
+
+Register the function with your agent using the Azure AI SDK:
+
+```python
+# Initialize agent toolset with user functions
+functions = FunctionTool(user_functions)
+toolset = ToolSet()
+toolset.add(functions)
+
+# Create your agent with the toolset
+agent = project_client.agents.create_agent(
+    model="gpt-4o-mini",
+    name="snowfall-agent",
+    instructions="You are a weather assistant tracking snowfall. Use the provided functions to answer questions.",
+    toolset={"functions": [recent_snowfall]}
+)
+```
+
+The agent can now call `recent_snowfall` dynamically when prompted by the user.
+
+## OpenAPI defined tools
+
+OpenAPI defined tools allow agents to interact with external APIs using standardized specifications. This approach simplifies API integration and ensures compatibility with various services. Azure AI Agent Service uses OpenAPI 3.0 specified tools.
+
+> [!TIP]
+> Currently, three authentication types are supported with OpenAPI 3.0 tools: `anonymous`, `API key`, and `managed identity`.
+
+### Example: Using an OpenAPI specification
+
+1. **Prepare the OpenAPI spec**: Create a JSON file (`snowfall_openapi.json`) describing the API.
+
+    ```json
+    {
+      "openapi": "3.0.0",
+      "info": {
+        "title": "Snowfall API",
+        "version": "1.0.0"
+      },
+      "paths": {
+        "/snow": {
+          "get": {
+            "summary": "Get snowfall information",
+            "parameters": [
+              {
+                "name": "location",
+                "in": "query",
+                "required": true,
+                "schema": {
+                  "type": "string"
+                }
+              }
+            ],
+            "responses": {
+              "200": {
+                "description": "Successful response",
+                "content": {
+                  "application/json": {
+                    "schema": {
+                      "type": "object",
+                      "properties": {
+                        "location": {"type": "string"},
+                        "snow": {"type": "string"}
+                      }
+                    }
+                  }
+                }
+              }
+            }
+          }
+        }
+      }
+    }
+    ```
+
+1. **Register the OpenAPI tool**:
+
+    ```python
+    from azure.ai.projects.models import OpenApiTool, OpenApiAnonymousAuthDetails
+    
+    with open("snowfall_openapi.json", "r") as f:
+        openapi_spec = json.load(f)
+    
+    auth = OpenApiAnonymousAuthDetails()
+    openapi_tool = OpenApiTool(name="snowfall_api", spec=openapi_spec, auth=auth)
+    
+    agent = project_client.agents.create_agent(
+        model="gpt-4o-mini",
+        name="openapi-agent",
+        instructions="You are a snowfall tracking assistant. Use the API to fetch snowfall data.",
+        tools=[openapi_tool]
+    )
+    ```
+
+The agent can now use the OpenAPI tool to fetch snowfall data dynamically.
+
+## Azure Functions integration
+
+Azure Functions provide serverless computing capabilities for real-time processing. This integration is ideal for event-driven workflows, enabling agents to respond to triggers such as HTTP requests or queue messages.
+
+### Example: Using Azure Functions with a queue trigger
+
+1. **Define an Azure Function**: Develop and deploy your Azure Function. In this example, imagine we have a function in our Azure subscription to fetch the snowfall for a given location.
+
+1. **Integrate the Azure Function with the agent**:
+
+    ```python
+    storage_service_endpoint = "https://<your-storage>.queue.core.windows.net"
+    
+    azure_function_tool = AzureFunctionTool(
+        name="get_snowfall",
+        description="Get snowfall information using Azure Function",
+        parameters={
+                "type": "object",
+                "properties": {
+                    "location": {"type": "string", "description": "The location to check snowfall."},
+                },
+                "required": ["location"],
+            },
+        input_queue=AzureFunctionStorageQueue(
+            queue_name="input",
+            storage_service_endpoint=storage_service_endpoint,
+        ),
+        output_queue=AzureFunctionStorageQueue(
+            queue_name="output",
+            storage_service_endpoint=storage_service_endpoint,
+        ),
+    )
+    
+    agent = project_client.agents.create_agent(
+        model=os.environ["MODEL_DEPLOYMENT_NAME"],
+        name="azure-function-agent",
+        instructions="You are a snowfall tracking agent. Use the provided Azure Function to fetch snowfall based on location.",
+        tools=azure_function_tool.definitions,
+    )
+    ```
+
+The agent can now send requests to the Azure Function via a storage queue and process the results.
+
+By using one of the above methods (or a combination of these options) for implementing a custom tool, you can create powerful, flexible, and intelligent agents with Azure AI Agent Service. These integrations enable seamless interaction with external systems, real-time processing, and scalable workflows, making it easier to build custom solutions tailored to your needs.
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/5-exercise.md b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/5-exercise.md
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/7-summary.md b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/includes/7-summary.md
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/index.yml b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/index.yml
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/media/agent-tool-diagram.png b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/media/agent-tool-diagram.png
diff --git a/learn-pr/wwl-data-ai/build-agent-with-custom-tools/media/launch-exercise.png b/learn-pr/wwl-data-ai/build-agent-with-custom-tools/media/launch-exercise.png
