Skip to content

Commit f11e311

Browse files
committed
feedback
1 parent a2687f0 commit f11e311

File tree

1 file changed

+13
-12
lines changed

1 file changed

+13
-12
lines changed

articles/ai-foundry/agents/how-to/tools/browser-automation.md

Lines changed: 13 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
---
2-
title: 'How to use browser automation in Azure AI Foundry Agent Service'
2+
title: 'How to use Browser Automation in Azure AI Foundry Agent Service'
33
titleSuffix: Azure AI Foundry
44
description: Learn how to automate browser tasks using AI agents.
55
services: cognitive-services
@@ -12,30 +12,31 @@ ms.author: aahi
1212
ms.custom: azure-ai-agents
1313
---
1414

15-
# Browser automation (preview)
15+
# Browser Automation (preview)
1616

1717
> [!WARNING]
18-
> The browser automation tool comes with significant security risks. Both errors in judgment by the AI and the presence of malicious or confusing instructions on web pages which the AI encounters may cause it to execute commands you or others do not intend, which could compromise the security of your or other users' browsers, computers, and any accounts to which the browser or AI has access, including personal, financial, or enterprise systems. By using the browser automation tool, you are acknowledging that you bear responsibility and liability for any use of it and of any resulting agents you create with it, including with respect to any other users to whom you make Browser Automation Tool functionality available. 
18+
> The browser automation tool comes with significant security risks. Both errors in judgment by the AI and the presence of malicious or confusing instructions on web pages which the AI encounters may cause it to execute commands you or others do not intend, which could compromise the security of your or other users' browsers, computers, and any accounts to which the browser or AI has access, including personal, financial, or enterprise systems. By using the Browser Automation Tool, you are acknowledging that you bear responsibility and liability for any use of it and of any resulting agents you create with it, including with respect to any other users to whom you make Browser Automation tool functionality available, including through resulting agents. We strongly recommend using the Browser Automation Tool on low-privilege virtual machines with no access to sensitive data or critical resources.
1919
20-
Browser automation tool enables users to perform real-world browser tasks through natural language prompts. Powered by [Microsoft Playwright Workspaces](/azure/playwright-testing/overview-what-is-microsoft-playwright-testing), it facilitates multi-turn conversations to automate browser-based workflows such as searching, navigating, filling forms, and booking.
20+
21+
The Browser Automation tool enables users to perform real-world browser tasks through natural language prompts. Powered by [Microsoft Playwright Workspaces](/azure/playwright-testing/overview-what-is-microsoft-playwright-testing), it facilitates multi-turn conversations to automate browser-based workflows such as searching, navigating, filling forms, and booking.
2122

2223
## How it works
2324

24-
The interaction begins when the user sends a user query to an agent connected to the browser automation tool. For example, *"Show me all available yoga classes this week from the following url \<url\>".* Upon receiving the request, Azure AI Foundry Agent Service creates an isolated browser session using your own provisioned Playwright workspace. Each session is sandboxed for privacy and security. The browser session mimics a real user browsing experience, enabling interaction with complex web UIs (for example, class schedules, filters, or booking pages). The browser performs Playwright-driven actions, such as navigating to relevent pages, and applying filters or parameters based on user preferences (such as time, location, instructor). Combining the model with Playwright allows the model to see the browser screen by parsing the HTML or XML pages into DOM documents, make decisions, and perform actions like clicking, typing, and navigating websites. You should exercise caution when using this tool.
25+
The interaction begins when the user sends a user query to an agent connected to the Browser Automation tool. For example, *"Show me all available yoga classes this week from the following url \<url\>."* Upon receiving the request, Azure AI Foundry Agent Service creates an isolated browser session using your own provisioned Playwright workspace. Each session is sandboxed for privacy and security. The browser session mimics a real user browsing experience, enabling interaction with complex web UIs (for example, class schedules, filters, or booking pages). The browser performs Playwright-driven actions, such as navigating to relevant pages, and applying filters or parameters based on user preferences (such as time, location, instructor). Combining the model with Playwright allows the model to see the browser screen by parsing the HTML or XML pages into DOM documents, make decisions, and perform actions like clicking, typing, and navigating websites. You should exercise caution when using this tool.
2526

2627
An example flow would be:
2728

28-
1. A user sends a request to the model that includes a call to the browser automation tool with the URL you want to go to.
29+
1. A user sends a request to the model that includes a call to the Browser Automation tool with the URL you want to go to.
2930

30-
1. The tool receives a response from the model. If the response has actions for it to take, those steps contain suggested actions to make progress toward the specified goal. For example an action might be a screenshot so the model can assess the current state with an updated screenshot or click with X/Y coordinates indicating where the mouse should be moved.
31+
1. The Browser Automation Tool receives a response from the model. If the response has action items, those items contain suggested actions to make progress toward the specified goal. For example an action might be a screenshot so the model can assess the current state with an updated screenshot or click with X/Y coordinates indicating where the mouse should be moved.
3132

32-
1. The tool executes the action in a sandboxed environment.
33+
1. The Browser Automation Tool executes the action in a sandboxed environment.
3334

34-
1. After executing the action, the tool captures the updated state of the environment as a screenshot.
35+
1. After executing the action, The Browser Automation Tool captures the updated state of the environment as a screenshot.
3536

3637
1. The tool sends a new request with the updated state, and repeats this loop until the model stops requesting actions or the user decides to stop.
3738

38-
The browser automation tool supports multi-turn conversations, allowing the user to refine their request and complete a task, such as booking a flight.
39+
The Browser Automation Tool supports multi-turn conversations, allowing the user to refine their request and complete a booking.
3940

4041
## Example scenarios:
4142

@@ -66,10 +67,10 @@ The browser automation tool supports multi-turn conversations, allowing the user
6667

6768
For more information on creating a connection, see [Create a connection](../../../how-to/connections-add.md?pivots=fdp-project).
6869

69-
1. Create a browser automation tool with your connection ID.
70+
1. Create a Browser Automation tool with your connection ID.
7071

7172
## Transparency note
7273

73-
Review the [transparency note](/azure/ai-foundry/responsible-ai/agents/transparency-note#enabling-autonomous-actions-with-or-without-human-input-through-action-tools) when using this tool. The browser automation tool is a tool that can perform real-world browser tasks through natural language prompts, enabling automated browsing activities without human intervention.
74+
Review the [transparency note](/azure/ai-foundry/responsible-ai/agents/transparency-note#enabling-autonomous-actions-with-or-without-human-input-through-action-tools) when using this tool. The Browser Automation tool is a tool that can perform real-world browser tasks through natural language prompts, enabling automated browsing activities without human intervention.
7475

7576
Review the [responsible AI considerations](/azure/ai-foundry/responsible-ai/agents/transparency-note#considerations-when-choosing-a-use-case) when using this tool.

0 commit comments

Comments
 (0)