Skip to content

Commit 2fe3283

Browse files
authored
Update prompt builder documentation to agent builder (#8297)
* Update prompt builder documentation to agent builder * docs: update the mcp debug gif * docs: update agent builder documentation for clarity and consistency * docs: update the MCP server section * update per comments
1 parent 9d438e7 commit 2fe3283

15 files changed

+199
-80
lines changed

docs/intelligentapps/agentbuilder.md

Lines changed: 162 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,162 @@
1+
---
2+
ContentId: bd3d7555-3d84-4500-ae95-6dcd39641af0
3+
DateApproved: 04/22/2025
4+
MetaDescription: Get Started with creating, iterating and optimizing your agents in AI Toolkit.
5+
---
6+
# Build agents and prompts in AI Toolkit
7+
8+
> [!NOTE]
9+
> Agent Builder was previously known as Prompt Builder. The name has been changed to better reflect the feature's capabilities and its focus on building agents.
10+
11+
Agent Builder in AI Toolkit streamlines the engineering workflow for building agents, including prompt engineering and integration with tools, such as MCP servers. It helps with common prompt engineering tasks:
12+
- Generate starter prompts
13+
- Iterate and refine with each run
14+
- Break down complex tasks through prompt chaining and structured outputs
15+
- Provide easy access to code for seamless Large Language Model (LLM) integration via APIs
16+
17+
Agent Builder also enhances intelligent app's capabilities with tool use:
18+
- Connect to existing MCP servers
19+
- Build a new MCP server from scaffold and test in Agent Builder
20+
21+
![Getting started with prompt builder](./images/promptbuilder/AgentBuilder.gif)
22+
23+
## Create, edit, and test prompts
24+
25+
To access Agent Builder, use either of these options:
26+
27+
- In the AI Toolkit view, select **Agent (Prompt) Builder**
28+
- Select **Try in Agent (Prompt) Builder** from a model card in the model catalog
29+
30+
To test a prompt in Agent Builder, follow these steps:
31+
32+
1. In **Models**, select a model from the dropdown list, or select **Browse models** to add another model from the model catalog.
33+
34+
![select a model](./images/promptbuilder/select_models.png)
35+
36+
1. Enter a **User prompt** and optionally enter a **System prompt**.
37+
38+
The *user prompt* is the input that you want to send to the model. The optional *system prompt* is used to provide instructions with relevant context to guide the model response.
39+
40+
> [!TIP]
41+
> You can describe your project idea by using natural language and let the AI-powered feature generate the prompts for you to experiment with.
42+
> ![generate prompts with natural language](./images/promptbuilder/generate_prompt_2.gif)
43+
44+
1. Select **Run** to send the prompts to the selected model.
45+
46+
1. Optionally, select **Add Prompt** to add more user and assistant prompts to the conversation, or select **Add to Prompts** as the history and context you send to the model to further guide the model's behavior.
47+
48+
1. Repeat the previous steps to iterate over your prompts by observing the model response and making changes to the prompts.
49+
50+
## Use MCP servers
51+
MCP server is a tool that allows you to connect to external APIs and services, enabling your agent to perform actions beyond just generating text. For example, you can use an MCP server to access databases, call web services, or interact with other applications.
52+
53+
You can use the agent builder to discover and configure featured MCP servers, connect to existing MCP servers or build a new MCP server from scaffold.
54+
55+
> [!NOTE]
56+
> Using MCP servers may require either [Node](https://nodejs.org/en/download) or [Python](https://www.python.org/downloads/) environment. AI Toolkit will validate your environment to ensure that the required dependencies are installed.
57+
> After installing, please use the command `npm install -g npx` to install `npx`. If you prefer Python, we recommend using [`uv`](https://docs.astral.sh/uv/getting-started/installation/)
58+
59+
### Configure a featured MCP server
60+
AI Toolkit provides a list of featured MCP servers that you can use to connect to external APIs and services.
61+
62+
To configure an MCP server from featured selections, follow these steps:
63+
1. In the **Tools** section, select **+ MCP Server**, and then select **+ Add server** in the Quick Pick.
64+
2. Select **Use Featured MCP Servers** from the dropdown list.
65+
3. Choose an MCP server that meets your needs.
66+
![connect to MCP server](./images/promptbuilder/Featured_server.png)
67+
68+
For servers that are ready to use, AI Toolkit will prompt you to select tools and add to your agent afterwards.
69+
70+
For servers that are marked with **(Configuration Required)**:
71+
1. You will be prompted to enter the required configurations in the `mcp.json` file.
72+
![config_server](./images/promptbuilder/config_server.png)
73+
2. Follow the guidelines to configure the server:
74+
1. [Filesystem](https://github.com/modelcontextprotocol/servers/tree/main/src/filesystem#npx)
75+
2. [GitHub](https://github.com/modelcontextprotocol/servers/tree/main/src/github#npx)
76+
3. [Google Maps](https://github.com/modelcontextprotocol/servers/tree/main/src/google-maps#npx)
77+
4. [Sqlite](https://github.com/modelcontextprotocol/servers/tree/main/src/sqlite#uv-1)
78+
3. After finishing the configuration, go back to Agent Builder and select **+ MCP Server** to locate the server you added from the list.
79+
4. Select the server and proceed to add tools.
80+
81+
### Use an existing MCP server
82+
To use an existing MCP server, follow these steps:
83+
1. In the **Tools** section, select **+ MCP Server**, and then select **+ Add server** in the quick pick.
84+
2. Select **Connect to an Existing MCP Server**
85+
3. Select an option from the dropdown list to specify how you want to connect to the MCP server:
86+
- **Command (stdio)**: Run a local command that implements the MCP protocol
87+
- **HTTP (server-sent events)**: Connect to a remote server that implements the MCP protocol
88+
4. Select tools from the MCP server if there are multiple tools available.
89+
5. Enter your prompts and select **Run** to test the connection.
90+
91+
![connect to MCP server](./images/promptbuilder/mcp_existing.gif)
92+
93+
AI Toolkit also provides a scaffold to help you build a new MCP server. The scaffold includes a basic implementation of the MCP protocol, which you can customize to suit your needs.
94+
95+
### Build a new MCP server
96+
To build a new MCP server, follow these steps:
97+
1. In the **Tools** section, select **+ MCP Server**, and then select **+ Add server** in the quick pick.
98+
2. Select **Create a New MCP Server**
99+
3. Select a programming language from the dropdown list: **Python** or **TypeScript**
100+
4. Select a folder to create the new MCP server project in.
101+
5. Enter a name for the MCP server project.
102+
103+
![scaffold mcp](./images/promptbuilder/scaffold_mcp.gif)
104+
105+
After you create the MCP server project, you can customize the implementation to suit your needs. The scaffold includes a basic implementation of the MCP protocol, which you can modify to add your own functionality.
106+
107+
You can also use the agent builder to test the MCP server. The agent builder will send the prompts to the MCP server and display the response.
108+
109+
Follow these steps to test the MCP server:
110+
111+
> [!NOTE]
112+
> To run the MCP Server in your local dev machine, you will need: [Node.js](https://nodejs.org/) or Python installed on your machine.
113+
114+
1. Open VS Code Debug panel. Select `Debug in Agent Builder` or press `F5` to start debugging the MCP server.
115+
2. Use AI Toolkit Agent Builder to test the server with the following prompt:
116+
1. System Prompt: You are a weather forecast professional that can tell weather information based on given location.
117+
3. The server will be automatically connected to Agent Builder.
118+
4. Select `Run` to test the server with the prompt.
119+
120+
![debug mcp](./images/promptbuilder/mcp_debug.gif)
121+
122+
## Structured output
123+
124+
Structured output support helps you design prompts to deliver outputs in a structured, predictable format.
125+
126+
![Use structured output](./images/promptbuilder/structured_output_2.gif)
127+
128+
To test using structured output in Agent Builder, follow these steps:
129+
130+
1. Select the **Structure output** from the left area, and select **json_schema**.
131+
132+
2. Select **Prepare schema**, and then select **Select local file** to use your own schema, or select **Use an example** to use a predefined schema.
133+
134+
If you proceed with an example, you can select a schema from the dropdown list.
135+
136+
3. Select **Run** to send the prompts to the selected model.
137+
138+
4. You can also edit the schema by selecting name of the schema.
139+
140+
![edit schema](./images/promptbuilder/edit_schema_2.png)
141+
142+
## Integrate prompt engineering into your application
143+
144+
After experimenting with models and prompts, you can get into coding right away with the automatically generated Python code.
145+
146+
![view code](./images/promptbuilder/view_code_2.gif)
147+
148+
To view the Python code, follow these steps:
149+
150+
1. Select **View Code**.
151+
152+
1. For models hosted on GitHub, select the inference SDK you want to use.
153+
154+
AI Toolkit generates the code for the model you selected by using the provider's client SDK. For models hosted by GitHub, you can choose which inference SDK you want to use: [Azure AI Inference SDK](https://learn.microsoft.com/python/api/overview/azure/ai-inference-readme?view=azure-python-preview) or the SDK from the model provider, such as [OpenAI SDK](https://platform.openai.com/docs/libraries) or [Mistral API](https://docs.mistral.ai/api).
155+
156+
1. The generated code snippet is shown in a new editor, where you can copy it into your application.
157+
158+
> To authenticate with the model, you usually need an API key from the provider. To access models hosted by GitHub, [generate a personal access token](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens) (PAT) in your GitHub settings.
159+
160+
## Next steps
161+
162+
- [Run an evaluation job](/docs/intelligentapps/evaluation.md) for the popular evaluators
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading
Lines changed: 3 additions & 0 deletions
Loading

0 commit comments

Comments
 (0)