Skip to content

Commit 98bd258

Browse files
sophia-ramseysophia-ramsey
authored
create agent in gunicorn if it does not exist (#15)
* in `gunicorn.conf.py`, call `list_agents()` to see if there is an agent called `agent-template-assistant`. If there is not, it will create one * in `main.py`, call `list_agents()` to fetch agent called `agent-template-assistant`. If it is not found, there is an error * in `main.py`, do not delete the agent/resources that are used, so it will be there for the future * edit `readme` to remove import agent section. `main.py` and `gunicorn.conf.py` have functionality to use agent_id once that is enabled * set `workers = (num_cpus * 2) + 1`
1 parent a84fa0b commit 98bd258

File tree

3 files changed

+77
-84
lines changed

3 files changed

+77
-84
lines changed

‎README.md

Lines changed: 1 addition & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -40,37 +40,6 @@ The default for the model capacity in deployment is 50k tokens. For optimal perf
4040
* Select Quota, click the GlobalStandard dropdown and select the model and region you are using for this accelerator to see your available quota. Please note GPT-4o mini and text-embedding-ada-002 are used as default.
4141
* Request more quota or delete any unused model deployments as needed.
4242

43-
#### Creating and Importing an Agent
44-
45-
You can create an agent in your project to import to this template. For more information on creating an agent, view [Quickstart: Create a New Agent](https://learn.microsoft.com/azure/ai-services/agents/quickstart?pivots=ai-foundry). There are three agent creation options:
46-
47-
<details>
48-
<summary><b>Azure AI Foundry Agents Playground</b></summary>
49-
50-
In [Azure AI Foundry](https://ai.azure.com/), navigate to the Agents Playground for your project. Here, you can create an agent and customize its name, model deployment, instructions, and tools, as well as upload files for file search.
51-
</details>
52-
53-
<details>
54-
<summary><b>SDK Samples</b></summary>
55-
56-
You can create an agent using the Azure AI Projects SDK. By following the [instructions to create an agent using Python](https://learn.microsoft.com/azure/ai-services/agents/quickstart?pivots=programming-language-python-azure), you can create an agent using code based on the SDK samples.
57-
58-
</details>
59-
60-
<details>
61-
<summary><b>Assistants Tool</b></summary>
62-
63-
Using the [Azure AI Assistants Tool](https://github.com/Azure-Samples/azureai-assistant-tool), you can create an agent and customize it with tools and functions. Then, you can export the agent and use the agent_id.
64-
<!-- TODO: do we need to implement agents.yaml changes in order to use this solution? -->
65-
</details>
66-
67-
After creating your agent, import the agent into this template by copying the agent_id and setting the following environment variable:
68-
69-
```shell
70-
azd env set AZURE_AI_AGENT_ID "<agent_id>"
71-
```
72-
73-
7443
## Development
7544

7645
#### Code
@@ -83,19 +52,13 @@ At this point you could make changes to the code if required. However, no change
8352

8453
#### Configure your Agent (optional)
8554
<!-- TODO where do we want this? probably after downloading the code -->
86-
If you want to personalize your agent, you can change the default configuration for your agent. This can include changing the model, adding tools, and uploading files to the agent. For more information on the Azure OpenAI models and non-Microsoft models that can be used in your deployment, view the [list of models supported by Azure AI Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support).
55+
If you want to personalize your agent, you can change the default configuration for your agent. Additional details on changing your agent can be found in [customizing model deployments](docs/deploy_customization.md#customizing-model-deployments). For more information on the Azure OpenAI models and non-Microsoft models that can be used in your deployment, view the [list of models supported by Azure AI Agent Service](https://learn.microsoft.com/azure/ai-services/agents/concepts/model-region-support).
8756

8857
To specify the model (e.g. gpt-4o-mini, gpt-4o) that is deployed for the agent when `azd up` is called, set the following environment variables:
8958
```shell
9059
azd env set AZURE_AI_AGENT_MODEL_NAME <MODEL_NAME>
9160
azd env set AZURE_AI_AGENT_MODEL_VERSION <MODEL_VERSION>
9261
```
93-
To add tools, update the `agents.yaml` file located in the repository.
94-
```python
95-
# TODO: add this once it is configured
96-
```
97-
<!-- To add files to be used for file search, upload your files to `src/api/files`. -->
98-
<!-- TODO: do we want this functionality? we would need to slightly alter the py file -->
9962

10063
#### Logging
10164
If you want to enable logging to a file, uncomment the following line in Dockerfile located in the src directory:

‎src/api/main.py

Lines changed: 8 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -97,34 +97,14 @@ async def lifespan(app: fastapi.FastAPI):
9797
logger.error(f"Error fetching agent: {e}", exc_info=True)
9898
create_new_agent = True
9999
if create_new_agent:
100-
logger.info("Creating agent")
101-
file_names = ["product_info_1.md", "product_info_2.md"]
102-
for file_name in file_names:
103-
file_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '..', 'files', file_name))
104-
file = await ai_client.agents.upload_file_and_poll(file_path=file_path, purpose=FilePurpose.AGENTS)
105-
logger.info(f"Uploaded file {file_path}, file ID: {file.id}")
106-
# Store both file id and the file path using the file name as key.
107-
files[file_name] = {"id": file.id, "path": file_path}
108-
109-
# Create the vector store using the file IDs.
110-
vector_store = await ai_client.agents.create_vector_store_and_poll(
111-
file_ids=[info["id"] for info in files.values()],
112-
name="sample_store"
113-
)
114-
115-
file_search_tool = FileSearchTool(vector_store_ids=[vector_store.id])
116-
toolset = AsyncToolSet()
117-
toolset.add(file_search_tool)
118-
119-
agent = await ai_client.agents.create_agent(
120-
model=os.environ["AZURE_AI_AGENT_DEPLOYMENT_NAME"],
121-
name="my-assistant",
122-
instructions="You are helpful assistant",
123-
toolset=toolset
124-
)
125-
126-
logger.info(f"Created agent, agent ID: {agent.id}")
127-
logger.info(f"Created agent, model name: {agent.model}")
100+
# Check if a previous agent created by the template exists
101+
agent_list = await ai_client.agents.list_agents()
102+
if agent_list.data:
103+
for agent_object in agent_list.data:
104+
if agent_object.name == "agent-template-assistant":
105+
agent = agent_object
106+
if agent == None:
107+
raise Exception("Agent not found")
128108

129109
except Exception as e:
130110
logger.error(f"Error creating agent: {e}", exc_info=True)
@@ -137,23 +117,6 @@ async def lifespan(app: fastapi.FastAPI):
137117
try:
138118
yield
139119
finally:
140-
# Cleanup on shutdown.
141-
if create_new_agent:
142-
try:
143-
for info in files.values():
144-
await ai_client.agents.delete_file(info["id"])
145-
logger.info(f"Deleted file {info['id']}")
146-
147-
if vector_store:
148-
await ai_client.agents.delete_vector_store(vector_store.id)
149-
logger.info(f"Deleted vector store {vector_store.id}")
150-
151-
if agent:
152-
await ai_client.agents.delete_agent(agent.id)
153-
logger.info(f"Deleted agent {agent.id}")
154-
except Exception as e:
155-
logger.error(f"Error during cleanup: {e}", exc_info=True)
156-
157120
try:
158121
await ai_client.close()
159122
logger.info("Closed AIProjectClient")

‎src/gunicorn.conf.py

Lines changed: 68 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,76 @@
11
import multiprocessing
22
import os
3+
from typing import Dict
4+
import asyncio
5+
from azure.ai.projects.aio import AIProjectClient
6+
from azure.ai.projects.models import FilePurpose, FileSearchTool, AsyncToolSet
7+
from azure.identity import DefaultAzureCredential
38

49
from dotenv import load_dotenv
510

611
load_dotenv()
712

13+
async def list_or_create_agent():
14+
files: Dict[str, Dict[str, str]] = {} # File name -> {"id": file_id, "path": file_path}
15+
vector_store = None
16+
agent = None
17+
18+
try:
19+
ai_client = AIProjectClient.from_connection_string(
20+
credential=DefaultAzureCredential(exclude_shared_token_cache_credential=True),
21+
conn_str=os.environ["AZURE_AIPROJECT_CONNECTION_STRING"],
22+
)
23+
24+
if os.environ.get("AZURE_AI_AGENT_ID"):
25+
try:
26+
agent = await ai_client.agents.get_agent(os.environ["AZURE_AI_AGENT_ID"])
27+
return
28+
except Exception as e:
29+
print(f"Error fetching agent: {e}")
30+
31+
# Check if a previous agent created by the template exists
32+
agent_list = await ai_client.agents.list_agents()
33+
if agent_list.data:
34+
for agent_object in agent_list.data:
35+
if agent_object.name == "agent-template-assistant":
36+
return
37+
38+
# Create a new agent with the required resources
39+
print(f"Creating new agent with resources")
40+
file_names = ["product_info_1.md", "product_info_2.md"]
41+
for file_name in file_names:
42+
file_path = os.path.abspath(os.path.join(os.path.dirname(__file__), 'files', file_name))
43+
file = await ai_client.agents.upload_file_and_poll(file_path=file_path, purpose=FilePurpose.AGENTS)
44+
# Store both file id and the file path using the file name as key.
45+
files[file_name] = {"id": file.id, "path": file_path}
46+
47+
# Create the vector store using the file IDs.
48+
vector_store = await ai_client.agents.create_vector_store_and_poll(
49+
file_ids=[info["id"] for info in files.values()],
50+
name="sample_store"
51+
)
52+
print(f"agent: file store and vector store success")
53+
54+
file_search_tool = FileSearchTool(vector_store_ids=[vector_store.id])
55+
toolset = AsyncToolSet()
56+
toolset.add(file_search_tool)
57+
58+
agent = await ai_client.agents.create_agent(
59+
model=os.environ["AZURE_AI_AGENT_DEPLOYMENT_NAME"],
60+
name="agent-template-assistant",
61+
instructions="You are helpful assistant",
62+
toolset=toolset
63+
)
64+
print(f"Created agent, agent ID: {agent.id}")
65+
66+
except Exception as e:
67+
print(f"Error creating agent: {e}", exc_info=True)
68+
raise RuntimeError(f"Failed to create the agent: {e}")
69+
70+
def on_starting(server):
71+
"""This code runs once before the workers will start."""
72+
asyncio.get_event_loop().run_until_complete(list_or_create_agent())
73+
874
max_requests = 1000
975
max_requests_jitter = 50
1076
log_file = "-"
@@ -13,8 +79,9 @@
1379
if not os.getenv("RUNNING_IN_PRODUCTION"):
1480
reload = True
1581

82+
preload_app = True
1683
num_cpus = multiprocessing.cpu_count()
17-
workers = 1 #(num_cpus * 2) + 1
84+
workers = (num_cpus * 2) + 1
1885
worker_class = "uvicorn.workers.UvicornWorker"
1986

2087
timeout = 120

0 commit comments

Comments
 (0)