You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have an Azure AI Foundry setup with AI Gateway (Azure API Management) enabled on my project. I'm trying to understand whether the AI Agent Service routes its internal model inference calls through the AI Gateway.
My setup:
Foundry resource with AI Gateway (APIM) enabled on the project
AI Gateway status shows "Enabled" in the Admin console > AI Gateway tab
Model deployment: gpt-4.1-mini
What I'm observing:
Direct model calls (Completions API) — When I call gpt-4.1-mini directly via the Completions API, I can see the request appearing in APIM logs and metrics. This works as expected.
Agent calls — When I invoke my agent via the Agent Service endpoint (services.ai.azure.com), the agent responds correctly, but I do NOT see any corresponding model inference request in APIM logs. The agent's internal LLM calls appear to bypass APIM entirely.
My questions:
When AI Gateway is enabled on a project, does the Agent Service route its internal model inference calls (the LLM calls the agent makes to "think" and reason) through the APIM gateway? Or does the Agent Service have a direct connection to the model that bypasses APIM?
If agents currently bypass APIM for model inference, is there a way to configure the Agent Service to route ALL model inference calls through the AI Gateway? For example, through the "Bring your own AI gateway" preview feature or a specific connection configuration?
Environment:
Region: eastus
APIM SKU: Standard V2
AI Gateway configured via Foundry Admin console > AI Gateway tab
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
I have an Azure AI Foundry setup with AI Gateway (Azure API Management) enabled on my project. I'm trying to understand whether the AI Agent Service routes its internal model inference calls through the AI Gateway.
My setup:
What I'm observing:
My questions:
Environment:
Region: eastus
APIM SKU: Standard V2
AI Gateway configured via Foundry Admin console > AI Gateway tab
Thanks!!
Beta Was this translation helpful? Give feedback.
All reactions