You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Infra shouldn’t slow you down. In May, we shipped the kind of upgrades that help you move fast into productiion and stay in control — whether you're scaling agents, securing AI behavior, or managing costs across teams.
7
+
In May, we shipped the kind of upgrades that help you move your AI Agents fast into productiion and stay in control — whether you're scaling, securing AI behavior, or bringing new models to your apps.
8
8
9
-
From deeper integrations with agent frameworks to support for newer models, Portkey keeps evolving as the AI infra layer teams can rely on. We also shipped observability upgrades, and added tighter controls for cost, access, and security.
9
+
We launched deep integrations with agent frameworks like PydanticAI and OpenAI Agents SDK, added enterprise-grade controls to Claude Code, made it simpler to call a remote MCP server simpler and much more!
10
10
11
11
Here’s everything new this month:
12
12
@@ -15,67 +15,55 @@ Here’s everything new this month:
15
15
16
16
| Area | Key Updates |
17
17
| :-- | :-- |
18
-
|**AI agent infrastructure**| • PydanticAI integration for modular agent development<br/>• OpenAI Agents SDK support with monitoring, guardrails, and cost tracking<br/>• Strands Agents integration with observability, retries, and load balancing<br/>• Remote MCP server support via Responses API<br/>• Arize Phoenix tracing integration for unified agent observability |
19
-
|**Platform**| • Deep integration into Azure AI ecosystem (OpenAI, Foundry, APIM, Marketplace)<br/>• Support for Claude Code with rate limits, observability, and access control<br/>• AI coding assistant integrations: Cline, Roo Code<br/>• Multimodal embedding support via Vertex AI (text, image, video)<br/>• Multi-label support for prompt versions<br/>• OpenAI Computer Use Tool routing and observability<br/>• Full support for `GET`, `PUT`, and `DELETE` HTTP methods<br/>• OTel analytics export to your existing observability stack |
20
-
|**Improvements**| • Token cost tracking for gpt-image-1<br/>• Ping messages removed from streamed responses<br/>• Resizing metadata columns in logs |
21
-
|**New models & providers**| • Claude 4 now live<br/>• PDF support for Claude via Anthropic and Bedrock<br/>• OpenAI’s Computer Use Tool supported via Responses API<br/>• Grok 3 and Grok 3 Mini on Azure<br/>• Gemini 2.5 Thinking Mode in Prompt Playground<br/>• Extended thinking for Claude 3.7 and Claude 4<br/>• Mistral supports function calling<br/>• WorkersAI supports image generation<br/>• Lepton AI, Nscale now integrated<br/>• Tool calling enabled for Mistral and OpenRouter<br/>• MIME type support for Vertex and Google<br/>• PDF support via Anthropic and Bedrock |
22
-
|**Guardrails**| • Prompt Security guardrails for injection detection and sensitive data protection<br/>• JWT validator input guardrail<br/>• PANW Prisma AIRS plugin for real-time prompt/response risk blocking<br/>• Model whitelist guardrail for org/environment/request-level control |
23
-
|**Documentation**| • Guardrail documentation moved under “Integrations”<br/>• New solution pages for AWS Bedrock and GovCloud<br/>• Cookbook: OpenAI Computer Use tool <br/>• Cookbook: Optimizing Prompts using LLama Prompt Ops |
18
+
|**AI agent infrastructure**| • Integration with PydanticAI, OpenAI Agents SDK, Strands Agents integration<br/>• Remote MCP server support via Responses API<br/>• Arize Phoenix tracing integration|
19
+
| **AI tools** | • Integration with Claude Code, Cline, Roo Code<br/>
20
+
|**Platform**| • Deep integration into Azure AI ecosystem<br/>• Multi-label support for prompt versions<br/>• Full support for `GET`, `PUT`, and `DELETE` HTTP methods<br/>• OTel analytics export |
21
+
|**New models & providers**| • Claude 4 now live<br/>• Grok 3 and Grok 3 Mini on Azure<br/>• Lepton AI, Nscale now integrated<br/>• PDF support for Claude via Anthropic and Bedrock<br/>• WorkersAI supports image generation<br/>• Tool calling enabled for Mistral and OpenRouter<br/>• MIME type support for Vertex and Google<br/> |
22
+
| **Guardrails** | • Prompt Security guardrails for injection detection and sensitive data protection<br/>• JWT validator input guardrail<br/>• PANW Prisma AIRS plugin for real-time prompt/response risk blocking<br/>• Model whitelist guardrail for org/environment/request-level control<br/>
24
23
---
25
24
26
25
## AI Agent Infrastructure
27
26
AI agent frameworks are helping teams prototype faster, but taking agents to production requires real infrastructure. Portkey integrates with leading frameworks to bring interoperability, observability, reliability, and cost management to your agent workflows.
28
27
29
-
**PydanticAI**
30
-
31
-
Portkey now integrates PydanticAI, a Python framework that brings FastAPI-like ergonomics to building AI agents. With Portkey, you can:
32
-
33
-
- Build modular, testable agents with a clean developer experience.
34
-
- Route all agent calls through Portkey for observability and debugging.
35
-
- Add retries, fallbacks, guardrails, and cost tracking without extra infra
36
-
37
-
See how it's done [here](https://portkey.ai/docs/integrations/agents/pydantic-ai#pydantic-ai)
38
-
39
-
**OpenAI Agents SDK**
40
-
41
-
Portkey integrates with the OpenAI Agents SDK to help teams ship production-grade agents with built-in planning, memory, and tool use. You can now:
28
+
<CardGroupcols={3}>
42
29
43
-
- Monitor and debug each step of the agent’s reasoning and tool use.
44
-
- Automatically track usage and cost for each agent call.
45
-
- Apply guardrails to both agent input and output.
46
-
- Scale agent-based workflows across environments with versioned control
Strands Agents is a lightweight agent framework built by AWS to simplify agent development.<br/><br/>
46
+
<br/>
47
+
</Card>
51
48
52
-
Strands Agents is a lightweight agent framework built by AWS to simplify agent development.
49
+
</CardGroup>
53
50
54
-
Portkey now integrates seamlessly with Strands Agents to make them production-ready. With this integration, you get:
51
+
**Tracing Integrations: Arize AI**
55
52
56
-
- Full observability into agent steps, tool calls, and interactions
57
-
- Built-in reliability through fallbacks, retries, and load balancing
58
-
- Cost tracking and spend optimization
53
+
For teams consolidating observability into Arize, you can now view Portkey’s logs directly into Arize Phoenix to get unified trace views across your LLM workflows.
59
54
60
-
See how it's done [here](https://portkey.ai/docs/integrations/agents/strands)
61
55
62
-
**Support for remote MCP servers!**
56
+
## Remote MCP servers
63
57
64
58
Portkey now supports calling a remote MCP server that is maintained by developers and organizations across the internet that expose these tools to MCP clients via the Responses API
65
-
Read more about the integration [here](https://portkey.ai/docs/product/ai-gateway/remote-mcp)
66
-
67
-
**Tracing Integrations: Arize AI**
59
+
Read more about the integration [here](https://portkey.ai/docs/product/ai-gateway/remote-mcp).
68
60
69
-
For teams consolidating observability into Arize, you can now view Portkey’s logs directly into Arize Phoenix to get unified trace views across your LLM workflows.
61
+
## Azure AI ecosystem
70
62
71
-
## Platform
72
-
73
-
**Azure AI ecosystem**
63
+
More than half of Fortune 500 companies use Azure OpenAI. But building GenAI apps in the enterprise is still messy, cost attribution, routing logic, usage tracking, model evaluation... all scattered.
More than half of Fortune 500 companies use Azure OpenAI. But building GenAI apps in the enterprise is still messy, cost attribution, routing logic, usage tracking, model evaluation... all scattered.
78
-
79
67
With Portkey’s deep integration into the Azure AI ecosystem (OpenAI, Foundry, APIM, Marketplace), teams can now build, scale, and govern GenAI apps without leaving their existing cloud setup.
80
68
81
69
@@ -84,57 +72,49 @@ Our customers are vouching for it!
<Cardhorizontaltitle="Working with Azure? Read more here."href="https://portkey.ai/for/azure">
88
-
</Card>
89
75
90
-
###Portkey for AI Tools
76
+
## Portkey for AI Tools
91
77
92
78
<CardGroupcols={2}>
93
79
94
80
<Card
95
81
title="Claude Code"
96
-
icon="terminal"
97
-
href="/docs/integrations/libraries/claude-code"
98
-
>
99
-
Bring enterprise-grade visibility, governance, and access control to Anthropic’s agentic coding assistant. Enforce rate limits, monitor usage with rich metadata, debug faster with detailed logs, and issue virtual keys for secure access across teams and infrastructures (Bedrock, Vertex AI).
Bring enterprise-grade visibility, governance, and access control to Claude Code.
100
84
</Card>
101
85
102
86
<Card
103
87
title="Cline"
104
-
icon="code"
105
-
href="/docs/integrations/libraries/cline"
106
-
>
107
-
Supercharge your AI-powered terminal with unified logging, granular cost tracking, access controls, and advanced observability. Portkey lets you audit every prompt, tool invocation, and generation for full developer productivity oversight.
Supercharge your AI-powered terminal with cost tracking, access controls, and observability.
108
90
</Card>
109
91
110
92
<Card
111
93
title="Roo Code"
112
-
icon="rocket"
113
-
href="/docs/integrations/libraries/roo-code"
114
-
>
115
-
Add security, compliance, and real-time analytics to your code assistant workflows. Track usage, control spend, and manage access across all Roo deployments—ensuring safe and optimized coding environments at scale.
Add security, compliance, and real-time analytics to your code assistant workflows.
116
96
</Card>
117
97
118
98
<Card
119
99
title="Goose"
120
-
icon="feather"
121
-
href="/docs/integrations/libraries/goose"
122
-
>
123
-
Enable enterprise features in Goose—AI code review and generation—by routing through Portkey. Gain full observability, cost controls, and secure team access for responsible and accountable AI coding, with seamless integration into your workflows.
Add essential enterprise controls to Goose's powerful autonomous coding capabilities
124
102
</Card>
125
103
126
104
</CardGroup>
127
105
128
-
**Multilmodal embeddings**
106
+
## Multilmodal embeddings
129
107
130
108
Portkey now supports embedding APIs from Vertex AI for text, image, and video—across multiple languages.
131
109
This unlocks the ability to:
132
-
- Build cross-language search and retrieval
110
+
- Build multimodal search and retrieval
133
111
- Power multimodal RAG pipelines
134
112
- Track, route, and optimize embedding usage at scale
135
113
136
114
Read more about the implementation [here](https://portkey.ai/docs/integrations/llms/vertex-ai/embeddings)
137
115
116
+
## Platform
117
+
138
118
**Multi-label support for prompts**
139
119
140
120
<Frame>
@@ -143,16 +123,6 @@ Read more about the implementation [here](https://portkey.ai/docs/integrations/l
143
123
144
124
You can now assign multiple labels to a single prompt version, making it easy to promote a version across environments like staging and production.
145
125
146
-
**OpenAI Computer Use Tool**
147
-
148
-
Build production-grade browser automation with enterprise-level controls using Portkey and:
149
-
150
-
- Route and monitor Computer Use API calls
151
-
- Build a complete Playwright-based browser automation solution
152
-
- Add observability, logging, and cost controls with Portkey
153
-
154
-
Explore the implementation [here](https://portkey.ai/docs/guides/use-cases/openai-computer-use)
155
-
156
126
**Gateway to any API**
157
127
158
128
Portkey now supports `GET`, `PUT`, and `DELETE` HTTP methods in addition to `POST`, allowing you to route requests to any external or self-hosted provider endpoint. This means you can connect to custom APIs directly through Portkey with full observability for every call.
@@ -166,89 +136,58 @@ You can now export Portkey analytics to any OpenTelemetry (OTel)-compatible coll
166
136
- Ping messages are removed from streamed responses.
167
137
- Resizing metadata columns in logs
168
138
169
-
<CardGroupcols={3}>
170
-
<Cardtitle="Claude 4">
171
-
Now live on Portkey for advanced reasoning and coding.
172
-
</Card>
173
-
<Cardtitle="Grok 3 & Grok 3 Mini">
174
-
Available on Azure for high-performance inference.
175
-
</Card>
176
-
<Cardtitle="Lepton AI Integration">
177
-
Integrate Lepton AI into your Portkey workflows.
178
-
</Card>
179
-
<Cardtitle="Nscale Models">
180
-
Access Nscale models through Portkey.
181
-
</Card>
182
-
</CardGroup>
183
-
184
-
<CardGroupcols={3}>
185
-
<Cardtitle="PDF Support for Claude">
186
-
Send PDFs to Claude via Anthropic and Bedrock.
187
-
</Card>
188
-
<Cardtitle="OpenAI Computer Use Tool">
189
-
Access Computer Use Tool via the Responses API.
190
-
</Card>
191
-
<Cardtitle="Gemini 2.5 Thinking Mode">
192
-
Thinking Mode now supported in Prompt Playground.
193
-
</Card>
194
-
<Cardtitle="Extended Thinking for Claude">
195
-
Claude 3.7 and Claude 4 support extended thinking.
196
-
</Card>
197
-
<Cardtitle="Mistral Function Calling">
198
-
Mistral now supports function calling.
199
-
</Card>
200
-
<Cardtitle="WorkersAI Image Generation">
201
-
Generate images directly using WorkersAI.
202
-
</Card>
203
-
<Cardtitle="Tool Calling for Mistral & OpenRouter">
<li><b>Claude 4</b> is now live for advanced reasoning and coding.</li>
151
+
<li><b>Grok 3 & Grok 3 Mini</b> are available on Azure</li>
152
+
<li><b>Lepton AI</b> is now live</li>
153
+
<li><b>Nscale Models</b> can now be accessed through Portkey.</li>
154
+
</ul>
155
+
</div>
156
+
<divstyle={{ flex: 1, minWidth: 300 }}>
157
+
<ul>
158
+
<b>Updates</b>
159
+
<li><b>PDF Support for Claude</b> via Anthropic and Bedrock.</li>
160
+
<li><b>Gemini 2.5 Thinking Mode</b> is now supported in Prompt Playground.</li>
161
+
<li><b>Extended Thinking</b> is available for Claude 3.7 and Claude 4.</li>
162
+
<li>Image generation now supported on WorkersAI</li>
163
+
<li><b>Tool Calling and Function Calling for Mistral</b> is now live.</li>
164
+
<li><b>MIME Type</b> is now supported for Vertex AI</li>
165
+
</ul>
166
+
</div>
167
+
</div>
214
168
215
169
## Guardrails
216
170
217
171
-**Prompt Security guardrails**: Integrate with Prompt Security to detect prompt injection and prevent sensitive data exposure in both prompts and responses.
218
172
219
173
-**JWT validator guardrail**: Added as an input guardrail to validate incoming JWT tokens before requests are sent to the LLM.
220
174
221
-
-**PANW Prisma AIRS Plugin**:Portkey now integrates with Palo Alto Networks' AIRS (AI Runtime Security) to enforce guardrails that block risky prompts or model responses based on real-time security analysis.
222
-
223
-
-**Model whitelist guardrail**:Restrict or deny specific models at the org, environment, or request level using a flexible whitelist/blacklist guardrail.
224
-
225
-
## Resources
226
-
227
-
**LLama Prompt Ops: Optimizing Prompts**
175
+
-**PANW Prisma AIRS Plugin**: Portkey now integrates with Palo Alto Networks' AIRS (AI Runtime Security) to enforce guardrails that block risky prompts or model responses based on real-time security analysis.
228
176
229
-
Looking to upgrade to the latest Llama models? Llama Prompt Ops makes it easy—transform your existing prompts for optimal performance with Llama models automatically, no manual rewriting needed.
177
+
-**Model whitelist guardrail**: Restrict or deny specific models at the org, environment, or request level using a flexible whitelist/blacklist guardrail.
230
178
231
-
For customer support teams, we provide a comprehensive guide to building systems that analyze support messages for urgency, sentiment, and categorization.
232
-
233
-
[Read the Llama Prompt Ops guide](https://portkey.ai/docs/guides/prompts/llama-prompts)
0 commit comments