Skip to content

Commit a08ee2c

Browse files
YilialinnYilialinn
authored
blog: add announcing apisix and ai/ml api integration (#1941)
1 parent 0292cb3 commit a08ee2c

File tree

1 file changed

+152
-0
lines changed

1 file changed

+152
-0
lines changed

‎blog/en/blog/2025/07/29/announcing-integration-of-apisix-and-ai-ml-api.md

Lines changed: 152 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,152 @@
1+
---
2+
title: "Announcing APISIX Integration with AI/ML API"
3+
authors:
4+
- name: "Yilia Lin"
5+
title: "Technical Writer"
6+
url: "https://github.com/Yilialinn"
7+
image_url: "https://github.com/Yilialinn.png"
8+
keywords:
9+
- API gateway
10+
- Apache APISIX
11+
- AI
12+
- AI/ML API
13+
- AI plugins
14+
description: "Apache APISIX supports 300+ LLMs through the integration with AI/ML API. Get your secure, single-endpoint access to AI models like GPT-4 and Claude, and more."
15+
tags: [Ecosystem]
16+
image: https://static.api7.ai/uploads/2025/07/23/d1O3mllW_apisix-ai-ml-api.webp
17+
---
18+
19+
> We're thrilled to announce that **AI/ML API** has become a supported provider to the `ai-proxy`, `ai-proxy-multi`, and `ai-request-rewrite` plugins in **Apache APISIX**. All the AI/ML APIs will be supported in the next APISIX version.
20+
<!--truncate-->
21+
22+
## Introduction
23+
24+
[AI/ML API](https://aimlapi.com/) is a single endpoint that gives you access to more than 300 ready-to-use AI models—large language models, embeddings, image and audio tools—through one standard REST interface. It is used by over 150,000 developers and organizations as a centralized LLM API gateway.
25+
26+
We're thrilled to announce that **AI/ML API** has become a supported provider to the `ai-proxy`, `ai-proxy-multi`, and `ai-request-rewrite` plugins in **Apache APISIX**.
27+
28+
AI/ML API provides a unified OpenAI-compatible API with access to **300+ LLMs** such as GPT-4, Claude, Gemini, DeepSeek, and others. This integration bridges the gap between your API infrastructure and leading AI services, enabling you to deploy intelligent features—like chatbots, real-time translations, and data analysis—faster than ever.
29+
30+
## Proxy to OpenAI via AI/ML API
31+
32+
### Prerequisites
33+
34+
1. [Install APISIX](https://apisix.apache.org/docs/apisix/installation-guide/).
35+
2. Generate your API key on [AI/ML API dashboard](https://platform.openai.com/api-keys).
36+
![Generate AI/ML API Key](https://static.api7.ai/uploads/2025/07/30/dGXA7d0r_ai-ml-api-key.webp)
37+
38+
### Configure the Route
39+
40+
Create a route and configure the `ai-proxy` plugin as such:
41+
42+
```yaml
43+
curl "http://127.0.0.1:9180/apisix/admin/routes" -X PUT \
44+
-H "X-API-KEY: ${ADMIN_API_KEY}" \
45+
-d '{
46+
"id": "ai-proxy-route",
47+
"uri": "/anything",
48+
"methods": ["POST"],
49+
"plugins": {
50+
"ai-proxy": {
51+
"provider": "aimlapi",
52+
"auth": {
53+
"header": {
54+
"Authorization": "Bearer '"$OPENAI_API_KEY"'" # Generated openai key from AI/ML API dashboard
55+
}
56+
},
57+
"options":{
58+
"model": "gpt-4"
59+
}
60+
}
61+
}
62+
}'
63+
```
64+
65+
### Test the Integration
66+
67+
Send a POST request to the route with a system prompt and a sample user question in the request body:
68+
69+
```bash
70+
curl "http://127.0.0.1:9080/anything" -X POST \
71+
-H "Content-Type: application/json" \
72+
-H "Host: api.openai.com" \
73+
-d '{
74+
"messages": [
75+
{ "role": "system", "content": "You are a mathematician" },
76+
{ "role": "user", "content": "What is 1+1?" }
77+
]
78+
}'
79+
```
80+
81+
### Verify Response
82+
83+
You should receive a response similar to the following:
84+
85+
```json
86+
{
87+
...,
88+
"choices": [
89+
{
90+
"index": 0,
91+
"finish_reason": "stop",
92+
"logprobs": null,
93+
"message": {
94+
"role": "assistant",
95+
"content": "1 + 1 equals 2.",
96+
"refusal": null,
97+
"annotations": []
98+
}
99+
}
100+
],
101+
"created": 1753845968,
102+
"model": "gpt-4-0613",
103+
"usage": {
104+
"prompt_tokens": 1449,
105+
"completion_tokens": 1008,
106+
"total_tokens": 2457
107+
...
108+
}
109+
```
110+
111+
## Core Use Cases
112+
113+
1. **Unified AI Service Management**
114+
115+
- **Multi-Model Proxy and Load Balancing**: Replace hardcoded vendor endpoints with a single APISIX interface, dynamically routing requests to models from OpenAI, Claude, DeepSeek, Gemini, Mistral, etc., based on cost, latency, or performance needs.
116+
- **Vendor-Agnostic Workflows**: Seamlessly switch between models (e.g., GPT-4 for creative tasks, Claude for document analysis) without code changes.
117+
118+
2. **Cost-Optimized Token Governance**
119+
120+
- **Token-Based Budget Enforcement**: Set per-team/monthly spending limits; auto-throttle requests when thresholds are exceeded.
121+
- **Caching & Fallbacks**: Cache frequent LLM responses (e.g., FAQ answers) or reroute to cheaper models during provider outages.
122+
123+
3. **Real-Time AI Application Scaling**
124+
125+
- **Chatbots & Virtual Agents**: Power low-latency conversational interfaces with streaming support for token-by-token responses.
126+
- **Data Enrichment Pipelines**: Augment APIs with AI—e.g., auto-summarize user reviews or translate product descriptions on-the-fly.
127+
128+
4. **Hybrid/Multi-Cloud AI Deployment**
129+
130+
- **Unified Control Plane**: Manage on-prem LLMs (e.g., Llama 3) alongside cloud APIs (OpenAI, Azure) with consistent policy enforcement.
131+
- **High Availability & Fault Tolerance**: Built-in health-checks, automatic retries and failover; if one LLM fails, traffic is rerouted within seconds to keep services alive.
132+
133+
5. **Enterprise AI Security & Compliance**
134+
135+
- **Data Security and Compliance**: Prompt Guard, content moderation, PII redaction and full audit logs in a single place.
136+
- **One Auth Layer for 300+ LLMs**: Unified authentication (JWT/OAuth2/OIDC) and authorization for 300+ LLM keys and policies.
137+
138+
## Conclusion
139+
140+
With AI/ML API now natively supported in Apache APISIX, you no longer have to choose between **speed**, **security**, or **scale**—you get all three.
141+
142+
- **One line of YAML** turns your gateway into a 300-model AI powerhouse.
143+
- **Zero code changes** let you hot-swap GPT-4 for Claude, or route 10 % of traffic to a cheaper model for instant cost savings.
144+
- **Built-in guardrails** (PII redaction, token budgets, content moderation) keep compliance teams happy while your product team ships faster.
145+
146+
### More Resources
147+
148+
- Related APISIX AI Plugins
149+
- [ai-proxy](https://apisix.apache.org/docs/apisix/plugins/ai-proxy/)
150+
- [ai-proxy-multi](https://apisix.apache.org/docs/apisix/plugins/ai-proxy-multi/)
151+
- [ai-request-rewrite](https://apisix.apache.org/docs/apisix/plugins/ai-request-rewrite/)
152+
- [AI/ML API Community](https://aimlapi.com/community)

0 commit comments

Comments
 (0)