Skip to content

Commit 7a119bc

Browse files
authored
docs: add diagrams for AI Gateway blogs (#1906)
1 parent 7dc0d5b commit 7a119bc

File tree

5 files changed

+21
-2
lines changed

5 files changed

+21
-2
lines changed

blog/en/blog/2025/02/24/apisix-ai-gateway-features.md

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ image: https://static.api7.ai/uploads/2025/03/07/Qs4WrU0I_apisix-ai-gateway.webp
2020

2121
## Introduction: The Rise of AI Agents and the Evolution of AI Gateway
2222

23-
In recent years, AI agents such as AutoGPT, Chatbots, and AI Assistants have seen rapid development. These applications rely heavily on API calls to large language models (LLMs), which has brought about challenges considering high concurrency, cost control, and security.
23+
In recent years, AI agents such as AutoGPT, Chatbots, and AI Assistants have seen rapid development. These applications rely heavily on API calls to large language models (LLMs), which have brought about challenges considering high concurrency, cost control, and security.
2424

2525
Traditional API gateways primarily serve Web APIs and microservices and are not optimized for the unique needs of AI applications. This has led to the emergence of the concept of AI gateway. An AI gateway needs to provide enhanced capabilities in the following areas:
2626

@@ -60,6 +60,8 @@ Users can flexibly allocate traffic weights among different DeepSeek providers b
6060

6161
These capabilities enable AI applications to adapt flexibly to different LLMs, improve reliability, and reduce API calling costs.
6262

63+
![AI Proxy](https://static.api7.ai/uploads/2025/08/01/TmTsNypy_ai-proxy-multi-workflow.webp)
64+
6365
## AI Security Protection: Ensuring Safe and Compliant Use of AI
6466

6567
AI APIs may involve sensitive data, misleading information, and potential misuse. Therefore, an AI gateway needs to provide security at multiple levels.
@@ -99,6 +101,8 @@ Through Apache APISIX, enterprises can achieve fine-grained management of token
99101

100102
## Smart Routing: Dynamic Traffic Management for AI APIs
101103

104+
![Smart Routing](https://static.api7.ai/uploads/2025/04/28/bzziWsxs_smart-routing.webp)
105+
102106
During AI API calls, different tasks may require different LLMs. For example:
103107

104108
- Code generation requests → sent to GPT-4 or DeepSeek.

blog/en/blog/2025/03/06/what-is-an-ai-gateway.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ As AI systems become integral to business operations, ensuring their reliability
9292

9393
To address these challenges, the concept of an AI gateway has emerged. An AI gateway extends the functionalities of a traditional API gateway by incorporating features specifically designed for AI applications and LLM scenarios. It serves as a unified endpoint for connecting AI infrastructure and services, providing comprehensive control, security, and observability of AI traffic between applications and models.
9494

95-
![API7 AI gateway architecture](https://static.api7.ai/uploads/2025/03/06/iCGmdwUZ_api7-ai-gateway.webp)
95+
![APISIX AI gateway architecture](https://static.api7.ai/uploads/2025/08/01/KvjMKKx2_apisix-ai-gateway-architecture.webp)
9696

9797
### Core Features of an AI Gateway
9898

@@ -104,6 +104,8 @@ An effective AI gateway encompasses several key functionalities:
104104
- **Prompt Protection**: Ensures that prompts sent to LLMs do not contain sensitive or inappropriate content, safeguarding against unintended data exposure.
105105
- **Content Moderation**: Monitors and filters responses from AI models to prevent the dissemination of harmful or non-compliant information.
106106

107+
![Security Workflow](https://static.api7.ai/uploads/2025/08/01/unlrtuQl_ai-gateway-security-feature.webp)
108+
107109
#### 2. Observability
108110

109111
- **Usage Tracking**: Monitors token consumption and provides insights into how AI services are utilized, aiding in cost management and capacity planning.
@@ -119,6 +121,9 @@ An effective AI gateway encompasses several key functionalities:
119121
#### 4. Reliability
120122

121123
- **Multi-LLM Load Balancing**: Distributes requests across multiple AI models to optimize performance and prevent overloading.
124+
125+
![AI Proxy](https://static.api7.ai/uploads/2025/08/01/TmTsNypy_ai-proxy-multi-workflow.webp)
126+
122127
- **Retry and Fallback Mechanisms**: Implements strategies to handle AI service failures gracefully, ensuring uninterrupted user experiences.
123128
- **Traffic Prioritization**: Routes high-priority requests to the most reliable AI services while deferring less critical tasks.
124129

blog/en/blog/2025/03/21/ai-gateway-vs-api-gateway-differences-explained.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,8 @@ To connect AI agents with external data and APIs, the **[Model Context Protocol
105105
2. **Gateway Routing**: The AI gateway validates permissions, injects API keys, and routes the request to relevant services.
106106
3. **Response Synthesis**: The gateway aggregates API responses (e.g., weather data + CRM contacts) and feeds them back to the AI model.
107107

108+
![How MCP Works](https://static.api7.ai/uploads/2025/08/01/zHkQ4hM0_how-mcp-works.webp)
109+
108110
**Example**: A user asks, "Email our top client in NYC about today's weather." The AI gateway uses MCP to:
109111

110112
- Fetch the top client from Salesforce.

blog/en/blog/2025/03/24/6-essential-ai-gateway-use-cases.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,8 @@ With this foundation in place, let's explore the six common application scenario
3434

3535
Modern enterprises increasingly rely on diverse AI models to address varied business needs, from customer-facing chatbots to internal document analysis. However, managing multiple vendors (e.g., OpenAI, Anthropic, Mistral) and deployment environments (cloud, on-prem, hybrid) introduces operational chaos.
3636

37+
![Centralized AI Service Management](https://static.api7.ai/uploads/2025/08/01/vwfP6Mwx_centralized-ai-gateway.webp)
38+
3739
Enterprises adopt specialized models for specific tasks:
3840

3941
- **GPT-4**: High-quality text generation for customer support.
@@ -79,6 +81,8 @@ AI services, particularly those based on large language models, can incur signif
7981
- **Budget Enforcement**: Setting spending limits for different teams or applications
8082
- **Caching Strategies**: Reducing redundant calls by storing frequent responses
8183

84+
![Cost Optimization and Rate Limiting](https://static.api7.ai/uploads/2025/08/01/D0JOkr1h_cost-optimization-and-rate-limiting.webp)
85+
8286
For instance, a customer service application might cache common questions about password resets or refund processes, significantly reducing the number of model invocations needed.
8387

8488
As AI adoption continues to accelerate, we can expect AI gateways to evolve with even more sophisticated cost management capabilities:

blog/en/blog/2025/04/08/introducing-apisix-ai-gateway.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -41,6 +41,8 @@ The [`ai-proxy-multi`](https://apisix.apache.org/docs/apisix/plugins/ai-proxy-mu
4141

4242
Additionally, the plugin supports logging LLM request information in the access log, such as token usage, model, time to first response, and more.
4343

44+
![AI Proxy](https://static.api7.ai/uploads/2025/08/01/TmTsNypy_ai-proxy-multi-workflow.webp)
45+
4446
**Example: Load Balancing**:
4547

4648
The following example demonstrates how to configure two models for load balancing, forwarding 80% of the traffic to one instance and 20% to another.
@@ -281,6 +283,8 @@ The [`ai-prompt-template`](https://apisix.apache.org/docs/apisix/plugins/ai-prom
281283
282284
The [`ai-prompt-guard`](https://apisix.apache.org/docs/apisix/plugins/ai-prompt-guard/) plugin protects your large language model (LLM) endpoints by inspecting and validating incoming prompt messages. It checks the request content against user-defined allow and deny patterns, ensuring only approved input is forwarded to the upstream LLM. Depending on its configuration, the plugin can check either the latest message or the entire conversation history and can be set to inspect prompts from all roles or only from the end user.
283285
286+
![ai-prompt-guard](https://static.api7.ai/uploads/2025/08/01/6Dl4AQGL_ai-prompt-guard-workflow.webp)
287+
284288
### Content Moderation
285289
286290
#### 8. ai-aws-content-moderation

0 commit comments

Comments
 (0)