You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: GOALS.md
+3-4Lines changed: 3 additions & 4 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,4 +1,3 @@
1
-
2
1
# Envoy AI Gateway GOALS.md
3
2
4
3
## Envoy AI Gateway Goals
@@ -21,12 +20,12 @@ Envoy AI Gateway will simplify the process of setting up an AI Gateway to manage
21
20
22
21
Envoy AI Gateway enables Platform Engineers to provide a Gateway solution that enables application developers to focus on leveraging GenAI for feature development.
23
22
24
-
***Preset Envoy Gateway Configurations:** Default configurations that simplify setup of routing to GenAI Services, making it accessible to application developers.
25
-
***Leveraging Envoy:** The project aims to leverage the functionality of the Envoy Gateway control plane and the Envoy Proxy data plane.
23
+
-**Preset Envoy Gateway Configurations:** Default configurations that simplify setup of routing to GenAI Services, making it accessible to application developers.
24
+
-**Leveraging Envoy:** The project aims to leverage the functionality of the Envoy Gateway control plane and the Envoy Proxy data plane.
26
25
27
26
## Non-Objectives
28
27
29
-
***Disruption of Existing Envoy Patterns:** This project is an additive layer designed to expand use cases for Envoy Proxy and Envoy Gateway without changing existing deployment or control patterns.
28
+
-**Disruption of Existing Envoy Patterns:** This project is an additive layer designed to expand use cases for Envoy Proxy and Envoy Gateway without changing existing deployment or control patterns.
Copy file name to clipboardExpand all lines: README.md
+8-9Lines changed: 8 additions & 9 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,12 +1,13 @@
1
1
# Envoy AI Gateway
2
+
2
3
Envoy AI Gateway is an open source project for using [Envoy Gateway](https://github.com/envoyproxy/gateway) to handle request traffic from application clients to Generative AI services.
3
4
4
5
## Usage
5
6
6
7
When using Envoy AI Gateway, we refer to a two-tier gateway pattern. **The Tier One Gateway** functions as a centralized entry point, and the **Tier Two Gateway** handles ingress traffic to a self-hosted model serving cluster.
7
8
8
-
+ The **Tier One Gateway** handles authentication, top-level routing, and global rate limiting
9
-
+ The **Tier Two Gateway** provides fine-grained control over self-hosted model access, with endpoint picker support for LLM inference optimization.
9
+
- The **Tier One Gateway** handles authentication, top-level routing, and global rate limiting
10
+
- The **Tier Two Gateway** provides fine-grained control over self-hosted model access, with endpoint picker support for LLM inference optimization.
10
11
11
12

12
13
@@ -85,17 +86,16 @@ Envoy AI Gateway supports a wide range of AI providers, making it easy to integr
85
86
</table>
86
87
</div>
87
88
88
-
89
89
## Documentation
90
90
91
-
*[Blog](https://aigateway.envoyproxy.io/blog) introducing Envoy AI Gateway.
92
-
*[Documentation](https://aigateway.envoyproxy.io/docs) for Envoy AI Gateway.
93
-
*[Quickstart](https://aigateway.envoyproxy.io/docs/getting-started/) to use Envoy AI Gateway in a few simple steps.
94
-
*[Concepts](https://aigateway.envoyproxy.io/docs/concepts/) to understand the architecture and resources of Envoy AI Gateway.
91
+
-[Blog](https://aigateway.envoyproxy.io/blog) introducing Envoy AI Gateway.
92
+
-[Documentation](https://aigateway.envoyproxy.io/docs) for Envoy AI Gateway.
93
+
-[Quickstart](https://aigateway.envoyproxy.io/docs/getting-started/) to use Envoy AI Gateway in a few simple steps.
94
+
-[Concepts](https://aigateway.envoyproxy.io/docs/concepts/) to understand the architecture and resources of Envoy AI Gateway.
95
95
96
96
## Contact
97
97
98
-
* Slack: Join the [Envoy Slack workspace][] if you're not already a member. Otherwise, use the
98
+
- Slack: Join the [Envoy Slack workspace][] if you're not already a member. Otherwise, use the
99
99
[Envoy AI Gateway channel][] to start collaborating with the community.
100
100
101
101
## Get Involved
@@ -113,7 +113,6 @@ which includes information on how to build and test the project.
113
113
114
114
The proposal of using Envoy Gateway as a [Cloud Native LLM Gateway][Cloud Native LLM Gateway] inspired the initiation of this project.
Copy file name to clipboardExpand all lines: RELEASES.md
+31-28Lines changed: 31 additions & 28 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -20,18 +20,19 @@ This document focuses on compatibility concerns of those using Envoy AI Gateway.
20
20
It is important to note that the support policy is subject to change at any time. The support policy is as follows:
21
21
22
22
First of all, there are four areas of compatibility that we are concerned with:
23
-
*[Using envoyproxy/ai-gateway as a Go package](#public-go-package).
24
-
*[Deploying the Envoy AI Gateway controller through the Kubernetes Custom Resource Definition (CRD)](#Custom-Resource-Definitions).
25
-
*[Upgrading the Envoy AI Gateway controller](#Upgrading-the-Envoy-AI-Gateway-controller).
26
-
*[Envoy Gateway vs Envoy AI Gateway compatibility](#Envoy-Gateway-vs-Envoy-AI-Gateway-compatibility).
23
+
24
+
-[Using envoyproxy/ai-gateway as a Go package](#public-go-package).
25
+
-[Deploying the Envoy AI Gateway controller through the Kubernetes Custom Resource Definition (CRD)](#Custom-Resource-Definitions).
26
+
-[Upgrading the Envoy AI Gateway controller](#Upgrading-the-Envoy-AI-Gateway-controller).
27
+
-[Envoy Gateway vs Envoy AI Gateway compatibility](#Envoy-Gateway-vs-Envoy-AI-Gateway-compatibility).
27
28
28
29
### Public Go package
29
30
30
31
Since we do not envision this repository ends up as a transitive dependency, i.e. only used as a direct dependency such as in a custom control plane, etc., we assume that any consumer of the project should have the full control over the source code depending on the project. This allows us to declare deprecation and introduce the breaking changes in the version after the next one since they can migrate the code at their discretion. For example, any public API that is marked as deprecated in the version N will be removed in the version N+2. We document how users should migrate to the new API will be documented in the release notes if applicable, but we do not guarantee that the migration path will be provided.
31
32
32
33
### Custom Resource Definitions
33
34
34
-
The Custom Resource Definitions (CRDs) are defined in api/${version}/*.go files. The CRDs are versioned as v1alpha1, v1alpha2, etc.
35
+
The Custom Resource Definitions (CRDs) are defined in api/${version}/\*.go files. The CRDs are versioned as v1alpha1, v1alpha2, etc.
35
36
36
37
**For alpha versions**, we simply employ the same deprecation policy as the Go package. In other words, the APIs will be marked as deprecated in the version N and will be removed in the version N+2 but without any guarantee of migration path.
37
38
@@ -62,32 +63,34 @@ This section is for maintainers of the project. Let's say we are going to releas
62
63
Each non-patch release should start with Release Candidate (RC) phase as follows:
63
64
64
65
1. First, notify the community that we are going to cut the release candidate and therefore the main branch is frozen.
65
-
The main branch should only accept the bug fixes, the security fixes, and documentation changes.
66
-
The release candidate should always be cut from the main branch.
66
+
The main branch should only accept the bug fixes, the security fixes, and documentation changes.
67
+
The release candidate should always be cut from the main branch.
67
68
68
69
2. Cut the request candidate tag from the main branch. The tag should be v0.50.0-rc1. Assuming the remote `origin` is the main envoyproxy/ai-gateway repository,
69
-
the command to cut the tag is:
70
-
```
71
-
git fetch origin # make sure you have the latest main branch locally.
72
-
git tag v0.50.0-rc1 origin/main
73
-
git push origin v0.50.0-rc1
74
-
```
70
+
the command to cut the tag is:
71
+
72
+
```
73
+
git fetch origin # make sure you have the latest main branch locally.
74
+
git tag v0.50.0-rc1 origin/main
75
+
git push origin v0.50.0-rc1
76
+
```
77
+
75
78
Pushing a tag will trigger the pipeline to build the release candidate image and the helm chart tagged with the release candidate tag.
76
79
The release candidate image will be available in the Docker Hub.
77
80
78
81
3. The release candidate should be tested by the maintainers and the community. If there is any issue, the issue should be fixed in the main branch
79
-
and the new rc tag should be created. For example, if there is an issue in the release candidate v0.50.0-rc1, replace `v0.50.0-rc1` with `v0.50.0-rc2`
80
-
in the above command and repeat the process.
82
+
and the new rc tag should be created. For example, if there is an issue in the release candidate v0.50.0-rc1, replace `v0.50.0-rc1` with `v0.50.0-rc2`
83
+
in the above command and repeat the process.
81
84
82
85
### Release Phase
83
86
84
87
1. Once the release candidate is stable, we will cut the release from the main branch, assuming that's exactly the same as the last release candidate.
85
-
The command to cut the release is exactly the same as the release candidate:
86
-
```
87
-
git fetch origin # make sure you have the latest main branch locally.
88
-
git tag v0.50.0 origin/main
89
-
git push origin v0.50.0
90
-
```
88
+
The command to cut the release is exactly the same as the release candidate:
89
+
```
90
+
git fetch origin # make sure you have the latest main branch locally.
91
+
git tag v0.50.0 origin/main
92
+
git push origin v0.50.0
93
+
```
91
94
Pushing a tag will trigger the pipeline to build the release image and the helm chart tagged with the release tag.
92
95
The release image will be available in the Docker Hub.
93
96
2. The draft release note will be created in the GitHub repository after the pipeline is completed.
@@ -102,15 +105,15 @@ Each non-patch release should start with Release Candidate (RC) phase as follows
102
105
2. Once the PR is merged, the maintainers will decide when to cut the patch release. There's no need to wait for multiple backports to cut the patch release, etc.
103
106
Do not cut the tag until all CI passes on the release/v0.50 branch.
104
107
3. The patch release should be cut from the `release/v0.50` branch. The command to cut the patch release is exactly the same as normal release:
105
-
```
106
-
git fetch origin # make sure you have the latest release/v0.50 branch locally.
107
-
git tag v0.50.1 origin/release/v0.50
108
-
git push origin v0.50.1
109
-
```
108
+
```
109
+
git fetch origin # make sure you have the latest release/v0.50 branch locally.
110
+
git tag v0.50.1 origin/release/v0.50
111
+
git push origin v0.50.1
112
+
```
110
113
Pushing a tag will trigger the pipeline to build the patch release image and the helm chart tagged with the patch release tag.
111
114
The patch release image will be available in the Docker Hub.
112
115
4. The draft release note will be created in the GitHub repository after the pipeline is completed.
113
116
Edit the release note nicely by hand to reflect the changes in the release.
114
117
5. Update the documentation on the main branch to reflect the new version. This has the following items:
115
-
* Change `v0.50.0` to `v0.50.1` in site/versioned_docs/version-0.50 directory.
116
-
* Update the site/src/pages/release-notes.md to add the new release note.
118
+
- Change `v0.50.0` to `v0.50.1` in site/versioned_docs/version-0.50 directory.
119
+
- Update the site/src/pages/release-notes.md to add the new release note.
0 commit comments