-
Notifications
You must be signed in to change notification settings - Fork 5.3k
add WG AI Gateway #8521
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
add WG AI Gateway #8521
Conversation
@shaneutt: GitHub didn't allow me to request PR reviews from the following users: david-martin, kflynn, rootfs, yuzisun. Note that only kubernetes members and repo collaborators can review this PR, and authors cannot review their own PRs. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: shaneutt The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
37481e4
to
84d372b
Compare
/hold |
9c05bc4
to
ea77fe1
Compare
@aojea a kind reminder... |
---> | ||
# AI Gateway Working Group | ||
|
||
The AI Gateway Working Group focuses on the intersection of AI and networking, particularly in the context of extending load-balancer, gateway and proxy technologies to manage and route traffic for AI Inference. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's not "particularly" in the gateway context, it's specifically/exclusively in that context, isn't it?
Saying the WG "focuses on the intersection of AI and networking" makes it sound like DRA and maybe MCP would be in-scope too.
EDIT: OMG, I just realized that even Gateway API Inference Extension is defined as being out of scope for this WG. I was assuming that was the main focus of the WG.
You need to be about 10,000% more explicit here (and in the charter) about what the WG is doing. I had to read the google doc to figure it out.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Appreciate the feedback Dan. I took a stab at changing some of the language in a way I hoped would align better with the original doc, and help clarify things. I specifically removed the text you quoted. Let me know what you think. However if you still think it needs more work, if you could point out all the exact text you find confusing, or suggest explicit things that need to be added in your opinion, that would be greatly appreciated as I did struggle a bit with where I should be focusing.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that scope, I would still assume that inference extension was in-scope for this WG, given that it is very much part of "load-balancing, routing and related features that support networking for AI use cases" and "policies,
filters, and extensions that support AI traffic management".
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think at the end of the day it's not that the GIE is completely and totally out of scope, in fact this WG in theory might make proposals to it, I think it's more that as soon as we would touch anything that's model serving to support networking functionality, we have to work with WG serving. I don't know if there's a fantastic way to describe this nuance in a few words, which is why we have the section below. Can we move forward with that section trying to explain the nuance? Or do you think we still need changes to the high level description? Do you have any suggestions on what would read better to you?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"what would read better" is the text in the google doc.
I don't have any suggestions on how to make that multi-paragraph explanation shorter, but "it's hard to explain well what our WG is for" is not a good reason for explaining it badly.
Even just saying "the scope is everything involving Gateway API and AI, except for Gateway API Inference Extension, which is out of scope" would be an improvement. Because otherwise, basically everyone is going to assume that the one existing AI-related k8s feature with "Gateway" in its name is supposed to be part of the "AI Gateway" WG... (Right?)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Alright thank you for the feedback, it's slightly modified but I worked in the bulk of the language from the document here in hopes of that providing more clarity. LMKWYT! 🖖
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, that's great and makes it much clearer
+1 on chartering and SIG Network sponsorship
I’ve put this on the agenda for our SIG call tomorrow and I’m aiming for a decision by Wednesday. |
@soltysh we now have another SIG Network +1. All that remains is SIG MC's decision. If they decide to join we'll incorporate scope updates to reflect that (the language should remain familiar as its still strongly networking focused and the "egress use case" is already relevant in a multi-cluster scenario). Let us know if there's anything else you'd like from us, otherwise we'll just plan on checking back in Wednesday. 🖖 |
Thanks @shaneutt for clarifying the scope. +1 looks good and looking forward to this one. |
There are some more questions I left just now, but correct the major blocker is SIG Multicluster decision. After that I'll poke the steering for final decision. |
SIG-Multicluster is willing to sponsor the working group. We’re particularly keen to avoid duplicating effort and learning from past endeavours, for example around |
Great! We've added multi-cluster as in-scope in the charter and marked you as a stakeholder, thanks for the update! 🎉 |
@soltysh thanks for the continued review and feedback. At this point we think we have everything covered and are ready to go, please let us know if there's anything else we need to get done first! 🖖 |
Signed-off-by: Shane Utt <[email protected]>
This PR requests the creation of the "AI Gateway Working Group" as discussed and defined throughout: