diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES
index 2f6915cfaed..592ce4e6cc8 100644
--- a/OWNERS_ALIASES
+++ b/OWNERS_ALIASES
@@ -130,6 +130,13 @@ aliases:
     - mfahlandt
     - ritazh
     - terrytangyuan
+  wg-ai-gateway-leads:
+    - keithmattix
+    - kflynn
+    - kfswain
+    - nirrozenbaum
+    - shaneutt
+    - xunzhuo
   wg-ai-integration-leads:
     - ardaguclu
     - rushmash91
diff --git a/sig-list.md b/sig-list.md
index 91410112069..456d8338dc1 100644
--- a/sig-list.md
+++ b/sig-list.md
@@ -62,6 +62,7 @@ When the need arises, a [new SIG can be created](sig-wg-lifecycle.md)
 | Name | Label | Stakeholder SIGs |Organizers | Contact | Meetings |
 |------|-------|------------------|-----------|---------|----------|
 |[AI Conformance](wg-ai-conformance/README.md)|[ai-conformance](https://github.com/kubernetes/kubernetes/labels/wg%2Fai-conformance)|* Architecture<br>* Testing<br>|* [Janet Kuo](https://github.com/janetkuo), Google<br>* [Mario Fahlandt](https://github.com/mfahlandt), Kubermatic GmbH<br>* [Rita Zhang](https://github.com/ritazh), Microsoft<br>* [Yuan Tang](https://github.com/terrytangyuan), Red Hat<br>|* [Slack](https://kubernetes.slack.com/messages/wg-ai-conformance)<br>* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-ai-conformance)|* Regular WG Meeting: [Thursdays at 10:00 PT (Pacific Time) (weekly)]()<br>
+|[AI Gateway](wg-ai-gateway/README.md)|[ai-gateway](https://github.com/kubernetes/kubernetes/labels/wg%2Fai-gateway)|* Network<br>|* [Keith Mattix](https://github.com/keithmattix), Microsoft<br>* [Flynn](https://github.com/kflynn), Buoyant<br>* [Kellen Swain](https://github.com/kfswain), Google<br>* [Nir Rozenbaum](https://github.com/nirrozenbaum), IBM<br>* [Shane Utt](https://github.com/shaneutt), Red Hat<br>* [Xunzhuo](https://github.com/xunzhuo), Tencent<br>|* [Slack](https://kubernetes.slack.com/messages/wg-ai-gateway)<br>* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-ai-gateway)|* WG AI Gateway Bi-Weekly Meeting (Earlier Option): [Mondays at 12PM UTC (bi-weekly)]()<br>* WG AI Gateway Bi-Weekly Meeting (Later Option): [Thursdays at 6PM UTC (bi-weekly)]()<br>
 |[AI Integration](wg-ai-integration/README.md)|[ai-integration](https://github.com/kubernetes/kubernetes/labels/wg%2Fai-integration)|* API Machinery<br>* Apps<br>* Architecture<br>* Auth<br>* CLI<br>|* [Arda Guclu](https://github.com/ardaguclu), Red Hat<br>* [Arush Sharma](https://github.com/rushmash91), Amazon<br>* [Zvonko Kaiser](https://github.com/zvonkok), NVIDIA<br>|* [Slack](https://kubernetes.slack.com/messages/wg-ai-integration)<br>* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-ai-integration)|* WG AI Integration Weekly Meeting: [Wednesdays at 10:00 PT (Pacific Time) (weekly)]()<br>
 |[Batch](wg-batch/README.md)|[batch](https://github.com/kubernetes/kubernetes/labels/wg%2Fbatch)|* Apps<br>* Autoscaling<br>* Node<br>* Scheduling<br>|* [Kevin Hannon](https://github.com/kannon92), Red Hat<br>* [Marcin Wielgus](https://github.com/mwielgus), Google<br>* [Maciej Szulik](https://github.com/soltysh), Defense Unicorns<br>* [Swati Sehgal](https://github.com/swatisehgal), Red Hat<br>|* [Slack](https://kubernetes.slack.com/messages/wg-batch)<br>* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-batch)|* Regular Meeting ([calendar](https://calendar.google.com/calendar/embed?src=8ulop9k0jfpuo0t7kp8d9ubtj4%40group.calendar.google.com)): [Thursdays (starting February 15th 2024)s at 3PM CET (Central European Time) (monthly)](https://zoom.us/j/98329676612?pwd=c0N2bVV1aTh2VzltckdXSitaZXBKQT09)<br>
 |[Data Protection](wg-data-protection/README.md)|[data-protection](https://github.com/kubernetes/kubernetes/labels/wg%2Fdata-protection)|* Apps<br>* Storage<br>|* [Xing Yang](https://github.com/xing-yang), VMware<br>* [Xiangqian Yu](https://github.com/yuxiangqian), Google<br>|* [Slack](https://kubernetes.slack.com/messages/wg-data-protection)<br>* [Mailing List](https://groups.google.com/a/kubernetes.io/g/wg-data-protection)|* Regular WG Meeting: [Wednesdays at 9:00 PT (Pacific Time) (bi-weekly)](https://zoom.us/j/6933410772)<br>
diff --git a/sig-network/README.md b/sig-network/README.md
index 7e9b40370ff..147003a553c 100644
--- a/sig-network/README.md
+++ b/sig-network/README.md
@@ -73,6 +73,7 @@ subprojects, and resolve cross-subproject technical issues and decisions.
 ## Working Groups
 
 The following [working groups][working-group-definition] are sponsored by sig-network:
+* [WG AI Gateway](/wg-ai-gateway)
 * [WG Device Management](/wg-device-management)
 * [WG Node Lifecycle](/wg-node-lifecycle)
 * [WG Serving](/wg-serving)
diff --git a/sigs.yaml b/sigs.yaml
index ce9f06dec5a..e9f60acc698 100644
--- a/sigs.yaml
+++ b/sigs.yaml
@@ -3561,6 +3561,57 @@ workinggroups:
     liaison:
       github: pohly
       name: Patrick Ohly
+- dir: wg-ai-gateway
+  name: AI Gateway
+  mission_statement: >
+    The AI Gateway Working Group focuses on the intersection of AI and networking,
+    particularly in the context of extending load-balancer, gateway and proxy technologies
+    to manage and route traffic for AI Inference.
+
+  charter_link: charter.md
+  stakeholder_sigs:
+  - Network
+  label: ai-gateway
+  leadership:
+    chairs:
+    - github: keithmattix
+      name: Keith Mattix
+      company: Microsoft
+      email: keithmattix2@gmail.com
+    - github: kflynn
+      name: Flynn
+      company: Buoyant
+      email: flynn@buoyant.io
+    - github: kfswain
+      name: Kellen Swain
+      company: Google
+      email: kfswain@google.com
+    - github: nirrozenbaum
+      name: Nir Rozenbaum
+      company: IBM
+      email: nirro@il.ibm.com
+    - github: shaneutt
+      name: Shane Utt
+      company: Red Hat
+      email: sutt@redhat.com
+    - github: xunzhuo
+      name: Xunzhuo
+      company: Tencent
+      email: mixdeers@gmail.com
+  meetings:
+  - description: WG AI Gateway Bi-Weekly Meeting (Earlier Option)
+    day: Monday
+    time: 12PM
+    tz: UTC
+    frequency: bi-weekly
+  - description: WG AI Gateway Bi-Weekly Meeting (Later Option)
+    day: Thursday
+    time: 6PM
+    tz: UTC
+    frequency: bi-weekly
+  contact:
+    slack: wg-ai-gateway
+    mailing_list: https://groups.google.com/a/kubernetes.io/g/wg-ai-gateway
 - dir: wg-ai-integration
   name: AI Integration
   mission_statement: >
diff --git a/wg-ai-gateway/OWNERS b/wg-ai-gateway/OWNERS
new file mode 100644
index 00000000000..64c0bd84c02
--- /dev/null
+++ b/wg-ai-gateway/OWNERS
@@ -0,0 +1,8 @@
+# See the OWNERS docs at https://go.k8s.io/owners
+
+reviewers:
+  - wg-ai-gateway-leads
+approvers:
+  - wg-ai-gateway-leads
+labels:
+  - wg/ai-gateway
diff --git a/wg-ai-gateway/README.md b/wg-ai-gateway/README.md
new file mode 100644
index 00000000000..8662947bb83
--- /dev/null
+++ b/wg-ai-gateway/README.md
@@ -0,0 +1,38 @@
+<!---
+This is an autogenerated file!
+
+Please do not edit this file directly, but instead make changes to the
+sigs.yaml file in the project root.
+
+To understand how this file is generated, see https://git.k8s.io/community/generator/README.md
+--->
+# AI Gateway Working Group
+
+The AI Gateway Working Group focuses on the intersection of AI and networking, particularly in the context of extending load-balancer, gateway and proxy technologies to manage and route traffic for AI Inference.
+
+The [charter](charter.md) defines the scope and governance of the AI Gateway Working Group.
+
+## Stakeholder SIGs
+* [SIG Network](/sig-network)
+
+## Meetings
+*Joining the [mailing list](https://groups.google.com/a/kubernetes.io/g/wg-ai-gateway) for the group will typically add invites for the following meetings to your calendar.*
+* WG AI Gateway Bi-Weekly Meeting (Earlier Option): [Mondays at 12PM UTC]() (bi-weekly). [Convert to your timezone](http://www.thetimezoneconverter.com/?t=12PM&tz=UTC).
+* WG AI Gateway Bi-Weekly Meeting (Later Option): [Thursdays at 6PM UTC]() (bi-weekly). [Convert to your timezone](http://www.thetimezoneconverter.com/?t=6PM&tz=UTC).
+
+## Organizers
+
+* Keith Mattix (**[@keithmattix](https://github.com/keithmattix)**), Microsoft
+* Flynn (**[@kflynn](https://github.com/kflynn)**), Buoyant
+* Kellen Swain (**[@kfswain](https://github.com/kfswain)**), Google
+* Nir Rozenbaum (**[@nirrozenbaum](https://github.com/nirrozenbaum)**), IBM
+* Shane Utt (**[@shaneutt](https://github.com/shaneutt)**), Red Hat
+* Xunzhuo (**[@xunzhuo](https://github.com/xunzhuo)**), Tencent
+
+## Contact
+- Slack: [#wg-ai-gateway](https://kubernetes.slack.com/messages/wg-ai-gateway)
+- [Mailing list](https://groups.google.com/a/kubernetes.io/g/wg-ai-gateway)
+- [Open Community Issues/PRs](https://github.com/kubernetes/community/labels/wg%2Fai-gateway)
+<!-- BEGIN CUSTOM CONTENT -->
+
+<!-- END CUSTOM CONTENT -->
diff --git a/wg-ai-gateway/charter.md b/wg-ai-gateway/charter.md
new file mode 100644
index 00000000000..5561129aa41
--- /dev/null
+++ b/wg-ai-gateway/charter.md
@@ -0,0 +1,101 @@
+# WG AI Gateway Charter
+
+This charter adheres to the conventions described in the [Kubernetes Charter
+README] and uses the Roles and Organization Management outlined in
+[wg-governance].
+
+[wg-governance]:https://github.com/kubernetes/community/blob/master/committee-steering/governance/wg-governance.md
+[Kubernetes Charter README]:https://github.com/kubernetes/community/blob/master/committee-steering/governance/README.md
+
+## Scope
+
+The AI Gateway Working Group focuses on the intersection of AI and
+networking, particularly in the context of extending load-balancer, gateway
+and proxy technologies to manage and route traffic for AI Inference.
+
+This working group will define terms like "AI Gateway" within the context of
+Kubernetes and key use cases for users and implementations. It will propose
+deliverables that need to be adopted in order to serve AI Inference on
+Kubernetes.
+
+This comes at a time where there is a proliferation of "AI Gateways" being used
+for AI Inference, and a strong need for focus and collaboration to ensure
+standards around this space so that Kubernetes users get the features they need
+in a consistent way on the platform.
+
+### In Scope
+
+Overall guidance for the WG is to control scope as much as is feasible. The WG
+should avoid AI-specific functionality where it can: instead favoring the
+addition of provisions that help with AI use-cases, but are otherwise normal
+networking facilities. Under that guidance, the following is in-scope:
+
+* Providing definitions for networking related AI terms in a Kubernetes
+  context.
+
+* Defining important AI networking use-cases for Kubernetes users.
+
+* Determining which common features and capabilities in the "AI Gateway" space
+  need to be covered by Kubernetes standards and APIs according to user and
+  implementation needs.
+
+* Creating proposals for "AI Gateway" features and capabilities to the
+  appropriate sub-projects.
+
+* Propose new sub-projects if existing sub-projects are not sufficient.
+
+### Out of Scope
+
+* Developing whole "AI Gateway" solutions. This group will focus on
+  enabling existing and new solutions to be more easily deployed and managed on
+  Kubernetes, not adding any new production solutions maintained thereafter by
+  upstream Kubernetes.
+
+* Any specific kind of hardware support is generally out of scope.
+
+* This group will not cover the entire spectrum of networking for AI. For
+  instance: RDMA networks are generally out of scope.
+
+## Deliverables
+
+* A compendium of AI related networking definitions (e.g. "AI Gateway") and a
+  key use-cases for Kubernetes users.
+
+* Provide a space for collaboration and experimentation to determine the most
+  viable features and capabilities that Kubernetes should support. If there is
+  strong consensus on any particular ideas, the WG will facilitate and
+  coordinate the delivery of proposals in the appropriate areas.
+
+## Stakeholders
+
+* SIG Network
+
+### Related WGs
+
+* WG Serving - The domain of WG Serving is AI Workloads, which can be served by
+  some of the networking support we want to add. When we have proposals that
+  are strongly relevant to serving, we will loop them in so they can provide
+  feedback.
+
+## Roles and Organization Management
+
+This working group adheres to the Roles and Organization Management outlined in
+[wg-governance] and opts-in to updates and modifications to [wg-governance].
+
+[wg-governance]:https://github.com/kubernetes/community/blob/master/committee-steering/governance/wg-governance.md
+
+## Exit Criteria
+
+The WG is done when its deliverables are complete, according to the defined
+scope and a list of key use cases and features agreed upon by the group.
+
+Ideally we want the lifecycle of the WG to go something like this:
+
+1. Determine definitions and key use cases for Kubernetes users and
+   implementations, and document those.
+2. Determine a list of key features that Kubernetes needs to best support the
+   defined use cases.
+3. For each feature in that list, make proposals which support them to the
+   appropriate sub-projects OR propose new sub-projects if deemed necessary.
+4. Once the feature list is complete, leave behind some guidance and best
+   practices for future implementations and then exit.