docs: use inference gateway terminology (#891)

* use inference gateway terminology

* fixed comments

* fixed comments

Author: capri-xiyue

site-src/concepts/api-overview.md

@@ -1,12 +1,24 @@
 # API Overview
 
 ## Background
-The Gateway API Inference Extension project is an extension of the Kubernetes Gateway API for serving Generative AI models on Kubernetes. Gateway API Inference Extension facilitates standardization of APIs for Kubernetes cluster operators and developers running generative AI inference, while allowing flexibility for underlying gateway implementations (such as Envoy Proxy) to iterate on mechanisms for optimized serving of models.
+Gateway API Inference Extension optimizes self-hosting Generative AI Models on Kubernetes. 
+It provides optimized load-balancing for self-hosted Generative AI Models on Kubernetes.
+The project’s goal is to improve and standardize routing to inference workloads across the ecosystem.
 
-<img src="/images/inference-overview.svg" alt="Overview of API integration" class="center" width="1000" />
+This is achieved by leveraging Envoy's [External Processing](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_proc_filter) to extend any gateway that supports both ext-proc and [Gateway API](https://github.com/kubernetes-sigs/gateway-api) into an [inference gateway](../index.md#concepts-and-definitions).
+This extension extends popular gateways like Envoy Gateway, kgateway, and GKE Gateway - to become [Inference Gateway](../index.md#concepts-and-definitions) -
+supporting inference platform teams self-hosting Generative Models (with a current focus on large language models) on Kubernetes.
+This integration makes it easy to expose and control access to your local [OpenAI-compatible chat completion endpoints](https://platform.openai.com/docs/api-reference/chat)
+to other workloads on or off cluster, or to integrate your self-hosted models alongside model-as-a-service providers 
+in a higher level **AI Gateways** like [LiteLLM](https://www.litellm.ai/), [Gloo AI Gateway](https://www.solo.io/products/gloo-ai-gateway), or [Apigee](https://cloud.google.com/apigee).
 
 ## API Resources
 
+Gateway API Inference Extension introduces two inference-focused API resources with distinct responsibilities, 
+each aligning with a specific user persona in the Generative AI serving workflow.
+
+<img src="/images/inference-overview.svg" alt="Overview of API integration" class="center" width="1000" />
+
 ### InferencePool
 
 InferencePool represents a set of Inference-focused Pods and an extension that will be used to route to them. Within the broader Gateway API resource model, this resource is considered a "backend". In practice, that means that you'd replace a Kubernetes Service with an InferencePool. This resource has some similarities to Service (a way to select Pods and specify a port), but has some unique capabilities. With InferencePool, you can configure a routing extension as well as inference-specific routing optimizations. For more information on this resource, refer to our [InferencePool documentation](/api-types/inferencepool) or go directly to the [InferencePool spec](/reference/spec/#inferencepool).

site-src/guides/index.md

@@ -1,4 +1,4 @@
-# Getting started with Gateway API Inference Extension
+# Getting started with an Inference Gateway
 
 ??? example "Experimental"
 
@@ -98,7 +98,7 @@ This quickstart guide is intended for engineers familiar with k8s and model serv
    kubectl apply -f https://github.com/kubernetes-sigs/gateway-api-inference-extension/raw/main/config/manifests/inferencepool-resources.yaml
    ```
 
-### Deploy Inference Gateway
+### Deploy an Inference Gateway
 
    Choose one of the following options to deploy an Inference Gateway.
 

site-src/guides/serve-multiple-genai-models.md

@@ -2,12 +2,12 @@
 A company wants to deploy multiple large language models (LLMs) to serve different workloads. 
 For example, they might want to deploy a Gemma3 model for a chatbot interface and a Deepseek model for a recommendation application. 
 The company needs to ensure optimal serving performance for these LLMs.
-Using Gateway API Inference Extension, you can deploy these LLMs on your cluster with your chosen accelerator configuration in an `InferencePool`. 
+By using an Inference Gateway, you can deploy these LLMs on your cluster with your chosen accelerator configuration in an `InferencePool`. 
 You can then route requests based on the model name (such as "chatbot" and "recommender") and the `Criticality` property.
 
 ## How
-The following diagram illustrates how Gateway API Inference Extension routes requests to different models based on the model name.
-The model name is extarcted by [Body-Based routing](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/bbr/README.md)
+The following diagram illustrates how an Inference Gateway routes requests to different models based on the model name.
+The model name is extracted by [Body-Based routing](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/bbr/README.md)
  from the request body to the header. The header is then matched to dispatch
  requests to different `InferencePool` (and their EPPs) instances.
 ![Serving multiple generative AI models](../images/serve-mul-gen-AI-models.png)

site-src/guides/serve-multiple-lora-adapters.md

@@ -1,11 +1,11 @@
 # Serve LoRA adapters on a shared pool
 A company wants to serve LLMs for document analysis and focuses on audiences in multiple languages, such as English and Spanish.
 They have a fine-tuned LoRA adapter for each language, but need to efficiently use their GPU and TPU capacity.
-You can use Gateway API Inference Extension to deploy dynamic LoRA fine-tuned adapters for each language (for example, `english-bot` and `spanish-bot`) on a common base model and accelerator.
+You can use an Inference Gateway to deploy dynamic LoRA fine-tuned adapters for each language (for example, `english-bot` and `spanish-bot`) on a common base model and accelerator.
 This lets you reduce the number of required accelerators by densely packing multiple models in a shared pool.
 
 ## How
-The following diagram illustrates how Gateway API Inference Extension serves multiple LoRA adapters on a shared pool.
+The following diagram illustrates how Inference Gateway serves multiple LoRA adapters on a shared pool.
 ![Serving LoRA adapters on a shared pool](../images/serve-LoRA-adapters.png)
 This example illustrates how you can densely serve multiple LoRA adapters with distinct workload performance objectives on a common InferencePool.
 ```yaml

site-src/index.md

@@ -1,8 +1,6 @@
 # Introduction
 
-Gateway API Inference Extension is an official Kubernetes project focused on
-extending [Gateway API](https://gateway-api.sigs.k8s.io/) with inference
-specific routing extensions.
+Gateway API Inference Extension is an official Kubernetes project that optimizes self-hosting Generative Models on Kubernetes.
 
 The overall resource model focuses on 2 new inference-focused
 [personas](/concepts/roles-and-personas) and corresponding resources that
@@ -11,20 +9,49 @@ they are expected to manage:
 <!-- Source: https://docs.google.com/presentation/d/11HEYCgFi-aya7FS91JvAfllHiIlvfgcp7qpi_Azjk4E/edit#slide=id.g292839eca6d_1_0 -->
 <img src="/images/resource-model.png" alt="Gateway API Inference Extension Resource Model" class="center" width="550" />
 
+## Concepts and Definitions
+
+The following specific terms to this project:
+
+- **Inference Gateway**: A proxy/load-balancer that has been coupled with the
+  EndPointer Picker extension. It provides optimized routing and load balancing for
+  serving Kubernetes self-hosted generative Artificial Intelligence (AI)
+  workloads. It simplifies the deployment, management, and observability of AI
+  inference workloads.
+- **Inference Scheduler**: An extendable component that makes decisions about which endpoint is optimal (best cost /
+  best performance) for an inference request based on `Metrics and Capabilities`
+  from [Model Serving](/docs/proposals/003-model-server-protocol/README.md).
+- **Metrics and Capabilities**: Data provided by model serving platforms about
+  performance, availability and capabilities to optimize routing. Includes
+  things like [Prefix Cache] status or [LoRA Adapters] availability.
+- **Endpoint Picker(EPP)**: An implementation of an `Inference Scheduler` with additional Routing, Flow, and Request Control layers to allow for sophisticated routing strategies. Additional info on the architecture of the EPP [here](https://github.com/kubernetes-sigs/gateway-api-inference-extension/tree/main/docs/proposals/0683-epp-architecture-proposal).
+
+[Inference Gateway]:#concepts-and-definitions
+
 ## Key Features 
-Gateway API Inference Extension, along with a reference implementation in Envoy Proxy, provides the following key features: 
+Gateway API Inference Extension optimizes self-hosting Generative AI Models on Kubernetes.
+It provides optimized load-balancing for self-hosted Generative AI Models on Kubernetes.
+The project’s goal is to improve and standardize routing to inference workloads across the ecosystem.
+
+This is achieved by leveraging Envoy's [External Processing](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/ext_proc_filter) to extend any gateway that supports both ext-proc and [Gateway API](https://github.com/kubernetes-sigs/gateway-api) into an [inference gateway](../index.md#concepts-and-definitions).
+This extension extends popular gateways like Envoy Gateway, kgateway, and GKE Gateway - to become [Inference Gateway](../index.md#concepts-and-definitions) -
+supporting inference platform teams self-hosting Generative Models (with a current focus on large language models) on Kubernetes.
+This integration makes it easy to expose and control access to your local [OpenAI-compatible chat completion endpoints](https://platform.openai.com/docs/api-reference/chat)
+to other workloads on or off cluster, or to integrate your self-hosted models alongside model-as-a-service providers
+in a higher level **AI Gateways** like [LiteLLM](https://www.litellm.ai/), [Gloo AI Gateway](https://www.solo.io/products/gloo-ai-gateway), or [Apigee](https://cloud.google.com/apigee).
 
-- **Model-aware routing**: Instead of simply routing based on the path of the request, Gateway API Inference Extension allows you to route to models based on the model names. This is enabled by support for GenAI Inference API specifications (such as OpenAI API) in the gateway implementations such as in Envoy Proxy. This model-aware routing also extends to Low-Rank Adaptation (LoRA) fine-tuned models.
 
-- **Serving priority**: Gateway API Inference Extension allows you to specify the serving priority of your models. For example, you can specify that your models for online inference of chat tasks (which is more latency sensitive) have a higher [*Criticality*](/reference/spec/#criticality) than a model for latency tolerant tasks such as a summarization. 
+- **Model-aware routing**: Instead of simply routing based on the path of the request, an **[inference gateway]** allows you to route to models based on the model names. This is enabled by support for GenAI Inference API specifications (such as OpenAI API) in the gateway implementations such as in Envoy Proxy. This model-aware routing also extends to Low-Rank Adaptation (LoRA) fine-tuned models.
 
-- **Model rollouts**:  Gateway API Inference Extension allows you to incrementally roll out new model versions by traffic splitting definitions based on the model names. 
+- **Serving priority**: an **[inference gateway]** allows you to specify the serving priority of your models. For example, you can specify that your models for online inference of chat tasks (which is more latency sensitive) have a higher [*Criticality*](/reference/spec/#criticality) than a model for latency tolerant tasks such as a summarization. 
 
-- **Extensibility for Inference Services**: Gateway API Inference Extension defines extensibility pattern for additional Inference services to create bespoke routing capabilities should out of the box solutions not fit your needs.
+- **Model rollouts**:  an **[inference gateway]** allows you to incrementally roll out new model versions by traffic splitting definitions based on the model names. 
 
+- **Extensibility for Inference Services**: an **[inference gateway]** defines extensibility pattern for additional Inference services to create bespoke routing capabilities should out of the box solutions not fit your needs.
 
-- **Customizable Load Balancing for Inference**: Gateway API Inference Extension defines a pattern for customizable load balancing and request routing that is optimized for Inference. Gateway API Inference Extension provides a reference implementation of model endpoint picking leveraging metrics emitted from the model servers. This endpoint picking mechanism can be used in lieu of traditional load balancing mechanisms. Model Server-aware load balancing ("smart" load balancing as its sometimes referred to in this repo) has been proven to reduce the serving latency and improve utilization of accelerators in your clusters.
+- **Customizable Load Balancing for Inference**: an **[inference gateway]** defines a pattern for customizable load balancing and request routing that is optimized for Inference. An **[inference gateway]** provides a reference implementation of model endpoint picking leveraging metrics emitted from the model servers. This endpoint picking mechanism can be used in lieu of traditional load balancing mechanisms. Model Server-aware load balancing ("smart" load balancing as its sometimes referred to in this repo) has been proven to reduce the serving latency and improve utilization of accelerators in your clusters.
 
+By achieving these, the project aims to reduce latency and improve accelerator (GPU) utilization for AI workloads.
 
 ## API Resources
 
@@ -42,7 +69,7 @@ that are relevant to this project:
 Gateway API has [more than 25
 implementations](https://gateway-api.sigs.k8s.io/implementations/). As this
 pattern stabilizes, we expect a wide set of these implementations to support
-this project.
+this project to become an **[inference gateway]**
 
 ### Endpoint Picker
 
@@ -71,16 +98,16 @@ to any Gateway API users or implementers.
 2. If the request should be routed to an InferencePool, the Gateway will forward
 the request information to the endpoint selection extension for that pool.
 
-3. The extension will fetch metrics from whichever portion of the InferencePool
+3. The inference gateway will fetch metrics from whichever portion of the InferencePool
 endpoints can best achieve the configured objectives. Note that this kind of
-metrics probing may happen asynchronously, depending on the extension.
+metrics probing may happen asynchronously, depending on the inference gateway.
 
-4. The extension will instruct the Gateway which endpoint the request should be
+4. The inference gateway will instruct the Gateway which endpoint the request should be
 routed to.
 
 5. The Gateway will route the request to the desired endpoint.
 
-<img src="/images/request-flow.png" alt="Gateway API Inference Extension Request Flow" class="center" />
+<img src="/images/request-flow.png" alt="Inference Gateway Request Flow" class="center" />
 
 
 ## Who is working on Gateway API Inference Extension?