Skip to content

Upgrade the inferencePool selector to a struct from a map. #1330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 8 additions & 7 deletions api/v1/inferencepool_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,15 +51,16 @@ type InferencePoolList struct {

// InferencePoolSpec defines the desired state of InferencePool
type InferencePoolSpec struct {
// Selector defines a map of labels to watch model server Pods
// that should be included in the InferencePool.
// In some cases, implementations may translate this field to a Service selector, so this matches the simple
// map used for Service selectors instead of the full Kubernetes LabelSelector type.
// If specified, it will be applied to match the model server pods in the same namespace as the InferencePool.
// Cross namesoace selector is not supported.
// Selector determines which Pods are members of this inference pool.
// It matches Pods by their labels only within the same namespace; cross-namespace
// selection is not supported.
//
// The structure of this LabelSelector is intentionally simple to be compatible
// with Kubernetes Service selectors, as some implementations may translate
// this configuration into a Service resource.
//
// +kubebuilder:validation:Required
Selector map[LabelKey]LabelValue `json:"selector"`
Selector LabelSelector `json:"selector"`

// TargetPortNumber defines the port number to access the selected model server Pods.
// The number must be in the range 1 to 65535.
Expand Down
10 changes: 10 additions & 0 deletions api/v1/shared_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -127,3 +127,13 @@ type LabelKey string
// +kubebuilder:validation:MaxLength=63
// +kubebuilder:validation:Pattern=`^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?$`
type LabelValue string

// LabelSelector defines a query for resources based on their labels.
// This simplified version uses only the matchLabels field.
type LabelSelector struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe call this SimpleLabelSelector or something similar so it's clear that this is a subset of the OSS type of the same name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we foresee adding more fields to this struct? My impression is we want to leave the possibility to grow this struct to the real LabelSelector

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't change the name for now, because my understanding is that this struct could grow to be not "simple". I keep the name as-is for now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @zetxqx. Otherwise, a)SimpleLabelSelector will no longer be a simple selector when MatchExpressions is added or b) we will have to create a new LabelSelector type that mirrors upstream and implementations will have to refactor to use the new type.

// matchLabels contains a set of required {key,value} pairs.
// An object must match every label in this map to be selected.
// The matching logic is an AND operation on all entries.
// +optional
MatchLabels map[LabelKey]LabelValue `json:"matchLabels,omitempty" protobuf:"bytes,1,rep,name=matchLabels"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we want MatchLabels to mirror upstream?

	// matchLabels is a map of {key,value} pairs. A single {key,value} in the matchLabels
	// map is equivalent to an element of matchExpressions, whose key field is "key", the
	// operator is "In", and the values array contains only "value". The requirements are ANDed.
	// +optional
	MatchLabels map[string]string `json:"matchLabels,omitempty" protobuf:"bytes,1,rep,name=matchLabels"`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, I wanted to leverage the LabelKey and LabelValue because I saw they have some kubebuiler validation on it. Does that need to be kept? If not, I'll change to map[string]string for the simplicity.

// +kubebuilder:validation:MinLength=1
// +kubebuilder:validation:MaxLength=253
// +kubebuilder:validation:Pattern=`^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?([A-Za-z0-9][-A-Za-z0-9_.]{0,61})?[A-Za-z0-9]$`
type LabelKey string
// +kubebuilder:validation:MinLength=0
// +kubebuilder:validation:MaxLength=63
// +kubebuilder:validation:Pattern=`^(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?$`
type LabelValue string

}
30 changes: 23 additions & 7 deletions api/v1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

143 changes: 143 additions & 0 deletions apix/v1alpha2/inferencepool_conversion.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,143 @@
/*
Copyright 2025 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1alpha2

import (
"fmt"

metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
runtime "k8s.io/apimachinery/pkg/runtime"
v1 "sigs.k8s.io/gateway-api-inference-extension/api/v1"
)

// ConvertTo converts this InferencePool (v1alpha2) to the v1 version.
func (src *InferencePool) ConvertTo() (*v1.InferencePool, error) {
if src == nil {
return nil, nil
}

v1EndPointPickerConfig, err := convertEndpointPickerConfigToV1(&src.Spec.EndpointPickerConfig)
if err != nil {
return nil, err
}
v1Status, err := converStatusToV1(src.Status)
if err != nil {
return nil, err
}
dst := &v1.InferencePool{
TypeMeta: src.TypeMeta,
ObjectMeta: src.ObjectMeta,
Spec: v1.InferencePoolSpec{
TargetPortNumber: src.Spec.TargetPortNumber,
EndpointPickerConfig: *v1EndPointPickerConfig,
},
Status: *v1Status,
}
if src.Spec.Selector != nil {
dst.Spec.Selector.MatchLabels = make(map[v1.LabelKey]v1.LabelValue, len(src.Spec.Selector))
for k, v := range src.Spec.Selector {
dst.Spec.Selector.MatchLabels[v1.LabelKey(k)] = v1.LabelValue(v)
}
}
return dst, nil
}

// ConvertFrom converts from the v1 version to this version (v1alpha2).
func ConvertFrom(src *v1.InferencePool) (*InferencePool, error) {
if src == nil {
return nil, nil
}

endPointPickerConfig, err := convertEndpointPickerConfigFromV1(&src.Spec.EndpointPickerConfig)
if err != nil {
return nil, err
}
status, err := converStatusFromV1(src.Status)
if err != nil {
return nil, err
}
dst := &InferencePool{
TypeMeta: metav1.TypeMeta{
Kind: "InferencePool",
APIVersion: "inference.networking.x-k8s.io/v1alpha2",
},
ObjectMeta: src.ObjectMeta,
Spec: InferencePoolSpec{
TargetPortNumber: src.Spec.TargetPortNumber,
EndpointPickerConfig: *endPointPickerConfig,
},
Status: *status,
}

if src.Spec.Selector.MatchLabels != nil {
dst.Spec.Selector = make(map[LabelKey]LabelValue, len(src.Spec.Selector.MatchLabels))
for k, v := range src.Spec.Selector.MatchLabels {
dst.Spec.Selector[LabelKey(k)] = LabelValue(v)
}
}

return dst, nil
}

func converStatusToV1(src InferencePoolStatus) (*v1.InferencePoolStatus, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we use manual conversation for label selector, I'm wondering whether it makes more sense to use manual conversion for all fields now. I know the code maybe a little bit cumbersome. But maybe AI tool can help. I know such cumbersome manual conversion is widely used in OSS, see https://github.com/google/knative-gcp/blob/4a435faedc46726299800e1cdf2ad998c357b25b/pkg/apis/convert/conversion_helper.go

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's leave it to go readability reviewer @liu-cong to make a call.

u, err := toUnstructured(&src)
if err != nil {
return nil, err
}
return convert[v1.InferencePoolStatus](u)
}

func converStatusFromV1(src v1.InferencePoolStatus) (*InferencePoolStatus, error) {
u, err := toUnstructured(&src)
if err != nil {
return nil, err
}
return convert[InferencePoolStatus](u)
}

func convertEndpointPickerConfigToV1(src *EndpointPickerConfig) (*v1.EndpointPickerConfig, error) {
u, err := toUnstructured(&src)
if err != nil {
return nil, err
}
return convert[v1.EndpointPickerConfig](u)
}

func convertEndpointPickerConfigFromV1(src *v1.EndpointPickerConfig) (*EndpointPickerConfig, error) {
u, err := toUnstructured(&src)
if err != nil {
return nil, err
}
return convert[EndpointPickerConfig](u)
}

func toUnstructured(obj any) (*unstructured.Unstructured, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you use the common util in https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/pkg/common/convert.go? At the same time, I think you can remove the

var ToInferencePool = convert[v1.InferencePool]
var ToXInferencePool = convert[v1alpha2.InferencePool]

in the common file.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I deleted the common/convert.go and moved it here.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think if you still need to use func toUnstructured(obj any) (*unstructured.Unstructured, error) in the codebase, it's better to leave it in common package instead of specific apix package as they are common util. But we can wait until cong to make a call to see whether we need to do manual conversion for all fields.

u, err := runtime.DefaultUnstructuredConverter.ToUnstructured(obj)
if err != nil {
return nil, err
}
return &unstructured.Unstructured{Object: u}, nil
}

func convert[T any](u *unstructured.Unstructured) (*T, error) {
var res T
if err := runtime.DefaultUnstructuredConverter.FromUnstructured(u.Object, &res); err != nil {
return nil, fmt.Errorf("error converting unstructured to T: %v", err)
}
return &res, nil
}
Loading