Skip to content

Add leader service #690

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
TaeSooRhee-moreh wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Add leader service #690

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
d49b8c2
feature: add leader service with boolean value
TaeSooRhee-moreh Oct 22, 2025
752fa35
feature: add const for service postfix
TaeSooRhee-moreh Oct 22, 2025
7e2244c
feature: add test for add leader service
TaeSooRhee-moreh Oct 22, 2025
1a1039d
feature: change addLeaderService to LeaderServicePort
TaeSooRhee-moreh Oct 24, 2025
ae70a23
feature: add omitempty at LeaderServicePort
TaeSooRhee-moreh Oct 29, 2025
f7d16f4
feature: change name to LeaderServicePorts
TaeSooRhee-moreh Nov 3, 2025
0e2e095
feature: change examples to use leaderServicePorts
TaeSooRhee-moreh Nov 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions api/leaderworkerset/v1/leaderworkerset_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -242,6 +242,11 @@ type NetworkConfig struct {
// the headless service, defaults to shared
// +kubebuilder:validation:Enum={Shared,UniquePerReplica}
SubdomainPolicy *SubdomainPolicy `json:"subdomainPolicy"`
// LeaderServicePorts indicates whether to create a ClusterIP service for
// the leader pods. If specified, a ClusterIP service will be created for
// the leader pods using the given ports.
// +optional
LeaderServicePorts []corev1.ServicePort `json:"leaderServicePorts,omitempty"`
}

type SubdomainPolicy string
Expand All @@ -259,6 +264,8 @@ const (
SubdomainUniquePerReplica SubdomainPolicy = "UniquePerReplica"
)

const LeaderServicePostfix string = "-leader"

// RollingUpdateConfiguration defines the parameters to be used for RollingUpdateStrategyType.
type RollingUpdateConfiguration struct {
// Partition indicates the ordinal at which the lws should be partitioned for updates.
Expand Down
7 changes: 7 additions & 0 deletions api/leaderworkerset/v1/zz_generated.deepcopy.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 75 additions & 0 deletions charts/lws/templates/crds/leaderworkerset.x-k8s.io_leaderworkersets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16612,6 +16612,81 @@ spec:
networkConfig:
description: NetworkConfig defines the network configuration of the group
properties:
leaderServicePorts:
description: |-
LeaderServicePorts indicates whether to create a ClusterIP service for
the leader pods. If specified, a ClusterIP service will be created for
the leader pods using the given ports.
items:
description: ServicePort contains information on service's port.
properties:
appProtocol:
description: |-
The application protocol for this port.
This is used as a hint for implementations to offer richer behavior for protocols that they understand.
This field follows standard Kubernetes label syntax.
Valid values are either:

* Un-prefixed protocol names - reserved for IANA standard service names (as per
RFC-6335 and https://www.iana.org/assignments/service-names).

* Kubernetes-defined prefixed names:
* 'kubernetes.io/h2c' - HTTP/2 prior knowledge over cleartext as described in https://www.rfc-editor.org/rfc/rfc9113.html#name-starting-http-2-with-prior-
* 'kubernetes.io/ws' - WebSocket over cleartext as described in https://www.rfc-editor.org/rfc/rfc6455
* 'kubernetes.io/wss' - WebSocket over TLS as described in https://www.rfc-editor.org/rfc/rfc6455

* Other protocols should use implementation-defined prefixed names such as
mycompany.com/my-custom-protocol.
type: string
name:
description: |-
The name of this port within the service. This must be a DNS_LABEL.
All ports within a ServiceSpec must have unique names. When considering
the endpoints for a Service, this must match the 'name' field in the
EndpointPort.
Optional if only one ServicePort is defined on this service.
type: string
nodePort:
description: |-
The port on each node on which this service is exposed when type is
NodePort or LoadBalancer. Usually assigned by the system. If a value is
specified, in-range, and not in use it will be used, otherwise the
operation will fail. If not specified, a port will be allocated if this
Service requires one. If this field is specified when creating a
Service which does not need it, creation will fail. This field will be
wiped when updating a Service to no longer need it (e.g. changing type
from NodePort to ClusterIP).
More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
format: int32
type: integer
port:
description: The port that will be exposed by this service.
format: int32
type: integer
protocol:
default: TCP
description: |-
The IP protocol for this port. Supports "TCP", "UDP", and "SCTP".
Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: |-
Number or name of the port to access on the pods targeted by the service.
Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named port in the
target Pod's container ports. If this is not specified, the value
of the 'port' field is used (an identity map).
This field is ignored for services with clusterIP=None, and should be
omitted or set equal to the 'port' field.
More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service
x-kubernetes-int-or-string: true
required:
- port
type: object
type: array
subdomainPolicy:
description: |-
SubdomainPolicy determines the policy that will be used when creating
Expand Down
14 changes: 13 additions & 1 deletion client-go/applyconfiguration/leaderworkerset/v1/networkconfig.go
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

75 changes: 75 additions & 0 deletions config/crd/bases/leaderworkerset.x-k8s.io_leaderworkersets.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17601,6 +17601,81 @@ spec:
description: NetworkConfig defines the network configuration of the
group
properties:
leaderServicePorts:
description: |-
LeaderServicePorts indicates whether to create a ClusterIP service for
the leader pods. If specified, a ClusterIP service will be created for
the leader pods using the given ports.
items:
description: ServicePort contains information on service's port.
properties:
appProtocol:
description: |-
The application protocol for this port.
This is used as a hint for implementations to offer richer behavior for protocols that they understand.
This field follows standard Kubernetes label syntax.
Valid values are either:

* Un-prefixed protocol names - reserved for IANA standard service names (as per
RFC-6335 and https://www.iana.org/assignments/service-names).

* Kubernetes-defined prefixed names:
* 'kubernetes.io/h2c' - HTTP/2 prior knowledge over cleartext as described in https://www.rfc-editor.org/rfc/rfc9113.html#name-starting-http-2-with-prior-
* 'kubernetes.io/ws' - WebSocket over cleartext as described in https://www.rfc-editor.org/rfc/rfc6455
* 'kubernetes.io/wss' - WebSocket over TLS as described in https://www.rfc-editor.org/rfc/rfc6455

* Other protocols should use implementation-defined prefixed names such as
mycompany.com/my-custom-protocol.
type: string
name:
description: |-
The name of this port within the service. This must be a DNS_LABEL.
All ports within a ServiceSpec must have unique names. When considering
the endpoints for a Service, this must match the 'name' field in the
EndpointPort.
Optional if only one ServicePort is defined on this service.
type: string
nodePort:
description: |-
The port on each node on which this service is exposed when type is
NodePort or LoadBalancer. Usually assigned by the system. If a value is
specified, in-range, and not in use it will be used, otherwise the
operation will fail. If not specified, a port will be allocated if this
Service requires one. If this field is specified when creating a
Service which does not need it, creation will fail. This field will be
wiped when updating a Service to no longer need it (e.g. changing type
from NodePort to ClusterIP).
More info: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
format: int32
type: integer
port:
description: The port that will be exposed by this service.
format: int32
type: integer
protocol:
default: TCP
description: |-
The IP protocol for this port. Supports "TCP", "UDP", and "SCTP".
Default is TCP.
type: string
targetPort:
anyOf:
- type: integer
- type: string
description: |-
Number or name of the port to access on the pods targeted by the service.
Number must be in the range 1 to 65535. Name must be an IANA_SVC_NAME.
If this is a string, it will be looked up as a named port in the
target Pod's container ports. If this is not specified, the value
of the 'port' field is used (an identity map).
This field is ignored for services with clusterIP=None, and should be
omitted or set equal to the 'port' field.
More info: https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service
x-kubernetes-int-or-string: true
required:
- port
type: object
type: array
subdomainPolicy:
description: |-
SubdomainPolicy determines the policy that will be used when creating
Expand Down
17 changes: 5 additions & 12 deletions docs/examples/llamacpp/k8s/lws.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ metadata:
name: llamacpp-llama3-8b-instruct-bartowski-q5km
spec:
replicas: 1
networkConfig:
leaderServicePorts:
- protocol: TCP
port: 80
targetPort: 8080
leaderWorkerTemplate:
size: 5
restartPolicy: RecreateGroupOnPodRestart
Expand All @@ -24,15 +29,3 @@ spec:
image: llamacpp-worker:latest
imagePullPolicy: IfNotPresent
args: ["--host", "0.0.0.0", "--mem", "4192"]
---
apiVersion: v1
kind: Service
metadata:
name: llamacpp
spec:
selector:
app: llamacpp-llama3-8b-instruct-bartowski-q5km
ports:
- protocol: TCP
port: 80
targetPort: 8080
21 changes: 6 additions & 15 deletions docs/examples/sample/lws-volume-claim-templates.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ metadata:
name: lws
spec:
replicas: 2
networkConfig:
leaderServicePorts:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
leaderWorkerTemplate:
size: 2
restartPolicy: RecreateGroupOnPodRestart
Expand Down Expand Up @@ -43,18 +49,3 @@ spec:
volumeMounts:
- mountPath: /mnt/volume
name: persistent-storage
---
apiVersion: v1
kind: Service
metadata:
name: lws-leader
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
leaderworkerset.sigs.k8s.io/name: lws
role: leader
type: ClusterIP
18 changes: 5 additions & 13 deletions docs/examples/sglang/lws.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ metadata:
name: sglang
spec:
replicas: 2
networkConfig:
leaderServicePorts:
- port: 40000
protocol: TCP
targetPort: 40000
leaderWorkerTemplate:
size: 2
restartPolicy: RecreateGroupOnPodRestart
Expand Down Expand Up @@ -87,16 +92,3 @@ spec:
- name: dshm
emptyDir:
medium: Memory
---
apiVersion: v1
kind: Service
metadata:
name: sglang-leader
spec:
selector:
leaderworkerset.sigs.k8s.io/name: sglang
role: leader
ports:
- protocol: TCP
port: 40000
targetPort: 40000
18 changes: 5 additions & 13 deletions docs/examples/tensorrt-llm/lws.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ metadata:
name: tensorrt
spec:
replicas: 1
networkConfig:
leaderServicePorts:
- name: http
port: 8000
targetPort: 8000
leaderWorkerTemplate:
size: 2
leaderTemplate:
Expand Down Expand Up @@ -61,16 +66,3 @@ spec:
- name: dshm
emptyDir:
medium: Memory
---
apiVersion: v1
kind: Service
metadata:
name: tensorrt-service
spec:
ports:
- name: http
port: 8000
targetPort: 8000
selector:
leaderworkerset.sigs.k8s.io/name: tensorrt
role: leader
21 changes: 6 additions & 15 deletions docs/examples/vllm/GPU/lws.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@ metadata:
name: vllm
spec:
replicas: 2
networkConfig:
leaderServicePort:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
leaderWorkerTemplate:
size: 2
restartPolicy: RecreateGroupOnPodRestart
Expand Down Expand Up @@ -74,18 +80,3 @@ spec:
emptyDir:
medium: Memory
sizeLimit: 15Gi
---
apiVersion: v1
kind: Service
metadata:
name: vllm-leader
spec:
ports:
- name: http
port: 8080
protocol: TCP
targetPort: 8080
selector:
leaderworkerset.sigs.k8s.io/name: vllm
role: leader
type: ClusterIP
Loading