v0.5.0-rc.3

Overview

Major Highlights

• Conformance Tests: Validate your controller’s behavior with end-to-end tests covering InferencePool,
  InferenceModel, HTTPRoute, and more.

• New Config API: A new Config API which allows the configuration of plugins through a config file without touching core code.

• Helm Charts: helm chart update to support the reuse of Config API easily.

What's Changed

• Add scripts for running e2es by @keithmattix in #978
• fix: istio example destination rule by @EyalPazz in #970
• Bump Istio tag reference by @keithmattix in #974
• adds New functions to the scorers for consistency by @nirrozenbaum in #975
• feat(conformance): enable multiple endpoints in header based filter for EPP's conformance testing. by @zetxqx in #964
• e2e makefile comment fix by @nirrozenbaum in #976
• API: Adds 5xx Status Code for Invalid ExtRef by @danehans in #991
• feat(conformance): Add test for invalid EPP service reference by @SinaChavoshi in #959
• moved the creation of the context to main.go. by @nirrozenbaum in #995
• doc: fix dead links by @caozhuozi in #989
• feat: add health check for epp cluster by @zhengkezhou1 in #966
• test: gRPC server unit tests and utilities for further end-to-end tests by @irar2 in #820
• Update dynamic-lora-sidecar to expose metrics to track loaded adapters by @shotarok in #980
• refactor: Replace prefix cache structure with golang-lru by @kfirtoledo in #928
• feat(conformance): Add HTTPRouteMultipleRulesDifferentPools test by @SinaChavoshi in #834
• feat: Load the SchedulerConfig from a configuration file/text and make it easier to add plugins by @shmuelk in #881
• adding pre-request plugin to requestcontrol layer by @nirrozenbaum in #1004
• feat(conformance): Add test execution instruction to the guide. by @SinaChavoshi in #878
• fix: Update bbr fqdn to use helm release namespace by @chewong in #1009
• feat(conformance): Add HTTPRoute port validation tests for InferencePool backends by @zetxqx in #911
• refactor(conformance): move some common resources to shared place and add EPP service to tests needed. by @zetxqx in #982
• fix(Conformance): Add namespace-(labels|annotations) flag parsing by @aslakknutsen in #984
• bump cpu deployment version by @nirrozenbaum in #1016
• fix: api doc typo InvalidExtnesionRef by @aslakknutsen in #1018
• Adds vLLM CPU and Sim Support to Release Script by @danehans in #1020
• Add Makefile to run unit tests of tools/dynamic-lora-sidecar locally by @shotarok in #1021
• profile handler ProcessResult returns additional return value by @nirrozenbaum in #1013
• cleanup after config api PR was merged by @nirrozenbaum in #1012
• Making inferenceModel optional by @kfswain in #1024
• Adding Design Principles by @robscott in #596
• Adding Nir as a maintainer! by @kfswain in #1026
• [Fix] Missing property "apiGroup" error by @yafengio in #1015
• API: Adds default status condition to InferencePool by @danehans in #830
• feat(conformance): Add EPP conformance test for Gateway routing by @zetxqx in #961
• update sim deployment tag to latest by @nirrozenbaum in #1041
• refactor: rename plugin.Name() => plugin.Type() by @elevran in #1038
• docs: update the Getting Started guide to use the latest CRDs by @kfirtoledo in #1045
• added cycle state to pick & process results in profile handler by @nirrozenbaum in #1040
• feat(conformance): Add HTTPRouteMultipleGatewaysDifferentPools test by @SinaChavoshi in #838
• feat(conformance) add EPP unavailable fail-open test by @zetxqx in #999
• Add APIs for the instantiated plugins to the EPP Handle by @shmuelk in #1039
• chore(deps): bump the kubernetes group with 6 updates by @dependabot[bot] in #1050
• chore(deps): bump github.com/prometheus/common from 0.64.0 to 0.65.0 by @dependabot[bot] in #1051
• Only create LOCALBIN directory when it does not exist by @elevran in #1054
• remove datastore dependency from the scheduler by @nirrozenbaum in #1049
• add e2e test for epp metrics by @delavet in #938
• refactor(confromance) use common resources for InferencePoolHTTPRoutePortValidation test by @zetxqx in #1034
• Reintroduce Plugin.Name() by @elevran in #1057
• Extensible/Pluggable data layer proposal by @elevran in #1023
• Add subsetting logic for epp by @rlakhtakia in #981
• docs: added gke clean up instructions by @capri-xiyue in #1064
• feat(flowcontrol): Add Foundational Types and Architecture by @LukeAVanDrie in #997
• refactor: Allow export prefix SchedulingContextState for use across plugins by @kfirtoledo in #1063
• feat: Added a factory function for the DecisionTree filter by @shmuelk in #1053
• Adding pprof endpoints to metrics port by @kfswain in #1069
• version in README by @nirrozenbaum in #1072
• feat: Add a context.Context to the plugins.HAndle interface by @shmuelk in #1076
• Update model server protocol with prefix cache reuse by @liu-cong in #1077
• Update prefix plugin guide to use vllm as default to be consistent by @liu-cong in #1078
• refactor(conformance) merge similar utility functions. by @zetxqx in #1055
• fix(conformance): fix conformance setup issue by not relying on suite.Setup from gateway-api by @zetxqx in #1060
• e2e cleanup by @nirrozenbaum in #988
• fix: add wait after both httproute deletes for status to update by @aslakknutsen in #1056
• API: Refine ResolvedRefs condition for invalid ExtensionReference and expand InferencePoolReason values by @zetxqx in #1070
• Tidy up Data Layer documentation by @elevran in #1087
• test(epp): add unit tests for error package by @nsega in #1052
• remove duplicate entry from mkdocs by @nirrozenbaum in #1090
• Added a helper function to get referenced plugins by type by @shmuelk in #1091
• Explicitly set CGO_ENABLED=1 when running go test with race detection. by @elevran in #1089
• convert subset filter from a plugin to logic in director by @nirrozenbaum in #1088
• Use Kubernetes Reader instead of full Client by @elevran in #1093
• Docs: Bumps Kgateway Version by @danehans in #1094
• chore(deps): bump sigs.k8s.io/yaml from 1.4.0 to 1.5.0 by @dependabot[bot] in #1095
• Update istio release by @LiorLieberman in #1098
• Make k8s access explicitly read only by @elevran in #1101
• updated Makefile to allow configurable e2e image by @nirrozenbaum in #1092
• chore(deps): bump golang.org/x/sync from 0.14.0 to 0.15.0 by @dependabot[bot] in #1096
• fix(conformance): change InferencePoolInvalidEPPService test to align with spec . by @zetxqx in #1061
• Introduce plugins.TypedName to be used for Plugin base implementation by @elevran in #1086
• Add unit tests for picker plugin by @pabhi18 in #1103
• Upper case Inference in metrics.RecordInferencePoolReadyPods for consistency by @elevran in #1108
• fixed sim deployment for e2e tests by @nirrozenbaum in #1112
• move the conversion from pod metrics to scheduler pod representation one level up by @nirrozenbaum in #1104
• verify fallback to default values if parameters are not defined in config file by @nirrozenbaum in #1111
• minor refactor to config loading from file/text by @nirrozenbaum in #1114
• minor updates to typed-name by @nirrozenbaum in #1113
• handle picking multiple destinations in scheduling layer by @nirrozenbaum in #1059
• Clarify that connections to EPP should use TLS by default by @robscott in #1085
• feat: 🔧 unify the default parameter value positions by @yafengio in #1119
• refactor: 🔨 use the more explicit singular form by @yafengio in #1129
• feat(conformance): loose the InferencePoolInvalidEPPService to only check httpRoute is accepted true. by @zetxqx in #1131
• embed handlePlugins inside the general plugins handle by @nirrozenbaum in #1128
• Fix makefile: test-integration by @carlory in #1142
• feat: make model metrics endpoints configurable by @nayihz in #1000
• removed unused config/manager dir by @nirrozenbaum in #1155
• feat(conformance): add version for conformance test report. by @zetxqx in #1133
• initial update to load plugins from file by @nirrozenbaum in #1151
• adds flag to disble pprof if desired, defaulted on by @kfswain in #1159
• Fix: don't use corev1.objectReference by @capri-xiyue in #1156
• load plugins from file in conformance tests by @nirrozenbaum in #1152
• remove NewScheduler function by @nirrozenbaum in #1153

New Contributors

• @caozhuozi made their first contribution in #989
• @zhengkezhou1 made their first contribution in #966
• @irar2 made their first contribution in #820
• @kfirtoledo made their first contribution in #928
• @aslakknutsen made their first contribution in #984
• @yafengio made their first contribution in #1015
• @nsega made their first contribution in #1052
• @pabhi18 made their first contribution in #1103
• @carlory made their first contribution in #1142

Full Changelog: v0.4.0...v0.5.0-rc.3