@@ -10,7 +10,7 @@ Author(s): @elevran @nirrozenbaum
10
10
11
11
The EPP Architecture proposal identifies the need for an extensible
12
12
[ Data Layer] ( ../0683-epp-architecture-proposal/README.md#data-layer ) .
13
- Recently, the scheduling subsystem underwent a major [ architecure change] ( ../0845-scheduler-architecture-proposal/README.md )
13
+ Recently, the scheduling subsystem underwent a major [ architecture change] ( ../0845-scheduler-architecture-proposal/README.md )
14
14
to allow easier extension and pluggability. This proposal aims to apply
15
15
similar extensibility to the Data Layer subsystem, allowing custom inference
16
16
gateways to extend the Gateway API Inference extension (GIE) for their use
@@ -20,7 +20,7 @@ See [this document](https://docs.google.com/document/d/1eCCuyB_VW08ik_jqPC1__z6F
20
20
21
21
## Goals
22
22
23
- The Data Layer pluggability effort aims to address the folllowing goals and
23
+ The Data Layer pluggability effort aims to address the following goals and
24
24
requirements:
25
25
26
26
- Make endpoint attributes used by GIE components accessible via well defined
@@ -56,6 +56,8 @@ The Data Layer pluggability effort aims to address the folllowing goals and
56
56
- Inference scheduler Plugins, that rely on custom data collection, accept that
57
57
the [ Model Server Protocol] ( ../003-model-server-protocol/README.md ) no longer
58
58
provides guarantees on portability of a model server out of the box.
59
+ - Intent is * not* to introduce a new scraping mechanism, and continue to support
60
+ the current model of Go-routine per endpoint.
59
61
60
62
## Proposal
61
63
@@ -72,7 +74,7 @@ There are two existing Data Sources in the Data Layer: a Pod reconciler that
72
74
The proposal is to make the Data Layer more extensible approaching by introducing
73
75
these two interfaces:
74
76
75
- - An ** Attribute Collection** plugin interface responsible for extratcing relevant
77
+ - An ** Attribute Collection** plugin interface responsible for extracting relevant
76
78
attributes from a data source and storing them into the Data Layer for consumption
77
79
by other components. The plugin can be registered with existing or new
78
80
* Data Sources* (see below) and sources would call their registered plugins
@@ -97,7 +99,7 @@ In order to make iterative progress and validate the design alongside, we
97
99
1 . Refactor the metrics scraping code into separate Data Source and Data Collection
98
100
plugin interfaces.
99
101
1 . Following that, and based on any lessons learnt, we’ll refactor the existing
100
- Kubernetes Pod recocilation loop to the new plugin interfaces.
102
+ Kubernetes Pod reconciliation loop to the new plugin interfaces.
101
103
102
104
### Suggested Data Layer Plugin Interfaces
103
105
@@ -163,9 +165,9 @@ type DataSource interface {
163
165
164
166
## Open Questions
165
167
166
- 1 . Type safety in extensible data colletion : ` map[string]interface{} ` seems
168
+ 1 . Type safety in extensible data collection : ` map[string]interface{} ` seems
167
169
like the simplest option to start, but may want to evolve to support
168
- type safety using generics or codegen .
170
+ type safety using generics or code generation .
169
171
1 . Should we design a separate interface specifically for k8s object watching
170
172
under GIE control or do we want these to be managed as yet another data source?
171
173
This affects the design (e.g., who owns the k8s caches, clients, etc.).
0 commit comments