chore: make CM content aligned to current EH Service dev#644
chore: make CM content aligned to current EH Service dev#644tarilabs wants to merge 3 commits intotrustyai-explainability:mainfrom
Conversation
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Reviewer's GuideSplits provider configuration into a dedicated providers ConfigMap mounted as a directory of per-provider YAML files, updates the EvalHub controller and deployment to use Eval Hub API-aligned provider types and layout, and adjusts tests accordingly. Sequence diagram for reconciling the providers ConfigMapsequenceDiagram
participant Reconciler as EvalHubReconciler
participant K8sAPI as KubernetesAPIServer
participant ProvidersCM as ProvidersConfigMap
Reconciler->>K8sAPI: Get EvalHub instance
K8sAPI-->>Reconciler: EvalHub instance
Reconciler->>Reconciler: buildEvalHubConfig(instance)
Reconciler->>Reconciler: generateProvidersData(config.Providers)
Reconciler->>K8sAPI: Get ConfigMap name=instance-name-providers
alt ConfigMap not found
K8sAPI-->>Reconciler: NotFound
Reconciler->>ProvidersCM: Set Data = per-provider YAML
Reconciler->>ProvidersCM: SetControllerReference(instance)
Reconciler->>K8sAPI: Create Providers ConfigMap
K8sAPI-->>Reconciler: Created
else ConfigMap exists
K8sAPI-->>Reconciler: Existing ConfigMap
Reconciler->>ProvidersCM: Update Data = per-provider YAML
Reconciler->>K8sAPI: Update Providers ConfigMap
K8sAPI-->>Reconciler: Updated
end
Reconciler-->>Reconciler: Continue reconciliation (Deployment, Proxy CM, etc.)
Class diagram for updated EvalHub provider configuration typesclassDiagram
class EvalHubConfig {
+[]ProviderResource Providers
+[]string Collections
+*DatabaseConfig Database
+*SecretsMapping Secrets
}
class DatabaseConfig {
+string Driver
+string Host
+int Port
+string User
+string Password
+string Name
+int MaxOpenConns
+int MaxIdleConns
}
class SecretsMapping {
+string DatabaseUserSecret
+string DatabasePasswordSecret
}
class ProviderResource {
+string ID
+string Name
+string Description
+string Type
+[]BenchmarkResource Benchmarks
+*Runtime Runtime
}
class BenchmarkResource {
+string ID
+*string ProviderId
+string Name
+string Description
+string Category
+[]string Metrics
+int NumFewShot
+int DatasetSize
+[]string Tags
}
class Runtime {
+*K8sRuntime K8s
+*LocalRuntime Local
}
class K8sRuntime {
+string Image
+[]string Entrypoint
+string CPURequest
+string MemoryRequest
+string CPULimit
+string MemoryLimit
+[]EnvVar Env
}
class LocalRuntime {
}
class EnvVar {
+string Name
+string Value
}
EvalHubConfig --> "*" ProviderResource : providers
EvalHubConfig --> "0..1" DatabaseConfig : database
EvalHubConfig --> "0..1" SecretsMapping : secrets
ProviderResource --> "*" BenchmarkResource : benchmarks
ProviderResource --> "0..1" Runtime : runtime
Runtime --> "0..1" K8sRuntime : k8s
Runtime --> "0..1" LocalRuntime : local
K8sRuntime --> "*" EnvVar : env
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
📝 WalkthroughWalkthroughThe PR refactors EvalHub provider configuration from hard-coded ProviderConfig entries to a ProviderResource model, builds an in-memory EvalHubConfig via a new builder, emits per-provider YAML into a dedicated Providers ConfigMap, and mounts that ConfigMap as a directory in the EvalHub deployment. Changes
Sequence Diagram(s)sequenceDiagram
participant Reconciler as Reconciler (EvalHub)
participant Builder as buildEvalHubConfig
participant ProvidersCM as Providers ConfigMap
participant K8sAPI as Kubernetes API
participant Deployment as Deployment (Pod)
Reconciler->>Builder: buildEvalHubConfig(instance)
Builder-->>Reconciler: EvalHubConfig (with ProviderResource entries)
Reconciler->>ProvidersCM: reconcileProvidersConfigMap(EvalHubConfig.Providers)
ProvidersCM->>K8sAPI: create/update ConfigMap keys (per-provider YAML)
K8sAPI-->>ProvidersCM: ack
Reconciler->>Deployment: ensure volume/volumeMount for providers dir
Deployment->>K8sAPI: patch/create Deployment with evalhub-providers volume
K8sAPI-->>Deployment: ack
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related issues
Possibly related PRs
Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
No actionable comments were generated in the recent review. 🎉 🧹 Recent nitpick comments
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
tarilabs
added
project/evalhub
needs-orchestrator-build
Signed-off-by: tarilabs <matteo.mortari@gmail.com>
tarilabs
force-pushed
the
tarilabs-20260213-similarConfigMap
branch
from
935030e to
1c357b3
Compare
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hey - I've found 3 issues, and left some high level feedback:
- In
generateProvidersDatayou key the ConfigMap entries offprovider.Name, butProviderResourcealready has anIDfield that is more stable and less likely to contain characters invalid in filenames; consider switching toID(and/or sanitizing the chosen field) for the<name>.yamlkey to avoid subtle issues if names change or contain path-unfriendly characters. - The default provider/benchmark definitions are now embedded directly in
buildEvalHubConfigand mirrored in several tests via hard-coded names; consider centralizing these defaults in a shared constant or helper so provider changes only need to be made in one place and tests can reference them symbolically.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- In `generateProvidersData` you key the ConfigMap entries off `provider.Name`, but `ProviderResource` already has an `ID` field that is more stable and less likely to contain characters invalid in filenames; consider switching to `ID` (and/or sanitizing the chosen field) for the `<name>.yaml` key to avoid subtle issues if names change or contain path-unfriendly characters.
- The default provider/benchmark definitions are now embedded directly in `buildEvalHubConfig` and mirrored in several tests via hard-coded names; consider centralizing these defaults in a shared constant or helper so provider changes only need to be made in one place and tests can reference them symbolically.
## Individual Comments
### Comment 1
<location> `controllers/evalhub/configmap.go:174-186` </location>
<code_context>
+// generateProvidersData generates per-provider YAML entries for the providers ConfigMap.
+// Each provider becomes a separate key like "lm-eval-harness.yaml".
+func (r *EvalHubReconciler) generateProvidersData(providers []ProviderResource) (map[string]string, error) {
+ data := make(map[string]string, len(providers))
+ for _, provider := range providers {
+ providerYAML, err := yaml.Marshal(provider)
+ if err != nil {
+ return nil, fmt.Errorf("failed to marshal provider %s: %w", provider.Name, err)
+ }
+ data[fmt.Sprintf("%s.yaml", provider.Name)] = string(providerYAML)
+ }
+ return data, nil
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Use provider ID (or ID with fallback) as the filename key instead of Name.
Using `provider.Name` here couples the ConfigMap key/filename to a mutable, user-facing field that may be non-unique or contain characters invalid for keys. Since `ProviderResource` has a stable `ID`, prefer `provider.ID` for the key (e.g. `fmt.Sprintf("%s.yaml", provider.ID)`), or fall back to `Name` only if `ID` is empty, to avoid collisions and unintended breakage when display names change.
```suggestion
// generateProvidersData generates per-provider YAML entries for the providers ConfigMap.
// Each provider becomes a separate key like "provider-id.yaml", preferring the stable provider ID.
func (r *EvalHubReconciler) generateProvidersData(providers []ProviderResource) (map[string]string, error) {
data := make(map[string]string, len(providers))
for _, provider := range providers {
key := provider.ID
if key == "" {
key = provider.Name
}
providerYAML, err := yaml.Marshal(provider)
if err != nil {
return nil, fmt.Errorf("failed to marshal provider %s: %w", key, err)
}
data[fmt.Sprintf("%s.yaml", key)] = string(providerYAML)
}
return data, nil
}
```
</issue_to_address>
### Comment 2
<location> `controllers/evalhub/configmap_test.go:132-141` </location>
<code_context>
+ It("should have valid per-provider YAML files in providers ConfigMap", func() {
</code_context>
<issue_to_address>
**suggestion (testing):** Strengthen providers ConfigMap test by asserting exact keys and expected provider shapes (type and benchmarks) rather than only non-empty name
Currently the test only checks that four expected keys exist and that each unmarshalled `ProviderResource` has a non-empty `Name`. To better match the new provider schema, tighten the assertions by:
- Verifying `configMap.Data` contains exactly the expected provider keys (e.g. `HaveLen(4)` and explicit key set).
- After unmarshalling, asserting `provider.Type` and the expected benchmark names per provider (using `benchmarkResourceNames`), not just a non-empty `Name`.
This will make the test more resistant to accidental changes in provider definitions or ConfigMap key names.
Suggested implementation:
```golang
It("should have valid per-provider YAML files in providers ConfigMap", func() {
By("Reconciling providers configmap")
err := reconciler.reconcileProvidersConfigMap(ctx, evalHub)
Expect(err).NotTo(HaveOccurred())
By("Getting providers configmap")
configMap := waitForConfigMap(evalHubName+"-providers", testNamespace)
By("Ensuring providers ConfigMap has exactly the expected provider keys")
Expect(configMap.Data).To(HaveLen(len(expectedProviderBenchmarks)))
for providerKey := range expectedProviderBenchmarks {
Expect(configMap.Data).To(HaveKey(providerKey))
}
By("Validating each provider's schema and benchmarks")
for providerKey, providerYAML := range configMap.Data {
expectedBenchmarks, ok := expectedProviderBenchmarks[providerKey]
Expect(ok).To(BeTrue(), "unexpected provider key %q in providers ConfigMap", providerKey)
expectedType, ok := expectedProviderTypes[providerKey]
Expect(ok).To(BeTrue(), "missing expected type for provider key %q", providerKey)
var provider ProviderResource
err := yaml.Unmarshal([]byte(providerYAML), &provider)
Expect(err).NotTo(HaveOccurred(), "provider %q YAML should be valid", providerKey)
Expect(provider.Name).NotTo(BeEmpty(), "provider %q should have a non-empty name", providerKey)
Expect(provider.Type).To(Equal(expectedType), "provider %q should have expected type", providerKey)
actualBenchmarkNames := benchmarkResourceNames(provider)
Expect(actualBenchmarkNames).To(ConsistOf(expectedBenchmarks), "provider %q benchmarks should match expected set", providerKey)
}
```
To make this compile and reflect the intended stronger assertions, you will also need to:
1. Define the expected per-provider benchmarks, keyed by the providers ConfigMap keys, near the top of the test file (or in a suitable shared test helper), for example:
```go
var expectedProviderBenchmarks = map[string][]string{
"aws": {"benchmark-1", "benchmark-2"},
"azure": {"benchmark-1", "benchmark-3"},
"gcp": {"benchmark-1", "benchmark-4"},
"onprem": {"benchmark-2"},
}
```
Adjust keys and benchmark names to match your actual provider schema and what `reconcileProvidersConfigMap` is expected to produce.
2. Define the expected provider types, keyed by the same provider keys:
```go
var expectedProviderTypes = map[string]string{
"aws": "cloud",
"azure": "cloud",
"gcp": "cloud",
"onprem": "datacenter",
}
```
Again, adjust to align with your real `ProviderResource.Type` values.
3. Ensure `ProviderResource` and `benchmarkResourceNames` are available in this test file:
- If `ProviderResource` is defined in another package, import that package and reference it as needed.
- `benchmarkResourceNames` should accept a `ProviderResource` and return `[]string` of benchmark names; if it currently has a different signature, adapt the call accordingly or add a thin helper with that signature.
4. Confirm that the file already imports the `yaml` package used for unmarshalling (e.g. `sigs.k8s.io/yaml`). If not, add the appropriate import consistent with the rest of the file.
</issue_to_address>
### Comment 3
<location> `controllers/evalhub/deployment_test.go:402` </location>
<code_context>
- By("Checking evalhub container has 2 volume mounts (config + DB secret)")
- Expect(evalHubContainer.VolumeMounts).To(HaveLen(2))
+ By("Checking evalhub container has 3 volume mounts (config + providers + DB secret)")
+ Expect(evalHubContainer.VolumeMounts).To(HaveLen(3))
</code_context>
<issue_to_address>
**suggestion (testing):** Deployment tests should assert providers volume mount path and ConfigMap name, not only counts and names
Since this PR changes `PROVIDERS_CONFIG_PATH` to `/etc/evalhub/providers/`, the tests should also:
- Verify the `evalhub-providers` VolumeMount has `MountPath == "/etc/evalhub/providers"`.
- Verify the `evalhub-providers` volume is a ConfigMap with `LocalObjectReference.Name == instance.Name + "-providers"`.
This will better protect against regressions in the mount path or ConfigMap wiring.
Suggested implementation:
```golang
By("Checking evalhub container has 3 volume mounts (config + providers + DB secret)")
Expect(evalHubContainer.VolumeMounts).To(HaveLen(3))
By("Checking evalhub-providers VolumeMount path")
var providersMount *corev1.VolumeMount
for i := range evalHubContainer.VolumeMounts {
vm := &evalHubContainer.VolumeMounts[i]
if vm.Name == "evalhub-providers" {
providersMount = vm
break
}
}
Expect(providersMount).NotTo(BeNil())
Expect(providersMount.MountPath).To(Equal("/etc/evalhub/providers"))
By("Checking evalhub-providers volume is a ConfigMap with the correct name")
var providersVolume *corev1.Volume
for i := range deployment.Spec.Template.Spec.Volumes {
v := &deployment.Spec.Template.Spec.Volumes[i]
if v.Name == "evalhub-providers" {
providersVolume = v
break
}
}
Expect(providersVolume).NotTo(BeNil())
Expect(providersVolume.ConfigMap).NotTo(BeNil())
Expect(providersVolume.ConfigMap.LocalObjectReference.Name).To(Equal(instance.Name + "-providers"))
```
1. Ensure `corev1` is already imported as `k8s.io/api/core/v1`; most likely it is since other volume assertions are present.
2. This snippet assumes an `instance` variable is in scope in this test (e.g. the CR under test). If the CR variable uses a different name, adjust `instance.Name` accordingly.
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
| It("should have valid per-provider YAML files in providers ConfigMap", func() { | ||
| By("Reconciling providers configmap") | ||
| err := reconciler.reconcileProvidersConfigMap(ctx, evalHub) | ||
| Expect(err).NotTo(HaveOccurred()) | ||
|
|
||
| By("Getting configmap") | ||
| configMap := waitForConfigMap(evalHubName+"-config", testNamespace) | ||
| By("Getting providers configmap") | ||
| configMap := waitForConfigMap(evalHubName+"-providers", testNamespace) | ||
|
|
||
| By("Parsing providers.yaml") | ||
| var providersData map[string]interface{} | ||
| err = yaml.Unmarshal([]byte(configMap.Data["providers.yaml"]), &providersData) | ||
| Expect(err).NotTo(HaveOccurred()) | ||
|
|
||
| By("Checking providers structure") | ||
| Expect(providersData).To(HaveKey("providers")) | ||
| providers, ok := providersData["providers"].([]interface{}) | ||
| Expect(ok).To(BeTrue()) | ||
| Expect(providers).To(HaveLen(4)) | ||
| By("Checking each provider file exists and is valid") | ||
| expectedProviders := []string{ |
There was a problem hiding this comment.
suggestion (testing): Strengthen providers ConfigMap test by asserting exact keys and expected provider shapes (type and benchmarks) rather than only non-empty name
Currently the test only checks that four expected keys exist and that each unmarshalled ProviderResource has a non-empty Name. To better match the new provider schema, tighten the assertions by:
- Verifying
configMap.Datacontains exactly the expected provider keys (e.g.HaveLen(4)and explicit key set). - After unmarshalling, asserting
provider.Typeand the expected benchmark names per provider (usingbenchmarkResourceNames), not just a non-emptyName.
This will make the test more resistant to accidental changes in provider definitions or ConfigMap key names.
Suggested implementation:
It("should have valid per-provider YAML files in providers ConfigMap", func() {
By("Reconciling providers configmap")
err := reconciler.reconcileProvidersConfigMap(ctx, evalHub)
Expect(err).NotTo(HaveOccurred())
By("Getting providers configmap")
configMap := waitForConfigMap(evalHubName+"-providers", testNamespace)
By("Ensuring providers ConfigMap has exactly the expected provider keys")
Expect(configMap.Data).To(HaveLen(len(expectedProviderBenchmarks)))
for providerKey := range expectedProviderBenchmarks {
Expect(configMap.Data).To(HaveKey(providerKey))
}
By("Validating each provider's schema and benchmarks")
for providerKey, providerYAML := range configMap.Data {
expectedBenchmarks, ok := expectedProviderBenchmarks[providerKey]
Expect(ok).To(BeTrue(), "unexpected provider key %q in providers ConfigMap", providerKey)
expectedType, ok := expectedProviderTypes[providerKey]
Expect(ok).To(BeTrue(), "missing expected type for provider key %q", providerKey)
var provider ProviderResource
err := yaml.Unmarshal([]byte(providerYAML), &provider)
Expect(err).NotTo(HaveOccurred(), "provider %q YAML should be valid", providerKey)
Expect(provider.Name).NotTo(BeEmpty(), "provider %q should have a non-empty name", providerKey)
Expect(provider.Type).To(Equal(expectedType), "provider %q should have expected type", providerKey)
actualBenchmarkNames := benchmarkResourceNames(provider)
Expect(actualBenchmarkNames).To(ConsistOf(expectedBenchmarks), "provider %q benchmarks should match expected set", providerKey)
}To make this compile and reflect the intended stronger assertions, you will also need to:
-
Define the expected per-provider benchmarks, keyed by the providers ConfigMap keys, near the top of the test file (or in a suitable shared test helper), for example:
var expectedProviderBenchmarks = map[string][]string{ "aws": {"benchmark-1", "benchmark-2"}, "azure": {"benchmark-1", "benchmark-3"}, "gcp": {"benchmark-1", "benchmark-4"}, "onprem": {"benchmark-2"}, }
Adjust keys and benchmark names to match your actual provider schema and what
reconcileProvidersConfigMapis expected to produce. -
Define the expected provider types, keyed by the same provider keys:
var expectedProviderTypes = map[string]string{ "aws": "cloud", "azure": "cloud", "gcp": "cloud", "onprem": "datacenter", }
Again, adjust to align with your real
ProviderResource.Typevalues. -
Ensure
ProviderResourceandbenchmarkResourceNamesare available in this test file:- If
ProviderResourceis defined in another package, import that package and reference it as needed. benchmarkResourceNamesshould accept aProviderResourceand return[]stringof benchmark names; if it currently has a different signature, adapt the call accordingly or add a thin helper with that signature.
- If
-
Confirm that the file already imports the
yamlpackage used for unmarshalling (e.g.sigs.k8s.io/yaml). If not, add the appropriate import consistent with the rest of the file.
|
|
||
| By("Checking evalhub container has 2 volume mounts (config + DB secret)") | ||
| Expect(evalHubContainer.VolumeMounts).To(HaveLen(2)) | ||
| By("Checking evalhub container has 3 volume mounts (config + providers + DB secret)") |
There was a problem hiding this comment.
suggestion (testing): Deployment tests should assert providers volume mount path and ConfigMap name, not only counts and names
Since this PR changes PROVIDERS_CONFIG_PATH to /etc/evalhub/providers/, the tests should also:
- Verify the
evalhub-providersVolumeMount hasMountPath == "/etc/evalhub/providers". - Verify the
evalhub-providersvolume is a ConfigMap withLocalObjectReference.Name == instance.Name + "-providers".
This will better protect against regressions in the mount path or ConfigMap wiring.
Suggested implementation:
By("Checking evalhub container has 3 volume mounts (config + providers + DB secret)")
Expect(evalHubContainer.VolumeMounts).To(HaveLen(3))
By("Checking evalhub-providers VolumeMount path")
var providersMount *corev1.VolumeMount
for i := range evalHubContainer.VolumeMounts {
vm := &evalHubContainer.VolumeMounts[i]
if vm.Name == "evalhub-providers" {
providersMount = vm
break
}
}
Expect(providersMount).NotTo(BeNil())
Expect(providersMount.MountPath).To(Equal("/etc/evalhub/providers"))
By("Checking evalhub-providers volume is a ConfigMap with the correct name")
var providersVolume *corev1.Volume
for i := range deployment.Spec.Template.Spec.Volumes {
v := &deployment.Spec.Template.Spec.Volumes[i]
if v.Name == "evalhub-providers" {
providersVolume = v
break
}
}
Expect(providersVolume).NotTo(BeNil())
Expect(providersVolume.ConfigMap).NotTo(BeNil())
Expect(providersVolume.ConfigMap.LocalObjectReference.Name).To(Equal(instance.Name + "-providers"))- Ensure
corev1is already imported ask8s.io/api/core/v1; most likely it is since other volume assertions are present. - This snippet assumes an
instancevariable is in scope in this test (e.g. the CR under test). If the CR variable uses a different name, adjustinstance.Nameaccordingly.
Signed-off-by: tarilabs <matteo.mortari@gmail.com>
|
@tarilabs: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
@tarilabs thanks for the PR! I like this, but I'd like to suggest that we have one ConfigMap per provider. OOTB providers can absolutely stay in the Go code, as this would also solve the problem of image injection! But I think that having separate CMs for each provider makes a smoother path for custom providers. A user who wants to register their own provider would just create a ConfigMap in the namespace and label it with something like Adding or removing a custom provider would trigger a normal deployment rollout, which I believe it's the right behaviour. wdyt? |
|
@ruivieira I fully agree the "to be" solution is for the operator to simply mount a number of providers (from CM, from CRD) to the EH Service without knowing the content, but this will require fixing how to deal with Multi-Tenancy, given for example we can't mount CM from Namespace (iff we make the assumption that a Tenant is a Namespace). In the meantime I just wanted to align closer the resemblance of the ootb providers(this PR), and indeed not added the dependency to the go.mod and vendor'd the struct It's fine if we want to skip this PR and go straight to MT once "fixed" too :) |
| Database *DatabaseConfig `yaml:"database,omitempty"` | ||
| Secrets *SecretsMapping `yaml:"secrets,omitempty"` | ||
| Providers []ProviderResource `yaml:"providers"` | ||
| Collections []string `yaml:"collections,omitempty"` |
There was a problem hiding this comment.
What is the goal for Collections as strings in this config ?
| Enabled: true, | ||
| Benchmarks: []string{ | ||
| "faithfulness", "answer_relevancy", "context_precision", "context_recall", | ||
| Providers: []ProviderResource{ |
There was a problem hiding this comment.
Do we want all BYOF providers to be specified in the CR or via separate ConfigMaps that the CR can point to ? If the number of providers grow keeping them in a single CR can be overwhelming leading to large objects. If we keep one ConfigMap per provider, can help keeping things more isolated.
| Config: map[string]string{ | ||
| "bias_threshold": "0.1", | ||
| }, | ||
| Collections: []string{ |
There was a problem hiding this comment.
It might not be sufficient to have collection names pre-defined. We need the actual collection specifications defined. Collection objects are not very small so this could overwhelm the CR.
|
no longer needed, also #649 |
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
4 participants
Introduce a evalhub-providers ConfigMap with 1 entry per
providers/<name>.yamlTo resemble current EH Service config/ directory content
in this case, we need 1 provider yaml per config/providers subdirectory.
We are still evaluating how to add "additional user-provided providers",
and likely we will need 1 Provider 1 ConfigMap with annotation (or a CRD)
But with this PR I make the content of
/etc/evalhubSimilar to
/app/config/providersLater, we can avoid harcoded value on Deployment creation of the CM, and just mount the ootb ConfigMaps and User-provided-providers ConfigMaps, this could be 1 step filling the gap
I avoid introducing dependency to eval-hub repo in go.mod (for now).
DEMO
Summary by Sourcery
Align EvalHub controller ConfigMap generation and tests with the latest Eval Hub service provider API and configuration layout.
New Features:
Enhancements:
Tests:
Summary by CodeRabbit
Release Notes