feat: implement config cache-proxy command#54
feat: implement config cache-proxy command#54tisutisu wants to merge 3 commits intokonflux-ci:mainfrom
Conversation
✅ Snyk checks have passed. No issues have been found so far.
💻 Catch issues earlier using the plugins for VS Code, JetBrains IDEs, Visual Studio, and Eclipse. |
cd0ef36 to
5e3688a
Compare
pkg/commands/cache_proxy.go
Outdated
| l.Logger.Infof("cluster-config config map data: %v", cm.Data) | ||
| allowCache = cm.Data["allow-cache-proxy"] | ||
| if allowCache == "true" { | ||
| httpProxy = cm.Data["http-proxy"] | ||
| noProxy = cm.Data["no-proxy"] | ||
| } else { | ||
| httpProxy = "" | ||
| noProxy = "" | ||
| } |
There was a problem hiding this comment.
Should this piece of code in else branch?
There was a problem hiding this comment.
Otherwise setting default*Proxy has no effect. They are reassigned by reading http-proxy and no-proxy from cm.Data.
There was a problem hiding this comment.
have felt this same while writing the logic, but kept the logic same as per the current bash implementation here
There was a problem hiding this comment.
The below if-else is relative to the link you mentioned.
https://github.com/konflux-ci/build-definitions/blob/a91adc4f10bf5dc684fe32b373fd718253945cd4/task/init/0.3/init.yaml#L44-L61 is relative to the commented lines I think. Code from line 44 to 61 has this logic:
- if configmap is present, http_proxy and no_proxy are assigned according to the config data.
- if configmap is not present, use the defaults.
There was a problem hiding this comment.
Got your point now, thanks for catching it.
Addressed it on the latest commit 787fee7
tkdchen
left a comment
There was a problem hiding this comment.
Maybe we can also wrap kubectl (it's included in task-runner) command instead of calling kubenetes APIs, which is similar with other nested commands. Invoking kubectl could also help us having less effort on maintaining dependencies updates.
@tkdchen even if we use cli, for reading/unmarshalling of kubectl cmd outputs, we need to depend on the kubernetes pkgs like |
mmorhun
left a comment
There was a problem hiding this comment.
This might be out of scope of this PR, but it anyway blocks it.
The main problem is that we accessing k8s cluster directly from within CLI making it tightly bound and dependent on k8s environment, making it impossible to run without a cluster, say locally or in another than Tekton CI. (Obviously, there are workarounds, but we need a proper solution).
My suggestion is to create a abstraction layer, say ConfigReader that would detect the environment (easiest is to use env var) and then read k8s resources if run within k8s cluster or access config in different source (say ini or env file) otherwise.
eddf14f to
8d1f0fd
Compare
mmorhun
left a comment
There was a problem hiding this comment.
I think we need ConfigReader interface with a factory (NewConfigReader) that constructs required implementation (K8sConfigReader or EnvFileConfigReader) based on an env var (if not set, k8s is the default). Then config command(s) can use it.
fa990a8 to
d53204e
Compare
pkg/commands/cache_proxy.go
Outdated
| Usage: "Whether to enable cache proxy or not. Required.", | ||
| Required: true, | ||
| }, | ||
| "config-file": { |
There was a problem hiding this comment.
Since the config contains global platform setting and not only proxy config, I think that should be configured on the ConfigReader level via an env var.
There was a problem hiding this comment.
Updated the code to read from a platform config file named PLATFORM_CONFIG_FILE from the env.
os.Getenv("PLATFORM_CONFIG_FILE")
PTAL again.
pkg/commands/cache_proxy.go
Outdated
| configMapNamespace = "cluster-config" | ||
| configMapName = "konflux-info" |
There was a problem hiding this comment.
Those are properties of k8sConfigReader
pkg/commands/cache_proxy.go
Outdated
| defaultHttpProxy = "squid.caching.svc.cluster.local:3128" | ||
| defaultNoProxy = "brew.registry.redhat.io,docker.io,gcr.io,ghcr.io,images.paas.redhat.com,mirror.gcr.io,nvcr.io,quay.io,registry-proxy.engineering.redhat.com,registry.access.redhat.com,registry.ci.openshift.org,registry.fedoraproject.org,registry.redhat.io,registry.stage.redhat.io,vault.habana.ai" |
There was a problem hiding this comment.
I'd prefer to not to have such defaults here, but have them in the Tekton task env.
pkg/commands/cache_proxy.go
Outdated
| if err != nil { | ||
| return err | ||
| } | ||
| c.Configs.ConfigReader = &config.K8sConfigMapReader{Name: configMapName, Namespace: configMapNamespace, Clientset: clientset} |
There was a problem hiding this comment.
We should have something like:
ConfigReader = NewConfigReader()
in the command constructor.
pkg/commands/cache_proxy.go
Outdated
| c.logParams() | ||
|
|
||
| l.Logger.Debug("Reading config-map data") | ||
| cmData, err := c.Configs.ConfigReader.ReadConfigData() |
There was a problem hiding this comment.
nitpick: instead of cmData, it's better to have configData or something like that (to not to have hard k8s connection)
There was a problem hiding this comment.
Renamed it to cacheProxyConfig now. agnostic of k8s connection.
pkg/commands/cache_proxy.go
Outdated
| } | ||
| } | ||
|
|
||
| // Apply ENABLE_CACHE_PROXY check (from task param) ONLY if cluster allows it |
There was a problem hiding this comment.
Better say if backend allows it and say, for example, k8s cluster.
pkg/config/config_map_reader.go
Outdated
|
|
||
| // ReadConfigData reads the YAML file, unmarshals it and returns the config data from the configmap. | ||
| func (y *YAMLFileReader) ReadConfigData() (map[string]string, error) { | ||
|
|
pkg/config/config_map_reader.go
Outdated
| return nil, fmt.Errorf("failed to read file %s: %w", y.FilePath, err) | ||
| } | ||
|
|
||
| configMap := corev1.ConfigMap{} |
There was a problem hiding this comment.
Let's assume that the command runs in non k8s environment. How comfortable it is to deal with k8s config map in such an environment (e.g. GitHub action?) to provide the config?
There was a problem hiding this comment.
now reading from an ini file.
d53204e to
cc6af4b
Compare
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
| return cacheProxy, nil | ||
| } | ||
|
|
||
| func (c *CacheProxy) initializeConfigs() error { |
There was a problem hiding this comment.
Not sure that we need a separate function for creating an instance of ConfigReader
| l.Logger.Warnf("Error while reading config map: %s", err.Error()) | ||
| // ConfigMap missing, use defaults |
There was a problem hiding this comment.
The command should know nothing about ConfigMaps
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &K8sConfigMapReader{Name: "cluster-config", Namespace: "konflux-info", Clientset: clientset}, nil |
There was a problem hiding this comment.
Should those be configurable via env vars too with the above defaults?
| "k8s.io/client-go/kubernetes" | ||
| ) | ||
|
|
||
| type CacheProxyConfig struct { |
There was a problem hiding this comment.
Since we are reading config from "konflux-info", should the struct have a generic name?
| @@ -0,0 +1,5 @@ | |||
| [cache-proxy] | |||
There was a problem hiding this comment.
Does it make sense to create this file in tempdir from tests?
This PR implements the
konflux-build-cli config cache-proxy --enable <true/false>cmd, to be used during build-definitionsinittask replacement using golang CLI.