Merge pull request #2 from compspec/finish-webhook

feat: all is working

Author: vsoch

README.md

@@ -16,7 +16,8 @@ This is a Kubernetes controller that will do the following:
 ## Notes
 
 For the above, you don't technically need NFD if you add and select your own labels, but I want to use NFD to better integrate with the larger 
-community. The controller webhook will work with or without it, assuming the labels you have on nodes that you are selecting for exist.
+community. I've also scoped the matching labels to be for `feature.node` instead of `feature.node.kubernetes.io` so any generatd compatibility specs can be used for cases outside of Kubernetes (without the user thinking it is weird).
+The controller webhook will work with or without it, assuming the labels you have on nodes that you are selecting for exist.
 Next, we place this controller on the level of a webhook, meaning that it isn't influencing scheduling, but is anticipating what container
 should be selected for a node based on either:
 
@@ -96,18 +97,18 @@ In our real world use case we would select based on operating system and kernel
 we are just going to add the same label to all nodes and then check our controller based on the image selected. Let's first add "vanilla":
 
 ```bash
-bash ./example/add-nfd-features.sh "compspec.ocifit-k8s.flavor=vanilla"
+bash ./example/add-nfd-features.sh "feature.node.ocifit-k8s.flavor=vanilla"
 ```
 
 View our added labels:
 
 ```bash
-$ kubectl get nodes -o json | jq .items[].metadata.labels | grep compspec.ocifit-k8s.flavor
+kubectl get nodes -o json | jq .items[].metadata.labels | grep feature.node.ocifit-k8s.flavor
 ```
 ```console
-  "compspec.ocifit-k8s.flavor": "vanilla",
-  "compspec.ocifit-k8s.flavor": "vanilla",
-  "compspec.ocifit-k8s.flavor": "vanilla",
+  "feature.node.ocifit-k8s.flavor": "vanilla",
+  "feature.node.ocifit-k8s.flavor": "vanilla",
+  "feature.node.ocifit-k8s.flavor": "vanilla",
 ```
 
 If you want to see actual NFD labels:
@@ -140,9 +141,9 @@ kubectl logs ocifit-k8s-deployment-68d5bf5865-494mg -f
 
 At this point, we want to test compatibility. This step is already done, but I'll show you how I designed the compatibility spec. The logic for this dummy case is the following:
 
-1. If our custom label "compspec.ocifit-k8s.flavor" is vanilla, we want to choose a debian container.
-1. If our custom label "compspec.ocifit-k8s.flavor" is chocolate, we want to choose a ubuntu container.
-1. If our custom label "compspec.ocifit-k8s.flavor" is strawberry, we want to choose a rockylinux container.
+1. If our custom label "feature.node.ocifit-k8s.flavor" is vanilla, we want to choose a debian container.
+1. If our custom label "feature.node.ocifit-k8s.flavor" is chocolate, we want to choose a ubuntu container.
+1. If our custom label "feature.node.ocifit-k8s.flavor" is strawberry, we want to choose a rockylinux container.
 
 Normally, we would attach a compatibility spec to an image, like [this](https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/docs/usage/image-compatibility.md#attach-the-artifact-to-the-image). But
 here we are flipping the logic a bit. We don't know the image, and instead we are directing the client to look directly at one artifact. Thus, the first step was to package the compatibility artifact and push to a registry (and make it public). I did that as follows (you don't need to do this):
@@ -151,9 +152,82 @@ here we are flipping the logic a bit. We don't know the image, and instead we ar
 oras push ghcr.io/compspec/ocifit-k8s-compatibility:kind-example ./example/compatibility-test.json:application/vnd.oci.image.compatibilities.v1+json
 ```
 
-We aren't going to be using any referrers API or linking this to an image. The target images are in the artifact.
+We aren't going to be using any referrers API or linking this to an image. The target images are in the artifact, and we get there directly from the associated manifest.
+Try creating the pod
 
-**being written**
+```bash
+kubectl apply -f ./example/pod.yaml
+```
+
+Now look the log, the selection was vanilla, so we matched to the debian image. 
+
+```console
+PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
+NAME="Debian GNU/Linux"
+VERSION_ID="11"
+VERSION="11 (bullseye)"
+VERSION_CODENAME=bullseye
+ID=debian
+HOME_URL="https://www.debian.org/"
+SUPPORT_URL="https://www.debian.org/support"
+BUG_REPORT_URL="https://bugs.debian.org/"
+```
+
+Because we matched the feature.node label and it was the best fit container. Now delete that label and install
+another:
+
+```bash
+kubectl delete -f example/pod.yaml 
+bash ./example/remove-nfd-features.sh
+bash ./example/add-nfd-features.sh "feature.node.ocifit-k8s.flavor=chocolate"
+```
+
+Now we match to ubuntu 22.04
+
+```console
+PRETTY_NAME="Ubuntu 24.04.2 LTS"
+NAME="Ubuntu"
+VERSION_ID="24.04"
+VERSION="24.04.2 LTS (Noble Numbat)"
+VERSION_CODENAME=noble
+ID=ubuntu
+ID_LIKE=debian
+HOME_URL="https://www.ubuntu.com/"
+SUPPORT_URL="https://help.ubuntu.com/"
+BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
+PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
+UBUNTU_CODENAME=noble
+LOGO=ubuntu-logo
+```
+
+And finally, strawberry.
+
+```bash
+kubectl delete -f example/pod.yaml 
+bash ./example/remove-nfd-features.sh 
+bash ./example/add-nfd-features.sh "feature.node.ocifit-k8s.flavor=strawberry"
+```
+```console
+NAME="Rocky Linux"
+VERSION="9.3 (Blue Onyx)"
+ID="rocky"
+ID_LIKE="rhel centos fedora"
+VERSION_ID="9.3"
+PLATFORM_ID="platform:el9"
+PRETTY_NAME="Rocky Linux 9.3 (Blue Onyx)"
+ANSI_COLOR="0;32"
+LOGO="fedora-logo-icon"
+CPE_NAME="cpe:/o:rocky:rocky:9::baseos"
+HOME_URL="https://rockylinux.org/"
+BUG_REPORT_URL="https://bugs.rockylinux.org/"
+SUPPORT_END="2032-05-31"
+ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9"
+ROCKY_SUPPORT_PRODUCT_VERSION="9.3"
+REDHAT_SUPPORT_PRODUCT="Rocky Linux"
+REDHAT_SUPPORT_PRODUCT_VERSION="9.3"
+```
+
+Boum! Conceptually, we are selecting a different image depending on the rules in the compatibility spec. Our node features were dummy, but they could be real attributes related to kernel, networking, etc.
 
 ## License
 

cmd/ocifit/main.go

@@ -76,9 +76,11 @@ func deny(ar *admissionv1.AdmissionReview, message string) *admissionv1.Admissio
 }
 
 // Determine if label is for NFD
-// We use this to assess uniqueness
+// We use this to assess uniqueness (homogeneity of cluster)
 func isCompatibilityLabel(key string) bool {
-	return strings.HasPrefix(key, "feature.node.kubernetes.io/") ||
+	// Note that the full URI is feature.node.kubernetes.io/
+	// I'm truncating to feature.node so the features aren't Kubernetes specific
+	return strings.HasPrefix(key, "feature.node") ||
 		key == "kubernetes.io/arch" ||
 		key == "kubernetes.io/os"
 }
@@ -300,9 +302,6 @@ func (ws *WebhookServer) mutate(ar *admissionv1.AdmissionReview) *admissionv1.Ad
 		return deny(ar, fmt.Sprintf("compatibility spec %s issue: %v", imageRef, err))
 	}
 
-	// Debug printing for me :)
-	fmt.Println(spec)
-
 	// 4. Evaluate the spec against the node's labels to find the winning tag.
 	// The "tag" attribute we are hijacking here to put the full container URI
 	finalImage, err := validator.EvaluateCompatibilitySpec(spec, nodeLabels)
@@ -314,7 +313,7 @@ func (ws *WebhookServer) mutate(ar *admissionv1.AdmissionReview) *admissionv1.Ad
 	var patches []JSONPatch
 	containerFound := false
 	for i, c := range pod.Spec.Containers {
-		if c.Name == targetRef {
+		if c.Image == targetRef {
 			patches = append(patches, JSONPatch{
 				Op:    "replace",
 				Path:  fmt.Sprintf("/spec/containers/%d/image", i),

example/add-nfd-features.sh

@@ -9,7 +9,7 @@ set -oe pipefail
 NFD_NAMESPACE="node-feature-discovery"
 
 # This is just a prefix - we will add an arbitrary selector here
-FEATURE_LABEL=${1:-"compspec.ocifit-k8s.flavor=vanilla"}
+FEATURE_LABEL=${1:-"feature.node.ocifit-k8s.flavor=vanilla"}
 echo "Planning to add ${FEATURE_LABEL} to worker nodes..."
 
 echo "Finding worker nodes (nodes without the control-plane role)..."

example/compatibility-test.json

@@ -11,7 +11,7 @@
               "matchExpressions": [
                 {
                   "op": "In",
-                  "key": "compspec.ocifit-k8s.flavor",
+                  "key": "feature.node.ocifit-k8s.flavor",
                   "value": [
                     "vanilla"
                   ]
@@ -32,7 +32,7 @@
               "matchExpressions": [
                 {
                   "op": "In",
-                  "key": "compspec.ocifit-k8s.flavor",
+                  "key": "feature.node.ocifit-k8s.flavor",
                   "value": [
                     "chocolate"
                   ]
@@ -53,7 +53,7 @@
               "matchExpressions": [
                 {
                   "op": "In",
-                  "key": "compspec.ocifit-k8s.flavor",
+                  "key": "feature.node.ocifit-k8s.flavor",
                   "value": [
                     "strawberry"
                   ]

example/pod.yaml

@@ -1,16 +1,17 @@
 apiVersion: v1
 kind: Pod
 metadata:
-  name: pod-1
+  name: pod
   labels:
     oci.image.compatibilities.selection/enabled: "true"
   annotations:
     oci.image.compatibilities.selection/image-ref: "ghcr.io/compspec/ocifit-k8s-compatibility:kind-example"
 spec:
   # The container uri should be placeholder:latest. If you want to change, add the annotation:
   # oci.image.compatibilities.selection/target-image: "myplaceholder"
-
   containers:
   - name: app
     image: placeholder:latest
-    command: ["sleep", "3600"]
+    command: ["/bin/sh", "-c"]
+    args:
+      - "cat /etc/os-release"

example/remove-nfd-features.sh

@@ -2,7 +2,7 @@
 set -e
 
 # List the label keys to remove, each followed by a minus sign.
-LABELS_TO_REMOVE="compspec.ocifit-k8s.flavor-"
+LABELS_TO_REMOVE="feature.node.ocifit-k8s.flavor-"
 
 echo "Finding worker nodes..."
 WORKER_NODES=$(kubectl get nodes -l '!node-role.kubernetes.io/control-plane' -o jsonpath='{.items[*].metadata.name}')

pkg/artifact/artifact.go

@@ -8,6 +8,7 @@ import (
 	"strings"
 
 	"ghcr.io/compspec/ocifit-k8s/pkg/types"
+	ocispec "github.com/opencontainers/image-spec/specs-go/v1"
 	"oras.land/oras-go/v2/content"
 	"oras.land/oras-go/v2/registry/remote"
 )
@@ -16,37 +17,63 @@ import (
 func DownloadCompatibilityArtifact(ctx context.Context, specRef string) (*types.CompatibilitySpec, error) {
 	log.Printf("Downloading compatibility spec from: %s", specRef)
 
-	// Use ORAS to fetch the artifact
+	// --- Step 1: Connect to the registry and resolve the manifest by its tag ---
 	reg, err := remote.NewRegistry(strings.Split(specRef, "/")[0])
 	if err != nil {
 		return nil, fmt.Errorf("failed to connect to registry for spec: %w", err)
 	}
-
-	repoName := strings.SplitN(strings.TrimPrefix(specRef, reg.Reference.Host()), ":", 2)[0]
-	repoName = strings.TrimPrefix(repoName, "/") // Clean up repo name
-	tag := strings.Split(specRef, ":")[1]
-
-	repo, err := reg.Repository(ctx, repoName)
+	repo, err := reg.Repository(ctx, strings.Trim(strings.SplitN(specRef, ":", 2)[0], "/"))
+	if err != nil {
+		return nil, fmt.Errorf("failed to access repository for spec: %w", err)
+	}
+	manifestDesc, err := repo.Resolve(ctx, strings.Split(specRef, ":")[1])
 	if err != nil {
-		return nil, fmt.Errorf("failed to access repository %s: %w", repoName, err)
+		return nil, fmt.Errorf("failed to resolve spec manifest %s: %w", specRef, err)
 	}
 
-	desc, err := repo.Resolve(ctx, tag)
+	// --- Step 2: Fetch and parse the OCI Manifest itself ---
+	log.Println("Fetching OCI manifest content...")
+	manifestBytes, err := content.FetchAll(ctx, repo, manifestDesc)
 	if err != nil {
-		return nil, fmt.Errorf("failed to resolve spec artifact %s: %w", specRef, err)
+		return nil, fmt.Errorf("failed to fetch manifest content: %w", err)
+	}
+	var manifest ocispec.Manifest
+	if err := json.Unmarshal(manifestBytes, &manifest); err != nil {
+		return nil, fmt.Errorf("failed to unmarshal OCI manifest: %w", err)
 	}
+	log.Printf("Successfully parsed OCI manifest (artifact type: %s)", manifest.ArtifactType)
 
-	specBytes, err := content.FetchAll(ctx, repo, desc)
+	// --- Step 3: Find the correct layer within the manifest ---
+	log.Printf("Searching for spec layer with media type: %s", types.CompatibilitySpecMediaType)
+	var specLayerDesc *ocispec.Descriptor
+	for _, layer := range manifest.Layers {
+		if layer.MediaType == types.CompatibilitySpecMediaType {
+			// Found it! Keep a pointer to this descriptor.
+			specLayerDesc = &layer
+			break
+		}
+	}
+
+	if specLayerDesc == nil {
+		return nil, fmt.Errorf("manifest does not contain a layer with media type %s", types.CompatibilitySpecMediaType)
+	}
+	log.Printf("Found spec layer with digest: %s", specLayerDesc.Digest)
+
+	// --- Step 4: Fetch the content of the spec layer using its descriptor ---
+	log.Println("Fetching compatibility spec content...")
+	specBytes, err := content.FetchAll(ctx, repo, *specLayerDesc)
 	if err != nil {
-		return nil, fmt.Errorf("failed to fetch spec artifact content: %w", err)
+		return nil, fmt.Errorf("failed to fetch spec layer content: %w", err)
 	}
 
-	// Unmarshal the JSON into our struct
+	// --- Step 5: Unmarshal the final spec JSON into our struct ---
 	var spec types.CompatibilitySpec
-	if err := json.Unmarshal(specBytes, &spec); err != nil {
+	err = json.Unmarshal(specBytes, &spec)
+	if err != nil {
 		return nil, fmt.Errorf("failed to unmarshal compatibility spec JSON: %w", err)
 	}
 
 	log.Printf("Successfully downloaded and parsed spec version %s", spec.Version)
 	return &spec, nil
+
 }

pkg/types/types.go

@@ -3,6 +3,8 @@ package types
 // --- Image Compatibility Spec Structs (from NFD)
 // https://github.com/kubernetes-sigs/node-feature-discovery/blob/master/api/image-compatibility/v1alpha1/spec.go
 
+const CompatibilitySpecMediaType = "application/vnd.oci.image.compatibilities.v1+json"
+
 // GroupRule is a list of node feature rules.
 type GroupRule struct {
 	// CORRECTED: This is a slice to match the JSON array `[ ... ]`

pkg/validator/rule.go

@@ -1,6 +1,7 @@
 package validator
 
 import (
+	"fmt"
 	"log"
 	"regexp"
 	"strconv"
@@ -10,6 +11,7 @@ import (
 
 func evaluateRule(expression types.MatchExpression, nodeLabels map[string]string) bool {
 	// Get the value of the label from the node. The 'ok' boolean is crucial.
+	fmt.Printf("Checking expression %s against node labels %s", expression, nodeLabels)
 	nodeVal, ok := nodeLabels[expression.Key]
 
 	switch expression.Op {

pkg/validator/validator.go

@@ -24,20 +24,19 @@ func EvaluateCompatibilitySpec(spec *types.CompatibilitySpec, nodeLabels map[str
 		for _, groupRule := range comp.Rules {
 			if !allRulesMatch {
 				break
-			} // Short-circuit if a previous rule failed
-
+			}
 			for _, featureMatcher := range groupRule.MatchFeatures {
 				if !allRulesMatch {
 					break
-				} // Short-circuit
+				}
 
 				// Each FeatureMatcher has a slice of MatchExpressions. Loop through them.
 				// All MatchExpressions must match (AND logic).
 				for _, expression := range featureMatcher.MatchExpressions {
 					if !evaluateRule(expression, nodeLabels) {
 						// As soon as one expression fails, the entire 'Compatibility' set is invalid.
 						allRulesMatch = false
-						break // Exit the inner 'expression' loop.
+						break
 					}
 				}
 			}