Merge pull request #85 from zvonkok/annotations

Added Annotations to CDI Spec

Author: bart0sh

README.md

@@ -4,64 +4,85 @@
 
 ## What is CDI?
 
-CDI (Container Device Interface), is a [specification](SPEC.md), for container runtimes, to support third party devices.
+CDI (Container Device Interface), is a [specification](SPEC.md), for container-
+runtimes, to support third-party devices.
 
-It introduces an abstract notion of a device as a resource. Such devices are uniquely specified by a fully-qualified name that is constructed from a vendor ID, a device class, and a name that is unique per vendor ID-device class pair.
+It introduces an abstract notion of a device as a resource. Such devices are
+uniquely specified by a fully-qualified name that is constructed from a vendor
+ID, a device class, and a name that is unique per vendor ID-device class pair.
 
 ```
 vendor.com/class=unique_name
 ```
 
-The combination of vendor ID and device class (`vendor.com/class` in the above example) is referred to as the device kind.
+The combination of vendor ID and device class (`vendor.com/class` in the above
+example) is referred to as the device kind.
 
-CDI concerns itself only with enabling container to be device aware. Areas like resource management are explicitly left out of CDI (and is expected to be handled by the orchestrator). Because of this focus, the CDI specification is simple to implement and allows great flexibility to runtimes and orchestrators.
+CDI concerns itself only with enabling containers to be device aware. Areas like
+resource management are explicitly left out of CDI (and are expected to be
+handled by the orchestrator). Because of this focus, the CDI specification is
+simple to implement and allows great flexibility for runtimes and orchestrators.
 
-Note: The CDI model is based on the Container Networking Interface (CNI) model and specification.
+Note: The CDI model is based on the Container Networking Interface (CNI) model
+and specification.
 
 ## Why is CDI needed?
 
-On Linux, enabling a container to be device aware used to be as simple as exposing a device node in that container.
-However, as devices and software grows more complex, vendors want to perform more operations, such as:
-
-- Exposing a device to a container can require exposing more than one device node, mounting files from the runtime namespace or hide procfs entries.
-- Performing compatibility checks between the container and the device (e.g: Can this container run on this device).
-- Performing runtime specific operations (e.g: VM vs linux containers based runtimes).
-- Performing device specific operations (e.g: scrubbing the memory of a GPU or reconfiguring an FPGA).
-
-In the absence of a standard for third party devices, vendors often have to write and maintain multiple plugins for different runtimes or even directly contribute vendor specific code in the runtime.
-Additionally runtimes don't uniformly expose a plugin system (or even expose a plugin system at all) leading to duplication of the functionality in higher level abstractions (such as Kubernetes device plugins).
+On Linux, enabling a container to be device aware used to be as simple as
+exposing a device node in that container. However, as devices and software grows
+more complex, vendors want to perform more operations, such as:
+
+- Exposing a device to a container can require exposing more than one device
+  node, mounting files from the runtime namespace, or hiding procfs entries.
+- Performing compatibility checks between the container and the device (e.g: Can
+  this container run on this device).
+- Performing runtime-specific operations (e.g: VM vs Linux containers-based
+  runtimes).
+- Performing device-specific operations (e.g: scrubbing the memory of a GPU or
+  reconfiguring an FPGA).
+
+In the absence of a standard for third-party devices, vendors often have to
+write and maintain multiple plugins for different runtimes or even directly
+contribute vendor-specific code in the runtime. Additionally, runtimes don't
+uniformly expose a plugin system (or even expose a plugin system at all) leading
+to duplication of the functionality in higher-level abstractions (such as
+Kubernetes device plugins).
 
 ## How does CDI work?
 
 For CDI to work the following needs to be done:
 
-- CDI file containing update for the OCI spec in JSON format should be present in the CDI
-  spec directory. Default directories are /etc/cdi and /var/run/cdi
-
-- Fully qualified device name should be passed to the runtime either
-  using command line parameters for podman or using container annotations
-  for CRI-O and Containerd
-
-- Container runtime should be able to find CDI file by the device name
-  and update container config using CDI file content.
+- CDI file containing updates for the OCI spec in JSON format should be present
+  in the CDI spec directory. Default directories are `/etc/cdi` and
+  `/var/run/cdi`
+- Fully qualified device name should be passed to the runtime either using
+  command line parameters for podman or using container annotations for CRI-O
+  and containerd
+- Container runtime should be able to find the CDI file by the device name and
+  update the container config using CDI file content.
 
 ## How to configure CDI?
 
 ### CRI-O configuration
 
-In CRI-O CDI support is enabled by default. It is configured with the default `/etc/cdi, /var/run/cdi`
-CDI directory locations. Therefore you can start using CDI simply by dropping CDI configuration files
-in either of those directories, static configuration into `/etc/cdi` and dynamically updated one into
-`/var/run/cdi`. If you are unsure of the configured directories you can run this command to find them
-out:
+In CRI-O CDI support is enabled by default. It is configured with the default
+`/etc/cdi, /var/run/cdi` CDI directory locations. Therefore you can start using
+CDI simply by dropping CDI configuration files in either of those directories,
+static configuration into `/etc/cdi` and dynamically updated one into
+`/var/run/cdi`. If you are unsure of the configured directories you can run this
+command to find them out:
 
 ```bash
 $ crio config |& grep -B1 -A5 cdi_spec_dirs
 ```
 
 ### Containerd configuration
 
-To enable and configure CDI support in the [containerd runtime](https://github.com/containerd/containerd) 2 configuration options `enable_cdi` and `cdi_spec_dirs` should be set in the `plugins."io.containerd.grpc.v1.cri` section of the containerd configuration file (`/etc/containerd/config.toml` by default):
+To enable and configure CDI support in the [containerd
+runtime](https://github.com/containerd/containerd) 2 configuration options
+`enable_cdi` and `cdi_spec_dirs` should be set in the
+`plugins."io.containerd.grpc.v1.cri` section of the containerd configuration
+file (`/etc/containerd/config.toml` by default):
 
 ```
 [plugins."io.containerd.grpc.v1.cri"]
@@ -70,19 +91,29 @@ To enable and configure CDI support in the [containerd runtime](https://github.c
 ```
 
 Remember to restart containerd for any configuration changes to take effect.
-
 ### Podman configuration
 
-[podman](https://github.com/containers/podman) does not require any specific configuration to enable CDI support and processes specified `--device` flags directly. If fully-qualified device selectors (e.g. `vendor.com/device=myDevice`) are included the CDI specifications at the default location (`/etc/cdi` and `/var/run/cdi`) are checked for matching devices.
+[podman](https://github.com/containers/podman) does not require any specific
+configuration to enable CDI support and processes specified `--device` flags
+directly. If fully-qualified device selectors (e.g.
+`vendor.com/device=myDevice`) are included the CDI specifications at the default
+location (`/etc/cdi` and `/var/run/cdi`) are checked for matching devices.
 
-*Note:* Although initial support was added in [`v3.2.0`](https://github.com/containers/podman/releases/tag/v3.2.0) this was updated for the tagged `v0.3.0` CDI spec in [`v4.1.0-rc.1`](https://github.com/containers/podman/releases/tag/v4.1.0-rc1) with [commit a234e4e](https://github.com/containers/podman/commit/a234e4e19662e172472877ce69523f4afea5c12e).
+*Note:* Although initial support was added in
+[`v3.2.0`](https://github.com/containers/podman/releases/tag/v3.2.0) this was
+updated for the tagged `v0.3.0` CDI spec in
+[`v4.1.0-rc.1`](https://github.com/containers/podman/releases/tag/v4.1.0-rc1)
+with [commit
+a234e4e](https://github.com/containers/podman/commit/a234e4e19662e172472877ce69523f4afea5c12e).
 
 ## Examples
+### Full-blown CDI specification
+
 ```bash
 $ mkdir /etc/cdi
 $ cat > /etc/cdi/vendor.json <<EOF
 {
-  "cdiVersion": "0.5.0",
+  "cdiVersion": "0.6.0",
   "kind": "vendor.com/device",
   "devices": [
     {
@@ -117,16 +148,46 @@ $ cat > /etc/cdi/vendor.json <<EOF
 EOF
 ```
 
-Assuming this specification has been generated and is available in either `/etc/cdi` or `/var/run/cdi` (or whereever a CDI-enabled consumer is configured to read CDI specifications from), the devices can be accessed through their fully-qualified device names.
+Assuming this specification has been generated and is available in either
+`/etc/cdi` or `/var/run/cdi` (or whereever a CDI-enabled consumer is configured
+to read CDI specifications from), the devices can be accessed through their
+fully-qualified device names.
 
 For example, in the case of `podman` the CLI for accessing the device would be:
 ```
 $ podman run --device vendor.com/device=myDevice ...
 ```
 
+### Using Annotations per device to add meta-information
+
+```bash
+$ mkdir /etc/cdi
+$ cat > /etc/cdi/vendor-annotations.json <<EOF
+{
+  "cdiVersion": "0.6.0",
+  "kind": "vendor.com/device",
+  "devices": [
+    {
+      "name": "myDevice",
+      "annotations": {
+        "whatever": "false"
+        "whenever": "true"
+      }
+      "containerEdits": {
+        "deviceNodes": [
+          {"path": "/dev/vfio/71"}
+        ]
+      }
+    }
+  ]
+}
+EOF
+```
+
+
 ## Issues and Contributing
 
-[Checkout the Contributing document!](CONTRIBUTING.md)
+[Check out the Contributing document!](CONTRIBUTING.md)
 
 * Please let us know by [filing a new issue](https://github.com/container-orchestrated-devices/container-device-interface/issues/new)
 * You can contribute by opening a [pull request](https://help.github.com/articles/using-pull-requests/)

SPEC.md

@@ -8,7 +8,7 @@
 
 ## Version
 
-This is CDI **spec** version **0.5.0**.
+This is CDI **spec** version **0.6.0**.
 
 ### Update policy
 
@@ -26,6 +26,7 @@ Released versions of the spec are available as Git tags.
 | v0.3.0 |   | Initial tagged release of Spec |
 | v0.4.0 |   | Added `type` field to Mount specification |
 | v0.5.0 |   | Add `HostPath` to `DeviceNodes` |
+| v0.6.0 |   | Add `Annotations` field to `Spec` and `Device` specifications |
 
 *Note*: The initial release of a **spec** with version `v0.x.0` will be tagged as
 `v0.x.0` with subsequent changes to the API applicable to this version tagged as `v0.x.y`.
@@ -80,13 +81,25 @@ The key words "must", "must not", "required", "shall", "shall not", "should", "s
 
 ```
 {
-    "cdiVersion": "0.5.0",
+    "cdiVersion": "0.6.0",
     "kind": "<name>",
 
+    // This field contains a set of key-value pairs that may be used to provide
+    // additional information to a consumer on the spec.
+    "annotations": { (optional)
+        "key": "value"
+    },
+
     "devices": [
         {
             "name": "<name>",
 
+            // This field contains a set of key-value pairs that may be used to provide
+            // additional information to a consumer on the specific device.
+            "annotations": { (optional)
+              "key": "value"
+            },
+
             // Same as the below containerSpec field.
             // This field should only be applied to the Container's OCI spec
             // if that specific device is requested.

internal/validation/k8s/objectmeta.go

@@ -0,0 +1,57 @@
+/*
+Copyright 2014 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+// Adapted from k8s.io/apimachinery/pkg/api/validation:
+// https://github.com/kubernetes/apimachinery/blob/7687996c715ee7d5c8cf1e3215e607eb065a4221/pkg/api/validation/objectmeta.go
+
+package k8s
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/container-orchestrated-devices/container-device-interface/internal/multierror"
+)
+
+// TotalAnnotationSizeLimitB defines the maximum size of all annotations in characters.
+const TotalAnnotationSizeLimitB int = 256 * (1 << 10) // 256 kB
+
+// ValidateAnnotations validates that a set of annotations are correctly defined.
+func ValidateAnnotations(annotations map[string]string, path string) error {
+	errors := multierror.New()
+	for k := range annotations {
+		// The rule is QualifiedName except that case doesn't matter, so convert to lowercase before checking.
+		for _, msg := range IsQualifiedName(strings.ToLower(k)) {
+			errors = multierror.Append(errors, fmt.Errorf("%v.%v is invalid: %v", path, k, msg))
+		}
+	}
+	if err := ValidateAnnotationsSize(annotations); err != nil {
+		errors = multierror.Append(errors, fmt.Errorf("%v is too long: %v", path, err))
+	}
+	return errors
+}
+
+// ValidateAnnotationsSize validates that a set of annotations is not too large.
+func ValidateAnnotationsSize(annotations map[string]string) error {
+	var totalSize int64
+	for k, v := range annotations {
+		totalSize += (int64)(len(k)) + (int64)(len(v))
+	}
+	if totalSize > (int64)(TotalAnnotationSizeLimitB) {
+		return fmt.Errorf("annotations size %d is larger than limit %d", totalSize, TotalAnnotationSizeLimitB)
+	}
+	return nil
+}