Skip to content

Introduce v1beta2 VolumeGroupSnapshot API #1312

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/trivy.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ jobs:
make
docker build -t test/csi-snapshotter:latest -f ./cmd/csi-snapshotter/Dockerfile --output=type=docker --label revision=latest .
docker build -t test/snapshot-controller:latest -f ./cmd/snapshot-controller/Dockerfile --output=type=docker --label revision=latest .
docker build -t test/snapshot-conversion-webhook:latest -f ./cmd/snapshot-conversion-webhook/Dockerfile --output=type=docker --label revision=latest .

- name: Run Trivy vulnerability scanner on csi-snapshotter image
uses: aquasecurity/trivy-action@master
Expand All @@ -47,3 +48,15 @@ jobs:
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN'

- name: Run Trivy vulnerability scanner on snapshot-conversion-webhook image
uses: aquasecurity/trivy-action@master
env:
TRIVY_DB_REPOSITORY: "public.ecr.aws/aquasecurity/trivy-db:2"
with:
image-ref: 'test/snapshot-conversion-webhook:latest'
format: 'table'
exit-code: '1'
ignore-unfixed: true
vuln-type: 'os,library'
severity: 'CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN'
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,9 @@
# See the License for the specific language governing permissions and
# limitations under the License.

.PHONY: all snapshot-controller csi-snapshotter clean test
.PHONY: all snapshot-controller csi-snapshotter snapshot-conversion-webhook clean test

CMDS=snapshot-controller csi-snapshotter
CMDS=snapshot-controller csi-snapshotter snapshot-conversion-webhook
all: build
include release-tools/build.make

Expand Down
48 changes: 25 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ The volume snapshot feature supports CSI v1.0 and higher. It was introduced as a

The volume group snapshot feature supports CSI v1.10.0 and higher, and have been introduced in [Kubernetes 1.27 as an alpha feature](https://kubernetes.io/blog/2023/05/08/kubernetes-1-27-volume-group-snapshot-alpha/).

> :warning: **WARNING**: The validation webhook was deprecated in v8.0.0 and it is now removed. The validation webhook would prevent creating multiple default volume snapshot classes and multiple default volume group snapshot classes for the same CSI driver. With the removal of the validation webhook, an error will still be raised when dynamically provisioning a VolumeSnapshot or VolumeGroupSnapshot when multiple default volume snapshot classes or multiple default volume group snapshot classes for the same CSI driver exist.

## Overview

With the promotion of Volume Snapshot to GA, the feature is enabled by default on standard Kubernetes deployments and cannot be turned off.
Expand All @@ -22,15 +20,15 @@ Blog post for the Volume Group Snapshot Alpha feature can be found [here](https:

This information reflects the head of this branch.

| Minimum CSI Version | Recommended CSI Version | Container Image | [Min K8s Version](https://kubernetes-csi.github.io/docs/kubernetes-compatibility.html#minimum-version) | [Recommended K8s Version](https://kubernetes-csi.github.io/docs/project-policies.html#recommended-version) |
| ------------------------------------------------------------------------------------------ | ----------------------------| --------------- | --------------- | --------------- |
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/csi-snapshotter | 1.20 | 1.20 |
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/snapshot-controller | 1.20 | 1.20 |
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/snapshot-validation-webhook | 1.20 | 1.20 |
| Minimum CSI Version | Recommended CSI Version | Container Image | [Min K8s Version](https://kubernetes-csi.github.io/docs/kubernetes-compatibility.html#minimum-version) | [Recommended K8s Version](https://kubernetes-csi.github.io/docs/project-policies.html#recommended-version) |
| ------------------------------------------------------------------------------------------ | ----------------------------|----------------------------------------------------| --------------- | --------------- |
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/csi-snapshotter | 1.20 | 1.20 |
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/snapshot-controller | 1.20 | 1.20 |
| [CSI Spec v1.0.0](https://github.com/container-storage-interface/spec/releases/tag/v1.0.0) | [CSI Spec v1.5.0](https://github.com/container-storage-interface/spec/releases/tag/v1.5.0) | k8s.gcr.io/sig-storage/snapshot-conversion-webhook | 1.20 | 1.20 |

Note: snapshot-controller, csi-snapshotter v4.1 requires v1 snapshot CRDs to be installed, but it serves both v1 and v1beta1 snapshot objects. Storage version is changed from v1beta1 to v1 in 4.1.0 so v1beta1 is deprecated and will be removed in a future release.
Note: snapshot-controller, snapshot-conversion-webhook, csi-snapshotter v4.1 requires v1 snapshot CRDs to be installed, but it serves both v1 and v1beta1 snapshot objects. Storage version is changed from v1beta1 to v1 in 4.1.0 so v1beta1 is deprecated and will be removed in a future release.

Note: when the volume group snapshot feature is enabled, snapshot-controller, csi-snapshotter require the v1alpha1 volumegroupsnapshot CRDs to be installed.
Note: when the volume group snapshot feature is enabled, snapshot-controller, snapshot-conversion-webhook, csi-snapshotter require the v1beta2 volumegroupsnapshot CRDs to be installed.

## Feature Status

Expand All @@ -48,16 +46,17 @@ The CSI external-snapshotter sidecar talks to CSI over socket (/run/csi/socket b

In the current release, both v1 and v1beta1 APIs are served while the stored API version is changed from v1beta1 to v1. v1beta1 APIs is deprecated and will be removed in a future release. It is recommended for users to switch to v1 APIs as soon as possible. Any previously created invalid v1beta1 objects have to be deleted before upgrading to version 4.1.

### Volume Group Snapshot v1alpha1 APIs
### Volume Group Snapshot v1beta2 APIs

When enabled, the VolumeGroupSnapshot v1alpha1 APIs are being served.
When enabled, the VolumeGroupSnapshot v1beta2 APIs are being served.

## Usage

Volume Snapshot feature contains the following components:

* [Kubernetes Volume Snapshot and Volume Group Snapshot CRDs](https://github.com/kubernetes-csi/external-snapshotter/tree/master/client/config/crd)
* [Volume snapshot and volume group snapshot controller](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/common-controller)
* [Volume group snapshot conversion webhook](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/webhook)
* CSI Driver along with [CSI Snapshotter sidecar](https://github.com/kubernetes-csi/external-snapshotter/tree/master/pkg/sidecar-controller)

The Volume Snapshot feature depends on a volume snapshot controller and the volume snapshot CRDs. Both the controller and the CRDs are independent of any CSI driver. The CSI Snapshotter sidecar must run once per CSI driver. The single snapshot controller deployment works for all CSI drivers in a cluster. With leader election configured, the CSI sidecars and snapshot controller elect one leader per deployment. If deployed with two or more pods and leader election is enabled, the non-leader containers will attempt to get the lease. If the leader container dies, a non-leader will take over.
Expand All @@ -66,6 +65,8 @@ Therefore, it is strongly recommended that Kubernetes distributors bundle and de

If your Kubernetes distribution does not bundle the snapshot controller, you may manually install these components by executing the following steps. Note that the snapshot controller YAML files in the git repository deploy into the default namespace for system testing purposes. For general use, update the snapshot controller YAMLs with an appropriate namespace prior to installing. For example, on a Vanilla Kubernetes cluster update the namespace from 'default' to 'kube-system' prior to issuing the kubectl create command.

There is a new conversion webhook server which provides conversion between v1beta1 and v1beta2 group snapshot objects. The cluster admin or Kubernetes distribution admin should install the webhook alongside the snapshot controllers and CRDs if they want to provide group snapshot v1beta1 API. More details [below](#conversion-webhook).).

Install Snapshot and Volume Group Snapshot CRDs:
* With the repo cloned locally: `kubectl kustomize client/config/crd | kubectl create -f -`
* From the repo remotely: `kubectl kustomize https://github.com/kubernetes-csi/external-snapshotter/client/config/crd | kubectl create -f -`
Expand All @@ -82,23 +83,22 @@ Install CSI Driver:
* With the repo cloned locally: `kubectl kustomize deploy/kubernetes/csi-snapshotter | kubectl create -f -`
* From the repo remotely: `kubectl kustomize https://github.com/kubernetes-csi/external-snapshotter/deploy/kubernetes/csi-snapshotter | kubectl create -f -`

##### Volume Snapshot
### Conversion Webhook


* Spec.VolumeSnapshotClassName must not be an empty string or nil on creation
* Spec.Source.PersistentVolumeClaimName must not be changed on update requests
* Spec.Source.VolumeSnapshotContentName must not be changed on update requests
The snapshot conversion webhook is an HTTP callback which responds to
[conversion requests](https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definition-versioning/#webhook-conversion),
allowing the API server to convert between the VolumeGroupSnapshotContent v1beta1 API to and from the v1beta2 API.

##### Volume Snapshot Content
Read more about how to install the example webhook [here](deploy/kubernetes/webhook-example/README.md).

* Spec.VolumeSnapshotRef.Name must not be an empty string on creation
* Spec.VolumeSnapshotRef.Namespace must not be an empty string on creation
* Spec.Source.VolumeHandle must not be changed on update requests
* Spec.Source.SnapshotHandle must not be changed on update requests
* Spec.SourceVolumeMode must not be changes on update requests
#### Conversion Webhook Command Line Options

##### Volume Snapshot Classes
* `--tls-cert-file`: File containing the x509 Certificate for HTTPS. (CA cert, if any, concatenated after server cert). Required.

* There can only be a single default volume snapshot class for a particular driver.
* `--tls-private-key-file`: File containing the x509 private key matching --tls-cert-file. Required.

* `--port`: Secure port that the webhook listens on (default 443)

### Distributed Snapshotting

Expand Down Expand Up @@ -242,6 +242,8 @@ If you have already deployed v1alpha1 snapshot APIs and external-snapshotter sid

### Upgrade from v1beta1 to v1

Validation webhook should be installed before upgrading to v1. Potential impacts of not installing the validation webhook before upgrading to v1 include being unable to delete invalid snapshot objects. See the section on Validation Webhook for details.
Comment on lines 243 to +245
Copy link
Contributor

@jsafrane jsafrane Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be more like v1beta1 to v1beta2 upgrade. IMHO we need a new chapter. Or remove v1beta1 to v1 completely, it happened a long time ago.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can remove the v1beta1 to v1 section for VolumeSnapshot API now.
You can add a new section for VolumeGroupSnapshot.


* When upgrading to 4.0, change from v1beta1 to v1 is backward compatible because both v1 and v1beta1 are served while the stored API version is still v1beta1. Future releases will switch the stored version to v1 and gradually remove v1beta1 support.
* When upgrading from 3.x to 4.1, change from v1beta1 to v1 is no longer backward compatible because stored API version is changed to v1 although both v1 and v1beta1 are still served. v1beta1 is deprecated in 4.1.
* v1beta1 support will be removed in a future release. It is recommended for users to switch to v1 as soon as possible. Any previously created invalid v1beta1 objects have to be deleted before upgrading to version 4.1.
Expand Down
20 changes: 20 additions & 0 deletions client/apis/volumegroupsnapshot/v1beta2/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
/*
Copyright 2025 The Kubernetes Authors.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// +k8s:deepcopy-gen=package
// +groupName=groupsnapshot.storage.k8s.io

package v1beta2
57 changes: 57 additions & 0 deletions client/apis/volumegroupsnapshot/v1beta2/register.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
/*
Copyright 2025 The Kubernetes Authors.
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package v1beta2

import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
)

// GroupName is the group name use in this package.
const GroupName = "groupsnapshot.storage.k8s.io"

var (
// SchemeBuilder is the new scheme builder
SchemeBuilder = runtime.NewSchemeBuilder(addKnownTypes)
// AddToScheme adds to scheme
AddToScheme = SchemeBuilder.AddToScheme
// SchemeGroupVersion is the group version used to register these objects.
SchemeGroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1beta2"}
)

func Resource(resource string) schema.GroupResource {
return SchemeGroupVersion.WithResource(resource).GroupResource()
}

func init() {
// We only register manually written functions here. The registration of the
// generated functions takes place in the generated files. The separation
// makes the code compile even when the generated files are missing.
SchemeBuilder.Register(addKnownTypes)
}

// addKnownTypes adds the set of types defined in this package to the supplied scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
scheme.AddKnownTypes(SchemeGroupVersion,
&VolumeGroupSnapshotClass{},
&VolumeGroupSnapshotClassList{},
&VolumeGroupSnapshot{},
&VolumeGroupSnapshotList{},
&VolumeGroupSnapshotContent{},
&VolumeGroupSnapshotContentList{},
)
metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
return nil
}
Loading
Loading