|
64 | 64 | - [DeleteVolumeGroupSnapshot](#deletevolumegroupsnapshot)
|
65 | 65 | - [ControllerGetVolumeGroupSnapshot](#controllergetvolumegroupsnapshot)
|
66 | 66 | - [ListVolumeGroupSnapshots](#listvolumegroupsnapshots)
|
67 |
| - - [Alternatives](#alternatives) |
68 |
| - - [Immutable VolumeGroup](#immutable-volumegroup) |
69 |
| - - [ModifyVolume](#modifyvolume) |
70 | 67 | - [Production Readiness Review Questionnaire](#production-readiness-review-questionnaire)
|
71 | 68 | - [Feature enablement and rollback](#feature-enablement-and-rollback)
|
72 | 69 | - [Rollout, Upgrade and Rollback Planning](#rollout-upgrade-and-rollback-planning)
|
|
75 | 72 | - [Scalability](#scalability)
|
76 | 73 | - [Troubleshooting](#troubleshooting)
|
77 | 74 | - [Implementation History](#implementation-history)
|
| 75 | +- [Drawbacks](#drawbacks) |
| 76 | +- [Alternatives](#alternatives) |
| 77 | + - [Immutable VolumeGroup](#immutable-volumegroup) |
| 78 | + - [ModifyVolume](#modifyvolume) |
78 | 79 | <!-- /toc -->
|
79 | 80 |
|
80 | 81 | ## Release Signoff Checklist
|
@@ -296,6 +297,8 @@ This feature requires coordination between several controllers including the new
|
296 | 297 |
|
297 | 298 | ### Test Plan
|
298 | 299 |
|
| 300 | +[X] I/we understand the owners of the involved components may require updates to existing tests to make this code solid enough prior to committing the changes necessary to implement this enhancement. |
| 301 | + |
299 | 302 | ##### Prerequisite testing updates
|
300 | 303 | N/A
|
301 | 304 |
|
@@ -1343,125 +1346,6 @@ message ListVolumeGroupSnapshotsResponse {
|
1343 | 1346 | }
|
1344 | 1347 | ```
|
1345 | 1348 |
|
1346 |
| -### Alternatives |
1347 |
| - |
1348 |
| -#### Immutable VolumeGroup |
1349 |
| - |
1350 |
| -During the design discussions, an immutable VolumeGroup was considered but was removed because this would add lots of complexity to the design without much gain. It would also make it impossible to support the current way PVCs are provisioned in a Statefulset. |
1351 |
| - |
1352 |
| -Immutable VolumeGroup - PVCList or PVC Selector in the ImmutableSource field in the Spec (optional field); PVCList is in the Status. |
1353 |
| -* Create a new VolumeGroup with existing PVCs by PVCList or PVC Selector in the Spec. The PVCList will be in the VolumeGroup Status as well. |
1354 |
| -* VolumeGroup Status has a boolean Mutable set to false. |
1355 |
| - |
1356 |
| -``` |
1357 |
| -ImmutableSource struct { |
1358 |
| - PVCList |
1359 |
| - Selector |
1360 |
| -} |
1361 |
| -``` |
1362 |
| - |
1363 |
| -``` |
1364 |
| -// VolumeGroupSpec describes the common attributes of group storage devices |
1365 |
| -// and allows a Source for provider-specific attributes |
1366 |
| -Type VolumeGroupSpec struct { |
1367 |
| - // +optional |
1368 |
| - VolumeGroupClassName *string |
1369 |
| -
|
1370 |
| - // If ImmutableSource is nil, an empty volume group will be created. |
1371 |
| - // Otherwise, a volume group will be created with PVCs (if PVCList or Select is set) |
1372 |
| - // If ImmutableSource is not nil, it indicates the VolumeGroup is immutable |
1373 |
| - // +optional |
1374 |
| - ImmutableSource *VolumeGroupSource |
1375 |
| -} |
1376 |
| -
|
1377 |
| -// VolumeGroupSource contains 3 options. If VolumeGroupSource is not nil, |
1378 |
| -// one of the 3 options must be defined. |
1379 |
| -Type VolumeGroupSource struct { |
1380 |
| - // A list of existing persistent volume claims |
1381 |
| - // +optional |
1382 |
| - PVCList []PersistentVolumeClaim |
1383 |
| -
|
1384 |
| - // A label query over existing persistent volume claims to be added to the volume group. |
1385 |
| - // +optional |
1386 |
| - Selector *metav1.LabelSelector |
1387 |
| - } |
1388 |
| -
|
1389 |
| -type VolumeGroupStatus struct { |
1390 |
| - // VolumeGroupId is a unique id returned by the CSI driver |
1391 |
| - // to identify the VolumeGroup on the storage system. |
1392 |
| - // If a storage system does not provide such an id, the |
1393 |
| - // CSI driver can choose to return the VolumeGroup name. |
1394 |
| - VolumeGroupId *string |
1395 |
| -
|
1396 |
| - GroupCreationTime *metav1.Time |
1397 |
| -
|
1398 |
| - // A list of persistent volume claims |
1399 |
| - // +optional |
1400 |
| - PVCList []PersistentVolumeClaim |
1401 |
| -
|
1402 |
| - Ready *bool |
1403 |
| -
|
1404 |
| - // Mutable indicates if a VolumeGroup can be modified |
1405 |
| - // after it is created. If false, it indicates it cannot be |
1406 |
| - // modified once created. If ImmutableSource is not nil |
1407 |
| - // in VolumeGroupSpec, Mutable must be false; otherwise |
1408 |
| - // it means the driver does not support ImmutableSource. |
1409 |
| - // VOLUMEGROUP_IMMUTABLE and VOLUMEGROUP_MUTABLE capability |
1410 |
| - // will be added to the CSI spec. |
1411 |
| - Mutable *bool |
1412 |
| -
|
1413 |
| - // If true, it indicates the CSI driver supports adding |
1414 |
| - // an existing volume to the VolumeGroup and removing a |
1415 |
| - // volume from the VolumeGroup without deleting it. |
1416 |
| - // Only mutable VolumeGroup can support AddRemoveExistingPVC. |
1417 |
| - // A corresponding VOLUMEGROUP_ADD_REMOVE_EXISTING_VOLUME |
1418 |
| - // capability will be added to the CSI spec. |
1419 |
| - AddRemoveExistingPVC *bool |
1420 |
| -
|
1421 |
| - // Last error encountered during group creation |
1422 |
| - Error *VolumeGroupError |
1423 |
| -} |
1424 |
| -``` |
1425 |
| - |
1426 |
| -VOLUMEGROUP_IMMUTABLE and VOLUMEGROUP_MUTABLE capability will be added to the CSI spec. |
1427 |
| -If VOLUMEGROUP_IMMUTABLE is supported, a VolumeGroup with an ImmutableSource can be created. Mutable will be false, PVCList will be set, and Ready will be true in the Status. |
1428 |
| -Otherwise, a VolumeGroup with an ImmutableSource will not be created successfully. |
1429 |
| - |
1430 |
| -#### ModifyVolume |
1431 |
| - |
1432 |
| -ModifyVolume CSI RPC was considered earlier to add/remove one volume to/from a group at a time but it was removed because ModifyVolumeGroup CSI RPC was added. |
1433 |
| - |
1434 |
| -A new MODIFY_VOLUME capability will be added to support this. |
1435 |
| -It indicates that the controller plugin supports modifying a volume. |
1436 |
| - |
1437 |
| -``` |
1438 |
| - rpc ModifyVolume(ModifyVolumeRequest) |
1439 |
| - returns (ModifyVolumeResponse) { |
1440 |
| - option (alpha_method) = true; |
1441 |
| - } |
1442 |
| -``` |
1443 |
| - |
1444 |
| -This RPC is called when an existing volume is added to an existing volume group or when a volume is removed from the volume group. |
1445 |
| -A volume group id parameter will be in the ModifyVolumeRequest for an add request. |
1446 |
| -A volume group id parameter will not be in the ModifyVolumeRequest for a delete request. |
1447 |
| -If user requests to add an existing volume to a consistency group, but the CSI driver cannot fulfill the request because the existing volume is placed on a different storage pool from the consistency group, then the CSI driver MUST return failure. |
1448 |
| -This RPC MUST be idempotent. |
1449 |
| - |
1450 |
| -``` |
1451 |
| -message ModifyVolumeRequest { |
1452 |
| - string volume_id = 1; |
1453 |
| -
|
1454 |
| - // This field is OPTIONAL. |
1455 |
| - repeated string volume_group_id = 2 [(alpha_field) = true]; |
1456 |
| -
|
1457 |
| - // Secrets required by plugin to complete modify volume request. |
1458 |
| - // This field is OPTIONAL. Refer to the `Secrets Requirements` |
1459 |
| - // section on how to use this field. |
1460 |
| - map<string, string> secrets = 3 [(csi_secret) = true]; |
1461 |
| -} |
1462 |
| -``` |
1463 |
| -External-provisioner will be modified so that modifying PVC by adding VolumeGroupName will trigger a ModifyVolume call (a new CSI controller RPC) to CSI driver. |
1464 |
| - |
1465 | 1349 | ## Production Readiness Review Questionnaire
|
1466 | 1350 |
|
1467 | 1351 | ### Feature enablement and rollback
|
@@ -1492,7 +1376,6 @@ _This section must be completed when targeting alpha to a release._
|
1492 | 1376 | We will be able to create new VolumeGroup and VolumeGroupSnapshot API objects again.
|
1493 | 1377 |
|
1494 | 1378 | * **Are there any tests for feature enablement/disablement?**
|
1495 |
| - Unit tests will be added for the in-tree feature enable/disablement. |
1496 | 1379 | Since there is no feature gate for this feature on the external controller side and the only way to
|
1497 | 1380 | enable or disable this feature is to install or unistall the sidecar, we cannot write
|
1498 | 1381 | tests for feature enablement/disablement.
|
@@ -1645,3 +1528,128 @@ _This section must be completed when targeting beta graduation to a release._
|
1645 | 1528 | [existing SLIs/SLOs]: https://git.k8s.io/community/sig-scalability/slos/slos.md#kubernetes-slisslos
|
1646 | 1529 |
|
1647 | 1530 | ## Implementation History
|
| 1531 | + |
| 1532 | +N/A |
| 1533 | + |
| 1534 | +## Drawbacks |
| 1535 | + |
| 1536 | +Adding more new APIs and more complexities. |
| 1537 | + |
| 1538 | +## Alternatives |
| 1539 | + |
| 1540 | +### Immutable VolumeGroup |
| 1541 | + |
| 1542 | +During the design discussions, an immutable VolumeGroup was considered but was removed because this would add lots of complexity to the design without much gain. It would also make it impossible to support the current way PVCs are provisioned in a Statefulset. |
| 1543 | + |
| 1544 | +Immutable VolumeGroup - PVCList or PVC Selector in the ImmutableSource field in the Spec (optional field); PVCList is in the Status. |
| 1545 | +* Create a new VolumeGroup with existing PVCs by PVCList or PVC Selector in the Spec. The PVCList will be in the VolumeGroup Status as well. |
| 1546 | +* VolumeGroup Status has a boolean Mutable set to false. |
| 1547 | + |
| 1548 | +``` |
| 1549 | +ImmutableSource struct { |
| 1550 | + PVCList |
| 1551 | + Selector |
| 1552 | +} |
| 1553 | +``` |
| 1554 | + |
| 1555 | +``` |
| 1556 | +// VolumeGroupSpec describes the common attributes of group storage devices |
| 1557 | +// and allows a Source for provider-specific attributes |
| 1558 | +Type VolumeGroupSpec struct { |
| 1559 | + // +optional |
| 1560 | + VolumeGroupClassName *string |
| 1561 | +
|
| 1562 | + // If ImmutableSource is nil, an empty volume group will be created. |
| 1563 | + // Otherwise, a volume group will be created with PVCs (if PVCList or Select is set) |
| 1564 | + // If ImmutableSource is not nil, it indicates the VolumeGroup is immutable |
| 1565 | + // +optional |
| 1566 | + ImmutableSource *VolumeGroupSource |
| 1567 | +} |
| 1568 | +
|
| 1569 | +// VolumeGroupSource contains 3 options. If VolumeGroupSource is not nil, |
| 1570 | +// one of the 3 options must be defined. |
| 1571 | +Type VolumeGroupSource struct { |
| 1572 | + // A list of existing persistent volume claims |
| 1573 | + // +optional |
| 1574 | + PVCList []PersistentVolumeClaim |
| 1575 | +
|
| 1576 | + // A label query over existing persistent volume claims to be added to the volume group. |
| 1577 | + // +optional |
| 1578 | + Selector *metav1.LabelSelector |
| 1579 | + } |
| 1580 | +
|
| 1581 | +type VolumeGroupStatus struct { |
| 1582 | + // VolumeGroupId is a unique id returned by the CSI driver |
| 1583 | + // to identify the VolumeGroup on the storage system. |
| 1584 | + // If a storage system does not provide such an id, the |
| 1585 | + // CSI driver can choose to return the VolumeGroup name. |
| 1586 | + VolumeGroupId *string |
| 1587 | +
|
| 1588 | + GroupCreationTime *metav1.Time |
| 1589 | +
|
| 1590 | + // A list of persistent volume claims |
| 1591 | + // +optional |
| 1592 | + PVCList []PersistentVolumeClaim |
| 1593 | +
|
| 1594 | + Ready *bool |
| 1595 | +
|
| 1596 | + // Mutable indicates if a VolumeGroup can be modified |
| 1597 | + // after it is created. If false, it indicates it cannot be |
| 1598 | + // modified once created. If ImmutableSource is not nil |
| 1599 | + // in VolumeGroupSpec, Mutable must be false; otherwise |
| 1600 | + // it means the driver does not support ImmutableSource. |
| 1601 | + // VOLUMEGROUP_IMMUTABLE and VOLUMEGROUP_MUTABLE capability |
| 1602 | + // will be added to the CSI spec. |
| 1603 | + Mutable *bool |
| 1604 | +
|
| 1605 | + // If true, it indicates the CSI driver supports adding |
| 1606 | + // an existing volume to the VolumeGroup and removing a |
| 1607 | + // volume from the VolumeGroup without deleting it. |
| 1608 | + // Only mutable VolumeGroup can support AddRemoveExistingPVC. |
| 1609 | + // A corresponding VOLUMEGROUP_ADD_REMOVE_EXISTING_VOLUME |
| 1610 | + // capability will be added to the CSI spec. |
| 1611 | + AddRemoveExistingPVC *bool |
| 1612 | +
|
| 1613 | + // Last error encountered during group creation |
| 1614 | + Error *VolumeGroupError |
| 1615 | +} |
| 1616 | +``` |
| 1617 | + |
| 1618 | +VOLUMEGROUP_IMMUTABLE and VOLUMEGROUP_MUTABLE capability will be added to the CSI spec. |
| 1619 | +If VOLUMEGROUP_IMMUTABLE is supported, a VolumeGroup with an ImmutableSource can be created. Mutable will be false, PVCList will be set, and Ready will be true in the Status. |
| 1620 | +Otherwise, a VolumeGroup with an ImmutableSource will not be created successfully. |
| 1621 | + |
| 1622 | +### ModifyVolume |
| 1623 | + |
| 1624 | +ModifyVolume CSI RPC was considered earlier to add/remove one volume to/from a group at a time but it was removed because ModifyVolumeGroup CSI RPC was added. |
| 1625 | + |
| 1626 | +A new MODIFY_VOLUME capability will be added to support this. |
| 1627 | +It indicates that the controller plugin supports modifying a volume. |
| 1628 | + |
| 1629 | +``` |
| 1630 | + rpc ModifyVolume(ModifyVolumeRequest) |
| 1631 | + returns (ModifyVolumeResponse) { |
| 1632 | + option (alpha_method) = true; |
| 1633 | + } |
| 1634 | +``` |
| 1635 | + |
| 1636 | +This RPC is called when an existing volume is added to an existing volume group or when a volume is removed from the volume group. |
| 1637 | +A volume group id parameter will be in the ModifyVolumeRequest for an add request. |
| 1638 | +A volume group id parameter will not be in the ModifyVolumeRequest for a delete request. |
| 1639 | +If user requests to add an existing volume to a consistency group, but the CSI driver cannot fulfill the request because the existing volume is placed on a different storage pool from the consistency group, then the CSI driver MUST return failure. |
| 1640 | +This RPC MUST be idempotent. |
| 1641 | + |
| 1642 | +``` |
| 1643 | +message ModifyVolumeRequest { |
| 1644 | + string volume_id = 1; |
| 1645 | +
|
| 1646 | + // This field is OPTIONAL. |
| 1647 | + repeated string volume_group_id = 2 [(alpha_field) = true]; |
| 1648 | +
|
| 1649 | + // Secrets required by plugin to complete modify volume request. |
| 1650 | + // This field is OPTIONAL. Refer to the `Secrets Requirements` |
| 1651 | + // section on how to use this field. |
| 1652 | + map<string, string> secrets = 3 [(csi_secret) = true]; |
| 1653 | +} |
| 1654 | +``` |
| 1655 | +External-provisioner will be modified so that modifying PVC by adding VolumeGroupName will trigger a ModifyVolume call (a new CSI controller RPC) to CSI driver. |
0 commit comments