doc/rgw: Updating doc for cloud-restore feature

soumyakoduri · soumyakoduri · commit 83672abf616b · 2025-04-02T07:14:16.000+05:30
Signed-off-by: Soumya Koduri &lt;skoduri@redhat.com&gt;
diff --git a/doc/radosgw/cloud-restore.rst b/doc/radosgw/cloud-restore.rst
@@ -0,0 +1,269 @@
+=============
+Cloud Restore
+=============
+
+`cloud-transition <https://docs.ceph.com/en/latest/radosgw/cloud-transition>`__   feature enables data transition to a remote cloud service. The ``cloud-restore`` feature enables restoration of those transitioned objects from the remote cloud S3 endpoints into RGW.
+
+This feature currently enables restore of objects that are transitioned to only S3 compatible cloud services. In order to validate this, ``retain_head_object`` option should be set to true in the ``tier-config`` while configuring storage class.
+
+The objects can be restored using `S3 RestoreObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_RestoreObject.html>`__ API . The restored copies will be retained on RGW only for the duration of ``Days`` specified. However if ``Days`` are not provided, the downloaded copy is considered permanent and will be treated as regular object.
+In addition, by enabling ``allow_read_through`` option, `S3 GetObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html>`__ API can be used as well to restore the object temporarily.
+
+
+Cloud Storage Class Tier Configuration
+--------------------------------------
+
+The `tier configuration <https://docs.ceph.com/en/latest/radosgw/cloud-transition/#cloud-storage-class-configuration>`__ of the cloud storage class configured for data transition is used to restore objects as well.
+
+::
+
+    {
+      "access_key": <access>,
+      "secret": <secret>,`
+      "endpoint": <endpoint>,
+      "region": <region>,
+      "host_style": <path | virtual>,
+      "acls": [ { "type": <id | email | uri>,
+                  "source_id": <source_id>,
+                  "dest_id": <dest_id> } ... ],
+      "target_path": <target_path>,
+      "target_storage_class": <target-storage-class>,
+      "multipart_sync_threshold": {object_size},
+      "multipart_min_part_size": {part_size},
+      "retain_head_object": <true | false>
+    }
+
+Additionally, below options have been added to the tier configuration to facilitate object restoration.
+
+* ``restore_storage_class`` (string)
+
+The storage class to which the object data needs to be restored to. Default value is `STANDARD`.
+
+
+read-through specific Configurables:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+* ``allow_read_through`` (true | false)
+
+If true, enables ``read-through``. Objects can then be restored using ``S3 GetObject`` API as well.
+
+* ``read_through_restore_days`` (interger)
+
+The duration for which objects restored via ``read-through`` are retained for. Default value is 1 day.
+
+For example:
+
+::
+
+    # radosgw-admin zonegroup placement modify --rgw-zonegroup default \
+                                               --placement-id default-placement \
+                                               --storage-class CLOUDTIER \
+                                               --tier-config=endpoint=http://XX.XX.XX.XX:YY,\
+                                               access_key=<access_key>,secret=<secret>, \
+                                               retain_head_object=true, \
+                                               restore_storage_class=COLDTIER, \
+                                               allow_read_through=true, \
+                                               read_through_restore_days=10
+
+
+
+S3 Glacier specific Configurables:
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+To restore objects archived in an S3 Glacier or Tape cloud storage, the data must first be restored to the cloud service before being read and downloaded into RGW. To enable this process, ensure the storage class is configured with ``--tier-type=cloud-s3-glacier``. Additionally, the following configurables should be set accordingly:
+
+* ``glacier_restore_days`` (integer)
+
+The duration of the objects to be restored on the remote cloud service.
+
+* ``glacier_restore_tier_type`` (Standard | Expedited)
+
+The type of retrieval of the object archived on the cloud service. Supported options are ``Standard`` and ``Expedited``.
+
+
+For example:
+
+::
+
+    # radosgw-admin zonegroup placement add --rgw-zonegroup=default \
+                                            --placement-id=default-placement \
+                                            --storage-class=CLOUDTIER-GLACIER --tier-type=cloud-s3-glacier
+
+    # radosgw-admin zonegroup placement modify --rgw-zonegroup default \
+                                               --placement-id default-placement \
+                                               --storage-class CLOUDTIER \
+                                               --tier-config=endpoint=http://XX.XX.XX.XX:YY,\
+                                               access_key=XXXXX,secret=YYYYY, \
+                                               retain_head_object=true, \
+                                               target_storage_class=Glacier, \
+                                               ............
+                                               ............
+                                               restore_storage_class=COLDTIER, \
+                                               glacier_restore_days=2, \
+                                               glacier_restore_tier_type=Expedited
+
+
+     [
+        {
+            "key": "default-placement",
+            "val": {
+                "name": "default-placement",
+                "tags": [],
+                "storage_classes": [
+                    "CLOUDTIER-GLACIER",
+                    "STANDARD"
+                ],
+                "tier_targets": [
+                    {
+                        "key": "CLOUDTIER-GLACIER",
+                        "val": {
+                            "tier_type": "cloud-s3-glacier",
+                            "storage_class": "CLOUDTIER-GLACIER",
+                            "retain_head_object": "true",
+                            "s3": {
+                                "endpoint": http://XX.XX.XX.XX:YY,
+                                "access_key": "XXXXX",
+                                "secret": "YYYYY",
+                                "host_style": "path",
+                                "target_storage_class": "Glacier",
+                                .......
+                                .......
+                            }
+                            "allow_read_through": true,
+                            "read_through_restore_days": 10,
+                            "restore_storage_class": "COLDTIER",
+                            "s3-glacier": {
+                                "glacier_restore_days": 2
+                                "glacier_restore_tier_type": "Expedited"
+                            }
+                        }
+                    }
+                ]
+            }
+        }
+    ]
+
+
+
+Examples to Restore Object:
+---------------------------
+
+Using S3 RestoreObject CLI
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Below options of `S3 restore-object <https://docs.aws.amazon.com/cli/latest/reference/s3api/restore-object.html>`__ CLI are supported  -
+
+
+Syntax
+
+::
+
+
+  $ aws s3api restore-object 
+            --bucket <value>
+            --key <value>
+            [--version-id <value>]
+            --restore-request (structure) {
+              Days=<integer>
+            }
+
+Note: ``Days`` is optional and if not provided, the object is restored permanently
+
+Example 1:
+
+::
+
+  $ aws s3api restore-object  --bucket bucket1 --key doc1.rtf 
+                              [--version-id 3sL4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo]
+                              --restore-request Days=10 
+                              ....
+
+This will restore the object `doc1.rtf` of the given version for the duration of 10 days.
+
+
+
+Example 2:
+
+::
+
+  $ aws s3api restore-object  --bucket bucket1 --key doc2.rtf --restore-request {} ....
+
+This will restore the object `doc2.rtf` permanently and will be treated as regular object.
+
+
+
+Using S3 GetObject CLI
+~~~~~~~~~~~~~~~~~~~~~
+Ensure ``allow_read_through`` tier-config option is enabled.
+
+Example 3:
+
+::
+
+  $ aws s3api get-object  --bucket bucket1 --key doc3.rtf ....
+
+This will restore the object `doc3.rtf` for the duration of the ``read_through_restore_days`` configured.
+
+
+Note: The above CLI command may time out if the object restoration takes too long. Before reissuing the command, you can verify the restoration status.
+
+
+Verifying the restore status
+----------------------------
+Verify the status of the restore by running an `S3 HeadObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_ResponseSyntax>`__  request. The response includes ``x-amz-restore`` header if either the object restoration is in progress or a copy of it is already restored.
+
+Example,
+
+::
+
+  $ aws s3api head-object --key doc1.rtf --bucket bucket1 ....
+
+
+In addition, ``radosgw-admin`` CLI can be used to check the restoration status and other details on the RGW server.
+
+Example,
+
+::
+  
+ $ radosgw-admin object stat --bucket bucket1 --object doc1.rtf
+
+
+Restored Object Properties
+--------------------------
+
+
+Storage
+~~~~~~
+The objects are restored to the storage class configured for ``restore_storage_class`` tier-config option. However, as per `AWS S3 RestoreObject <https://docs.aws.amazon.com/cli/latest/reference/s3api/restore-object.html>`__ the storage class of restored objects should remain unchanged. Therefore, for temporary copies, the x-amz-storage-class will continue to reflect the original cloud-tier storage class.
+
+
+mtime
+~~~~
+The `mtime` of the transitioned and restored objects should remain unchanged.
+
+
+Lifecycle
+~~~~~~~~
+`Temporary` copies are not subjected to any further transition to the cloud. However (as is the case with `cloud-transitioned objects`) they can be deleted via regular LC expiration rules or via external S3 Delete request.
+`Permanent` copies are treated as any regular objects and are subjected to any LC rules applicable.
+
+
+Replication
+~~~~~~~~~~
+`Temporary` copies are not replicated and will be retained only on the zones the restore request is initiated on.
+`Permanent` copies are replicated like other regular objects.
+
+
+Versioned Objects
+~~~~~~~~~~~~~~~~
+For versioned objects, if an object has been `cloud-transitioned`, it would be in a non-current state. After a restore, the same non-current object will be updated with the downloaded data, and its HEAD object will be modified accordingly.
+
+
+
+Future Work
+-----------
+
+* Admin Ops
+
+* Notifications
+
diff --git a/doc/radosgw/cloud-transition.rst b/doc/radosgw/cloud-transition.rst
@@ -4,14 +4,27 @@ Cloud Transition
 
 This feature enables data transition to a remote cloud service as part of `Lifecycle Configuration <https://docs.aws.amazon.com/AmazonS3/latest/dev/object-lifecycle-mgmt.html>`__ via :ref:`storage_classes`. The transition is unidirectional; data cannot be transitioned back from the remote zone. The goal of this feature is to enable data transition to multiple cloud providers. The currently supported cloud providers are those that are compatible with AWS (S3).
 
-Special storage class of tier type ``cloud-s3`` is used to configure the remote cloud S3 object store service to which the data needs to be transitioned. These are defined in terms of zonegroup placement targets and unlike regular storage classes, do not need a data pool.
+Special storage class of tier type ``cloud-s3`` or ``cloud-s3-glacier`` is used to configure the remote cloud S3 object store service to which the data needs to be transitioned. These are defined in terms of zonegroup placement targets and unlike regular storage classes, do not need a data pool.
 
 User credentials for the remote cloud object store service need to be configured. Note that source ACLs will not
 be preserved. It is possible to map permissions of specific source users to specific destination users.
 
 
-Cloud Storage Class Configuration
----------------------------------
+Cloud Storage Class Tier Type
+-----------------------------
+
+* ``tier-type`` (string)
+
+The type of remote cloud service that will be used to transition objects. Below tier types are supported:
+
+* ``cloud-s3`` : S3 compatible object store service
+
+* ``cloud-s3-glacier`` : S3 Glacier or Tape storage services where in objects are archived
+
+
+
+Cloud Storage Class Tier Configuration
+--------------------------------------
 
 ::
 
@@ -120,7 +133,7 @@ Minimum parts size to use when transitioning objects using multipart upload.
 How to Configure
 ~~~~~~~~~~~~~~~~
 
-See :ref:`adding_a_storage_class` for how to configure storage-class for a zonegroup. The cloud transition requires a creation of a special storage class with tier type defined as ``cloud-s3``
+See :ref:`adding_a_storage_class` for how to configure storage-class for a zonegroup. The cloud transition requires a creation of a special storage class with tier type defined as ``cloud-s3`` or ``cloud-s3-glacier``
 
 .. note:: If you have not done any previous `Multisite Configuration`_,
           a ``default`` zone and zonegroup are created for you, and changes
@@ -179,7 +192,7 @@ For example:
     ]
 
 
-.. note:: Once a storage class is created of ``--tier-type=cloud-s3``, it cannot be later modified to any other storage class type.
+.. note:: Once a storage class is created of ``--tier-type=cloud-s3`` or ``--tier-type=cloud-s3-glacier`` , it cannot be later modified to any other storage class type.
 
 The tier configuration can be then done using the following command
 
@@ -354,15 +367,19 @@ For versioned and locked objects, similar semantics as that of LifecycleExpirati
 * If the object is noncurrent and is locked, its transition is skipped.
 
 
+Restoring Objects
+-----------------
+The objects transitioned to cloud can now be restored. For more information, refer to 
+`Restoring Objects from Cloud <https://docs.aws.amazon.com/AmazonS3/latest/dev/cloud-restore.html>`
+
+
 Future Work
 -----------
 
 * Send presigned redirect or read-through the objects transitioned to cloud
 
 * Support s3:RestoreObject operation on cloud transitioned objects.
 
-* Federation between RGW and Cloud services.
-
 * Support transition to other cloud providers (like Azure).
 
 .. _`Multisite Configuration`: ../multisite
diff --git a/doc/radosgw/index.rst b/doc/radosgw/index.rst
@@ -87,6 +87,7 @@ Cluster with one API and then retrieve that data with the other API.
    Lua Scripting <lua-scripting>
    D3N Data Cache <d3n_datacache>
    Cloud Transition <cloud-transition>
+   Cloud Restore <cloud-restore>
    Metrics <metrics>
    UADK Acceleration for Compression <uadk-accel>
    Bucket Logging <bucket_logging>
