Skip to content

Commit cdf81fa

Browse files
anthonyelevenanthonyeleven
authored
Merge pull request ceph#62661 from anthonyeleven/improve-62597
doc/radosgw: Improve cloud-restore and cloud-transition
2 parents 200c0af + 6a68074 commit cdf81fa

File tree

2 files changed

+220
-164
lines changed

2 files changed

+220
-164
lines changed

‎doc/radosgw/cloud-restore.rst‎

Lines changed: 102 additions & 73 deletions
Original file line numberDiff line numberDiff line change
@@ -1,22 +1,34 @@
1-
=============
2-
Cloud Restore
3-
=============
1+
===============
2+
Cloud Restore
3+
===============
44

5-
`cloud-transition <https://docs.ceph.com/en/latest/radosgw/cloud-transition>`__ feature enables data transition to a remote cloud service. The ``cloud-restore`` feature enables restoration of those transitioned objects from the remote cloud S3 endpoints into RGW.
5+
The :doc:`cloud-transition>` feature makes it possible to transition objects to a remote
6+
cloud service. The ``cloud-restore`` feature described below enables restoration
7+
of those transitioned objects from the remote S3 endpoints into the local
8+
RGW deployment.
69

7-
This feature currently enables restore of objects that are transitioned to only S3 compatible cloud services. In order to validate this, ``retain_head_object`` option should be set to true in the ``tier-config`` while configuring storage class.
10+
This feature currently enables the restoration of objects transitioned to
11+
S3-compatible cloud services. In order to faciliate this,
12+
the ``retain_head_object`` option should be set to ``true``
13+
in the ``tier-config`` when configuring the storage class.
814

9-
The objects can be restored using `S3 RestoreObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_RestoreObject.html>`__ API . The restored copies will be retained on RGW only for the duration of ``Days`` specified. However if ``Days`` are not provided, the downloaded copy is considered permanent and will be treated as regular object.
10-
In addition, by enabling ``allow_read_through`` option, `S3 GetObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html>`__ API can be used as well to restore the object temporarily.
15+
Objects can be restored using the `S3 RestoreObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_RestoreObject.html>`
16+
PI . The restored copies will be retained within RGW only for the number
17+
of ``days`` specified. However if ``days`` is not provided, the restored copies
18+
are considered permanent and will be treated as regular objects.
19+
In addition, by enabling the ``allow_read_through`` option,
20+
the `S3 GetObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_GetObject.html>`
21+
API can be used to restore the object temporarily.
1122

1223

1324
Cloud Storage Class Tier Configuration
1425
--------------------------------------
1526

16-
The `tier configuration <https://docs.ceph.com/en/latest/radosgw/cloud-transition/#cloud-storage-class-configuration>`__ of the cloud storage class configured for data transition is used to restore objects as well.
17-
18-
::
27+
The `tier configuration <https://docs.ceph.com/en/latest/radosgw/cloud-transition/#cloud-storage-class-configuration>`
28+
of the cloud storage class configured for data transition is used to restore
29+
objects as well:
1930

31+
```
2032
{
2133
"access_key": <access>,
2234
"secret": <secret>,`
@@ -32,29 +44,30 @@ The `tier configuration <https://docs.ceph.com/en/latest/radosgw/cloud-transitio
3244
"multipart_min_part_size": {part_size},
3345
"retain_head_object": <true | false>
3446
}
47+
```
3548

36-
Additionally, below options have been added to the tier configuration to facilitate object restoration.
49+
The below options have been added to the tier configuration to facilitate object restoration.
3750

3851
* ``restore_storage_class`` (string)
3952

40-
The storage class to which the object data needs to be restored to. Default value is `STANDARD`.
53+
The storage class to which object data is to be restored. Default value is ``STANDARD``.
4154

4255

4356
read-through specific Configurables:
4457
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
4558

46-
* ``allow_read_through`` (true | false)
59+
* ``allow_read_through`` (``true`` | ``false``)
4760

48-
If true, enables ``read-through``. Objects can then be restored using ``S3 GetObject`` API as well.
61+
If true, enables ``read-through``. Objects can then be restored using the ``S3 GetObject`` API.
4962

50-
* ``read_through_restore_days`` (interger)
63+
* ``read_through_restore_days`` (integer)
5164

52-
The duration for which objects restored via ``read-through`` are retained for. Default value is 1 day.
65+
The duration for which objects restored via ``read-through`` are retained.
66+
Default value is 1 day.
5367

5468
For example:
5569

56-
::
57-
70+
```
5871
# radosgw-admin zonegroup placement modify --rgw-zonegroup default \
5972
--placement-id default-placement \
6073
--storage-class CLOUDTIER \
@@ -64,27 +77,31 @@ For example:
6477
restore_storage_class=COLDTIER, \
6578
allow_read_through=true, \
6679
read_through_restore_days=10
67-
80+
```
6881

6982

7083
S3 Glacier specific Configurables:
7184
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
7285

73-
To restore objects archived in an S3 Glacier or Tape cloud storage, the data must first be restored to the cloud service before being read and downloaded into RGW. To enable this process, ensure the storage class is configured with ``--tier-type=cloud-s3-glacier``. Additionally, the following configurables should be set accordingly:
86+
To restore objects archived in an S3 Glacier or Tape cloud storage class, the
87+
data must first be restored to the cloud service before being read and
88+
downloaded into RGW. To enable this process, ensure the storage class
89+
is configured with ``--tier-type=cloud-s3-glacier``. Additionally,
90+
the following configurables should be set accordingly:
7491

7592
* ``glacier_restore_days`` (integer)
7693

77-
The duration of the objects to be restored on the remote cloud service.
94+
The duration for which the objects are to be restored on the remote cloud service.
7895

79-
* ``glacier_restore_tier_type`` (Standard | Expedited)
96+
* ``glacier_restore_tier_type`` (``Standard`` | ``Expedited``)
8097

81-
The type of retrieval of the object archived on the cloud service. Supported options are ``Standard`` and ``Expedited``.
98+
The type of retrieval within the cloud service, which may represent different
99+
pricing. Supported options are ``Standard`` and ``Expedited``.
82100

83101

84102
For example:
85103

86-
::
87-
104+
```
88105
# radosgw-admin zonegroup placement add --rgw-zonegroup=default \
89106
--placement-id=default-placement \
90107
--storage-class=CLOUDTIER-GLACIER --tier-type=cloud-s3-glacier
@@ -142,121 +159,133 @@ For example:
142159
}
143160
}
144161
]
162+
```
145163

146164

165+
Examples of Restore Objects:
166+
----------------------------
147167

148-
Examples to Restore Object:
149-
---------------------------
150-
151-
Using S3 RestoreObject CLI
152-
~~~~~~~~~~~~~~~~~~~~~~~~~
153-
154-
Below options of `S3 restore-object <https://docs.aws.amazon.com/cli/latest/reference/s3api/restore-object.html>`__ CLI are supported -
155-
156-
157-
Syntax
158-
159-
::
168+
Using the S3 RestoreObject CLI
169+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
160170

171+
Th `S3 restore-object <https://docs.aws.amazon.com/cli/latest/reference/s3api/restore-object.html>`
172+
CLI supports these options:
161173

174+
```
162175
$ aws s3api restore-object
163176
--bucket <value>
164177
--key <value>
165178
[--version-id <value>]
166179
--restore-request (structure) {
167180
Days=<integer>
168181
}
182+
```
169183

170-
Note: ``Days`` is optional and if not provided, the object is restored permanently
184+
Note: ``Days`` is optional and if not provided, the object is restored permanently.
171185

172186
Example 1:
173187

174-
::
175-
188+
```
176189
$ aws s3api restore-object --bucket bucket1 --key doc1.rtf
177190
[--version-id 3sL4kqtJlcpXroDTDmJ+rmSpXd3dIbrHY+MTRCxf3vjVBH40Nr8X8gdRQBpUMLUo]
178191
--restore-request Days=10
179192
....
193+
```
180194

181-
This will restore the object `doc1.rtf` of the given version for the duration of 10 days.
182-
183-
195+
This will restore the object ``doc1.rtf`` at an optional version,
196+
for the duration of 10 days.
184197

185198
Example 2:
186199

187-
::
188-
200+
```
189201
$ aws s3api restore-object --bucket bucket1 --key doc2.rtf --restore-request {} ....
202+
```
190203

191-
This will restore the object `doc2.rtf` permanently and will be treated as regular object.
192204

205+
This will restore the object ``doc2.rtf`` permanently and it will be treated as regular object.
193206

194207

195-
Using S3 GetObject CLI
196-
~~~~~~~~~~~~~~~~~~~~~
197-
Ensure ``allow_read_through`` tier-config option is enabled.
208+
Using the S3 GetObject CLI
209+
~~~~~~~~~~~~~~~~~~~~~~~~~~
198210

199-
Example 3:
211+
Ensure that the ``allow_read_through`` tier-config option is enabled.
200212

201-
::
213+
Example 3:
202214

215+
```
203216
$ aws s3api get-object --bucket bucket1 --key doc3.rtf ....
217+
```
204218

205-
This will restore the object `doc3.rtf` for the duration of the ``read_through_restore_days`` configured.
206-
219+
This will restore the object `doc3.rtf`` for ``read_through_restore_days`` days.
207220

208-
Note: The above CLI command may time out if the object restoration takes too long. Before reissuing the command, you can verify the restoration status.
221+
Note: The above CLI command may time out if object restoration takes too long.
222+
You can verify the restore status before reissuing the command.
209223

210224

211-
Verifying the restore status
212-
----------------------------
213-
Verify the status of the restore by running an `S3 HeadObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_ResponseSyntax>`__ request. The response includes ``x-amz-restore`` header if either the object restoration is in progress or a copy of it is already restored.
225+
Verifying the restoration status
226+
--------------------------------
227+
Verify the status of the restoration by issuing
228+
an `S3 HeadObject <https://docs.aws.amazon.com/AmazonS3/latest/API/API_HeadObject.html#API_HeadObject_ResponseSyntax>`
229+
request. The response includes the ``x-amz-restore`` header if object restoration
230+
is in progress or a copy of it is already restored.
214231

215-
Example,
216-
217-
::
232+
Example:
218233

234+
```
219235
$ aws s3api head-object --key doc1.rtf --bucket bucket1 ....
236+
```
220237

238+
The ``radosgw-admin`` CLI can be used to check restoration status and other
239+
details.
221240

222-
In addition, ``radosgw-admin`` CLI can be used to check the restoration status and other details on the RGW server.
223-
224-
Example,
241+
Example:
225242

226-
::
227-
243+
```
228244
$ radosgw-admin object stat --bucket bucket1 --object doc1.rtf
245+
```
229246

230247

231248
Restored Object Properties
232249
--------------------------
233250

234-
235251
Storage
236252
~~~~~~
237-
The objects are restored to the storage class configured for ``restore_storage_class`` tier-config option. However, as per `AWS S3 RestoreObject <https://docs.aws.amazon.com/cli/latest/reference/s3api/restore-object.html>`__ the storage class of restored objects should remain unchanged. Therefore, for temporary copies, the x-amz-storage-class will continue to reflect the original cloud-tier storage class.
253+
Objects are restored to the storage class configured via ``restore_storage_class``
254+
in the tier-config. However, as
255+
per `<https://docs.aws.amazon.com/cli/latest/reference/s3api/restore-object.html>`
256+
the storage class of restored objects should remain unchanged. Therefore, for
257+
temporary copies, the ```x-amz-storage-class``` will continue to reflect the
258+
original cloud-tier storage class.
238259

239260

240261
mtime
241262
~~~~
242-
The `mtime` of the transitioned and restored objects should remain unchanged.
263+
The ``mtime`` of the transitioned and restored objects should remain unchanged.
243264

244265

245266
Lifecycle
246267
~~~~~~~~
247-
`Temporary` copies are not subjected to any further transition to the cloud. However (as is the case with `cloud-transitioned objects`) they can be deleted via regular LC expiration rules or via external S3 Delete request.
248-
`Permanent` copies are treated as any regular objects and are subjected to any LC rules applicable.
268+
``Temporary`` copies are not subject to transition to the cloud. However, as is the
269+
case with cloud-transitioned objects, they can be deleted via regular LC (Life Cycle)
270+
expiration rules or an external S3 ``delete`` request.
271+
272+
``Permanent`` copies are treated as regular objects and are subject to applicable LC
273+
policies.
249274

250275

251276
Replication
252277
~~~~~~~~~~
253-
`Temporary` copies are not replicated and will be retained only on the zones the restore request is initiated on.
254-
`Permanent` copies are replicated like other regular objects.
278+
``Temporary`` copies are not replicated and will be retained only by the zone
279+
on which the restore request is initiated.
280+
281+
``Permanent`` copies are replicated like other regular objects.
255282

256283

257284
Versioned Objects
258285
~~~~~~~~~~~~~~~~
259-
For versioned objects, if an object has been `cloud-transitioned`, it would be in a non-current state. After a restore, the same non-current object will be updated with the downloaded data, and its HEAD object will be modified accordingly.
286+
For versioned objects, if an object has been cloud-transitioned, it is in a
287+
non-current state. After a restore, the same non-current object will be
288+
updated with the downloaded data, and its ``HEAD`` object will be modified accordingly.
260289

261290

262291

0 commit comments

Comments
 (0)