|
1 | | -- Proposal Name: (`undelete`) |
| 1 | +- Proposal Name: (`restore_api`) |
2 | 2 | - Start Date: 2025-02-04 |
3 | 3 | - RFC PR: [apache/opendal#7178](https://github.com/apache/opendal/pull/7178) |
4 | 4 | - Tracking Issue: [apache/opendal#4321](https://github.com/apache/opendal/issues/4321) |
5 | 5 |
|
6 | 6 | # Summary |
7 | 7 |
|
8 | | -Implement two complementary approaches for restoring deleted files in OpenDAL: |
| 8 | +Implement restoring a deleted version or a file. |
| 9 | + |
| 10 | +# Motivation |
| 11 | + |
| 12 | +Cloud storage providers implement data recovery through different mechanisms: |
| 13 | + |
| 14 | +- Example 1: **AWS S3** uses versioning exclusively. Deleted objects become "delete markers" and can be restored by copying a previous version or removing the delete marker. |
| 15 | +- Example 2: **GCS and Azure Blob Storage** provide both versioning AND soft delete as separate, independent features that can be enabled separately or together. |
| 16 | + |
| 17 | +Currently, there is no standardized way in OpenDAL to restore deleted objects across these different paradigms. This creates challenges for downstream projects that need data recovery capabilities. For example, iceberg-java [already supports file restore](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/io/SupportsRecoveryOperations.java) in some cloud environments, while iceberg-rust, written with OpenDAL, does not provide this functionality yet. |
| 18 | + |
| 19 | +In order to secure data pipelines, users currently have to maintain implementations of restore logic across multiple languages and storage backends. Having this functionality in OpenDAL would centralize this capability and make it available to all downstream users. |
| 20 | + |
| 21 | +# Guide-level explanation |
| 22 | + |
| 23 | +OpenDAL provides two ways to restore deleted files, depending on the storage backend's capabilities: |
9 | 24 |
|
10 | 25 | 1. **Version-based restoration**: Extend the `copy` operation with an optional source `version` parameter to enable promoting non-current versions to the current version (for storage systems with versioning support) |
11 | 26 | 2. **Soft delete restoration**: Add an `undelete` operation for storage systems that implement soft delete as a distinct feature from versioning (GCS and Azure Blob Storage) |
12 | | -3. Provide a high-level `restore` API that automatically chooses the right approach based on service capabilities, defaulting to versioning approach: |
| 27 | + |
| 28 | +We might also provide a high-level `restore` API that automatically chooses the right approach based on service capabilities, defaulting to versioning approach: |
13 | 29 |
|
14 | 30 | ```rust |
15 | 31 | impl Operator { |
@@ -39,20 +55,6 @@ impl Operator { |
39 | 55 | } |
40 | 56 | ``` |
41 | 57 |
|
42 | | -# Motivation |
43 | | - |
44 | | -Cloud storage providers implement data recovery through different mechanisms: |
45 | | - |
46 | | -- Example 1: **AWS S3** uses versioning exclusively. Deleted objects become "delete markers" and can be restored by copying a previous version or removing the delete marker. |
47 | | -- Example 2: **GCS and Azure Blob Storage** provide both versioning AND soft delete as separate, independent features that can be enabled separately or together. |
48 | | - |
49 | | -Currently, there is no standardized way in OpenDAL to restore deleted objects across these different paradigms. This creates challenges for downstream projects that need data recovery capabilities. For example, iceberg-java [already supports file restore](https://github.com/apache/iceberg/blob/main/api/src/main/java/org/apache/iceberg/io/SupportsRecoveryOperations.java) in some cloud environments, while iceberg-rust, written with OpenDAL, does not provide this functionality yet. |
50 | | - |
51 | | -In order to secure data pipelines, users currently have to maintain implementations of restore logic across multiple languages and storage backends. Having this functionality in OpenDAL would centralize this capability and make it available to all downstream users. |
52 | | - |
53 | | -# Guide-level explanation |
54 | | - |
55 | | -OpenDAL provides two ways to restore deleted files, depending on the storage backend's capabilities: |
56 | 58 |
|
57 | 59 | ## Approach 1: Version-Based Restoration (Copy with Version) |
58 | 60 |
|
|
0 commit comments