Skip to content

Commit 0d43341

Browse files
Add information about S3 Versioning to README (#170)
* Add information about S3 Versioning to README Co-authored-by: James Bornholt <[email protected]> --------- Co-authored-by: Ilya Isaev <[email protected]> Co-authored-by: James Bornholt <[email protected]>
1 parent 10246b9 commit 0d43341

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

README.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -149,6 +149,15 @@ trainer = Trainer(
149149
trainer.fit(model)
150150
```
151151

152+
## Using S3 Versioning to Manage Checkpoints
153+
When working with model checkpoints, you can use the [S3 Versioning](https://docs.aws.amazon.com/AmazonS3/latest/userguide/Versioning.html) feature to preserve, retrieve, and restore every version of your checkpoint objects. With versioning, you can recover more easily from unintended overwrites or deletions of existing checkpoint files due to incorrect configuration or multiple hosts accessing the same storage path.
154+
155+
When versioning is enabled on an S3 bucket, deletions insert a delete marker instead of removing the object permanently. The delete marker becomes the current object version. If you overwrite an object, it results in a new object version in the bucket. You can always restore the previous version. See [Deleting object versions from a versioning-enabled bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/DeletingObjectVersions.html) for more details on managing object versions.
156+
157+
To enable versioning on an S3 bucket, see [Enabling versioning on buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/manage-versioning-examples.html). Normal Amazon S3 rates apply for every version of an object stored and transferred. To customize your data retention approach and control storage costs for earlier versions of objects, use [object versioning with S3 Lifecycle](https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lifecycle-mgmt.html).
158+
159+
S3 Versioning and S3 Lifecycle are not supported by S3 Express One Zone.
160+
152161
## Contributing
153162
We welcome contributions to Amazon S3 Connector for PyTorch. Please
154163
see [CONTRIBUTING](https://github.com/awslabs/s3-connector-for-pytorch/blob/main/doc/CONTRIBUTING.md)

0 commit comments

Comments
 (0)