Skip to content

Releases: awslabs/s3-connector-for-pytorch

v1.5.0

20 Feb 13:17
c4f4826

Choose a tag to compare

v1.5.0 (February 20, 2026)

New features

  • Add DCPOptimizedS3Reader as new default for faster and partial DCP loading (#378, #419)
  • Add support for Python 3.14 (#408)
  • Add weights_only parameter support for Lightning 2.6.0 compatibility (#388)

Bug fixes

  • Add py.typed file to enable external type checking (#406)
  • Override S3Writer closed property and block writes after close (#360)
  • Fix SequentialS3Reader to stay at EOF when seeking beyond object size (#362)

Other changes

  • Add seekable() method in S3Reader to eliminate tensor copies during DCP loading (#359)
  • Add load ordering optimization to S3StorageReader for sequential access patterns (#372)
  • Add os, arch, and PyTorch version to user agent string (#397)
  • Add benchmark to run DCP Loading Workloads (#357)
  • Add thread_count parameter to S3StorageWriter (#370)
  • Add macOS x86_64 and Python 3.8 deprecation warnings (#400)

Breaking changes

  • No breaking changes, but DCPOptimizedS3Reader as the new default reader for S3StorageReader might lead to behavioral changes. See DCPOptimizedS3Reader Errors for more details.

Full Changelog: v1.4.3...v1.5.0

v1.4.3

25 Jul 16:45
e6ec32c

Choose a tag to compare

v1.4.3 (July 25, 2025)

New features

  • Add support for Python 3.13 (#350)

Other changes

  • Modify test fixtures to mitigate CI/CD S3Express boto3 400 errors (#347)
  • Add profile config guide and improve docs (#349)

Breaking changes

  • No breaking changes.

Full Changelog: v1.4.2...v1.4.3

v1.4.2

14 Jul 14:00
9688aab

Choose a tag to compare

v1.4.2 (July 14, 2025)

New features

  • Implement range-based S3 Reader for efficient partial read patterns (#339)
  • Implemented AWS profile credentials support as part of S3ClientConfig (#341)

Bug fixes

  • Enable multiple CRT clients per process with different configs (#340)

Other changes

  • Expose configuration needed for testing profile credentials (#344)
  • Keep usage of manylinux_2014 for wider suport of glibc (#342, #343)

Breaking changes

  • No breaking changes.

Full Changelog: v1.4.1...v1.4.2

v1.4.1

21 May 13:18
e7b238a

Choose a tag to compare

v1.4.1 (May 20, 2025)

New features

  • Add S3 prefix strategies to prevent throttling in distributed checkpointing (#327)

Bug fixes

  • Consume mountpoint-s3-client 0.14.1 to address AWS_ERROR_HTTP_RESPONSE_FIRST_BYTE_TIMEOUT error (#332)
  • Prevent S3Writer stream closure on exception to preserve original error context (#331)
  • Add PyTorch 2.7.0 support (#329)

Other changes

  • Add Python version to user agent string (#333)
  • Address DCP test hanging and distributed test errors (#330)
  • Update GitHub Actions runners to ubuntu-24.04 (#328)

Breaking changes

  • No breaking changes.

Full Changelog: v1.4.0...v1.4.1

v1.4.0 (April 14, 2025)

14 Apr 09:47
b8ec748

Choose a tag to compare

v1.4.0 (April 14, 2025)

New features

  • Introducing support of FSDP in DCP benchmark (#313)
  • Exposing max_attempts setting through S3ClientConfig and adding support of S3ClientConfig to DCP (#322)

Bug fixes

  • Consume mountpoint-s3-client changes for race condition on GET request path, that may lead to an empty response (#324)

Other changes

  • Update Pyo3 version (#314)

Breaking changes

  • We changed the way we handle fork operations. The CRT client used for communication with S3 is not stable during fork due to global state and locks held by background threads. To address this, we now clean up all existing CRT clients before a fork operation and create new CRT clients in the child process. This change prevents segfaults and hanging GET requests for training workloads that rely on fork (#320)

Full Changelog: v1.3.2...v1.4.0

v1.3.2

06 Feb 11:09
032d11a

Choose a tag to compare

v1.3.2 (February 5, 2025)

New features

  • Consume mountpoint-s3-client changes with support for dots in bucket name for COPY operation introduced in CRT (#300)
  • Escape special characters in rename operation (#297)
  • Support dots in bucket name for rename operation (#300)
  • Handle torch.load changes in PyTorch 2.6 (#306)
  • Remove dependency on mountpoint-s3-crt (#307)

Breaking changes

  • Internal S3Client now returns HeadObjectResult instead of ObjectInfo. The only difference is that HeadObjectResult doesn't have key field.

Full Changelog: v1.3.1...v1.3.2

v1.3.1

10 Jan 17:39
79582bd

Choose a tag to compare

v1.3.1 (January 10, 2025)

New features

  • Fix Rust build (#275)
  • Update user agent for DCP (#276)
  • Address Rust security issue (#279)
  • Refactor(benchmarks): Overhaul Lightning Checkpointing, DCP, dataset scenarios; add DynamoDB writes and results exploitation notebook (#274, #280, #285, #286)
  • Add single rank PyTorch checkpoint benchmark (#289)
  • Update torch version restriction (<2.5.0) and bind torchdata to last version with DataPipes (#283)

Breaking changes

  • No breaking changes.

Full Changelog: v1.3.0...v1.3.1

v1.3.0

22 Nov 13:30

Choose a tag to compare

v1.3.0 (November 22, 2024)

New features

  • Add support of PyTorch distributed checkpoints (#269)
  • Extend benchmark framework to support distributed checkpoints (#269)
  • Add support of distributed training to S3IterableDataset (#269)

Breaking changes

  • No breaking changes.

v1.2.7

29 Oct 14:20
732ca96

Choose a tag to compare

v1.2.7 (October 29, 2024)

New features

v1.2.6

09 Oct 16:27
b9831b5

Choose a tag to compare

v1.2.6 (October 9, 2024)

New features

  • Add support of PyTorch Lightning checkpoints to benchmark suit (#226).

Bug fixes

  • Fix potential race condition while instantiating the S3Client (#237).

Breaking changes

  • No breaking changes.