Skip to content

Commit f47d304

Browse files
New petastorm release for local disk cache (#818)
* New petastorm release for local disk cache * update from rc to major version
1 parent 3cae688 commit f47d304

File tree

2 files changed

+17
-3
lines changed

2 files changed

+17
-3
lines changed

docs/release-notes.rst

Lines changed: 16 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,12 +4,26 @@
44
Release notes
55
=============
66

7-
Release 0.13.1 (unreleased)
7+
Release 0.13.2 (unreleased)
88
===========================
99

10+
Release 0.13.1
11+
===========================
12+
- `PR 814 <https://github.com/uber/petastorm/pull/814>`_: Support multiple incremental improvements to local disk cache to prepare for Michelangelo + Ray + disk cache changes:
13+
- Close threads in teardown before removing underlying files or closing cache.
14+
- Use cache volume and size to detect whether to insert data in cache or not if eviction policy is none
15+
- Eviction policy can be set to none by the client. The reason for this is, if we warm the cache in first epoch for ~40% of data, we can just keep it cached for subsequent epoch as evicting it with newer data would basically reduce our cache hit to 0
16+
- `PR 815 <https://github.com/uber/petastorm/pull/815>`_: Add support for pre-converting from pyarrow to numpy at the time of reading. When this is enabled; it will allow us to save time when we use disk cache. This way whatever is being read from the cache is ready to be consumed by downstream petastorm libraries.
17+
- `PR 816 <https://github.com/uber/petastorm/pull/816>`_: Support to cleanup disk cache when enabled. This change will also enable the cleanup=True for local disk cache as that will be the right thing to do most of the times.
18+
- `PR 817 <https://github.com/uber/petastorm/pull/817>`_: Reshuffle the data after reading from cache if shuffle_rows is true.
19+
20+
New features
21+
--------------------------
22+
- **Cache training data to local disk to improve GPU utilization**: The local disk cache can be used to cache training data to local disk to improve GPU utilization. This is useful when training data is large and can be cached to local disk to improve GPU utilization.
1023

24+
- ``enable_local_disk_cache: True``: Enable all the relevant configurations to use local disk cache.
1125

12-
Release 0.13.0
26+
Release 0.13.0
1327
===========================
1428
- `PR 810 <https://github.com/uber/petastorm/pull/810>`_: Support reproducible ordering in Petastorm.
1529
- `PR 809 <https://github.com/uber/petastorm/pull/809>`_: Remove CI tests for unsupported PyArrow versions (3.x and 4.x)

petastorm/__init__.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,4 +16,4 @@
1616
from petastorm.reader import make_reader, make_batch_reader # noqa: F401
1717
from petastorm.transform import TransformSpec # noqa: F401
1818

19-
__version__ = '0.13.0rc0'
19+
__version__ = '0.13.1'

0 commit comments

Comments
 (0)