FIX: Fix DBNStore reader to read across zstd frame

nmacholl · nmacholl · commit 6e059f99797c · 2025-06-10T19:17:36.000Z
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,9 +1,12 @@
 # Changelog
 
-## 0.57.0 - TBD
+## 0.56.1 - TBD
 
 #### Deprecations
-- Deprecated `int` and `pd.Timestamp` types for `start_date` and `end_date` parameters which will be removed in v0.59.0
+- Deprecated `int` and `pd.Timestamp` types for `start_date` and `end_date` parameters which will be removed in a future release
+
+#### Bug fixes
+- Fixed an issue where the zstandard frame size could limit the size of `DataFrame` objects returned by `DBNStore.to_df()` when a `count` was specified
 
 ## 0.56.0 - 2025-06-03
 
diff --git a/databento/common/dbnstore.py b/databento/common/dbnstore.py
@@ -535,6 +535,7 @@ def reader(self) -> IO[bytes]:
         if self.compression == Compression.ZSTD:
             return zstandard.ZstdDecompressor().stream_reader(
                 self._data_source.reader,
+                read_across_frames=True,
             )
         return self._data_source.reader
 

Original file line number	Diff line number	Diff line change
`@@ -535,6 +535,7 @@ def reader(self) -> IO[bytes]:`
`535`	`535`	`if self.compression == Compression.ZSTD:`
`536`	`536`	`return zstandard.ZstdDecompressor().stream_reader(`
`537`	`537`	`self._data_source.reader,`
	`538`	`+ read_across_frames=True,`
`538`	`539`	`)`
`539`	`540`	`return self._data_source.reader`
`540`	`541`