v0.9.1 (2022-01-19)
New features:
- Added support for compare dictionary-encoded with scalar #686 (jorgecarleitao)
Fixed bugs:
- Allowed passing
Noneas ipc_fields in flight API #780 (jorgecarleitao)
Enhancements:
- Read dict binary from parquet #781 (jorgecarleitao)
- Added support to read and write float dict from parquet #778 (jorgecarleitao)
Testing updates:
- Fixed CI for SIMD #779 (jorgecarleitao)
v0.9.0 (2022-01-14)
Breaking changes:
- Added number of rows read in CSV inference #765 (jorgecarleitao)
- Refactored
nullif#753 (jorgecarleitao) - Migrated to latest parquet2 #752 (jorgecarleitao)
- Replace flatbuffers dependency by Planus #732 (jorgecarleitao)
- Simplified
SchemaandField#728 (jorgecarleitao) - Replaced
RecordBatchbyChunk#717 (jorgecarleitao) - Removed
Optionfrom fields' metadata #715 (jorgecarleitao) - Moved dict_id to IPC-specific IO #713 (jorgecarleitao)
- Moved is_ordered from
FieldtoDataType::Dictionary#711 (jorgecarleitao) - Refactored JSON writing (5-10x) #709 (jorgecarleitao)
- Made Avro read API use
BlockandCompressedBlock#698 (jorgecarleitao) - Simplified most traits #696 (jorgecarleitao)
- Replaced
DisplaybyDebugforArray#694 (jorgecarleitao) - Replaced
MutableBufferbystd::Vec#693 (jorgecarleitao) - Simplified
Utf8ScalarandBinaryScalar#660 (jorgecarleitao) - Simplified Primitive and Boolean scalar #648 (jorgecarleitao)
New features:
- Add
and_scalarandor_scalarfor boolean_kleene #662 - Add
loweranduppersupport for string #635 - Added support to cast decimal #761 (jorgecarleitao)
- Added support to deserialize JSON (!= NDJSON) #758 (jorgecarleitao)
- Added support to infer nested json structs #750 (jorgecarleitao)
- Added support to compare intervals #746 (jorgecarleitao)
- Added
anyandallkernel #739 (ritchie46) - Added support to write Avro async #736 (jorgecarleitao)
- Added support to write interval to Avro #734 (jorgecarleitao)
- Added
and_scalarandor_scalarfor boolean kleene #723 (silathdiir) - Added
and_scalarandor_scalarfor boolean #707 (silathdiir) - Refactored JSON read to split IO-bounded from CPU-bounded tasks #706 (jorgecarleitao)
- Added more conversions from parquet #701 (jorgecarleitao)
- Added support for compressed Avro write #699 (jorgecarleitao)
- Added support to write to Avro #690 (jorgecarleitao)
- Added dynamic version of negation #685 (jorgecarleitao)
- Added support to read dictionary-encoded required parquet pages #683 (mdrach)
- Added
upper#664 (Xuanwo) - Added
lower#641 (Xuanwo) - Added support for
asyncread of Avro #620 (jorgecarleitao)
Fixed bugs:
- Pyarrow and Arrow2 don't agree on Timestamp resolution #700
- Writing compressed dictionary in parquet corrupts the files #667
- Replaced assert by error in IPC read #748 (jorgecarleitao)
- Made all panics in IPC read errors #722 (jorgecarleitao)
- Fixed error in compare booleans #721 (jorgecarleitao)
- Fixed error in dispatching scalar arithmetics #682 (jorgecarleitao)
- Fixed error in reading negative decimals from parquet #679 (mdrach)
- Made IPC reader less restrictive #678 (jorgecarleitao)
- Fixed error in trait constraint in compute #665 (jorgecarleitao)
- Fixed performance regression of CSV reading #657 (jorgecarleitao)
- Fixed filter of predicate with validity #653 (ritchie46)
- Made
Scalar: Send+Sync#644 (jorgecarleitao)
Enhancements:
- Feature: JSON IO? #712
- Simplified code #760 (jorgecarleitao)
- Added iterator of values of
FixedBinaryArray#757 (jorgecarleitao) - Remove un-needed
unsafe#756 (jorgecarleitao) - Replaced un-needed
unsafe#755 (jorgecarleitao) - Made IO
#[forbid(unsafe)]#749 (jorgecarleitao) - Improved reading nullable Avro arrays #727 (Igosuki)
- Allow to create primitive array by vec without extra memcopy #710 (sundy-li)
- Removed requirement of
use Arrayto access primitives'data_type#697 (jorgecarleitao) - Cleaned up trait usage and added forbid_unsafe to parts #695 (jorgecarleitao)
- Migrated from
avro-rstoavro-schema#692 (jorgecarleitao) - Added
MutablePrimitiveArray::extend_constant#689 (jorgecarleitao) - Do not write validity without nulls in IPC #688 (jorgecarleitao)
- DRY code via macro #681 (jorgecarleitao)
- Made
dyn ArrayandScalarusable in#[derive(PartialEq)]#680 (jorgecarleitao) - Made IPC ZSTD-compressed consumable by pyarrow #675 (jorgecarleitao)
- Simplified trait bounds in arithmetics #671 (jorgecarleitao)
- Improved performance of reading utf8 required from parquet (-15%) #670 (jorgecarleitao)
- Avoid double utf8 checks on MutableUtf8 -> Utf8 #655 (jorgecarleitao)
- Made
Buffer::offsetpublic #652 (ritchie46) - Improved performance in cast Primitive to Binary/String (2x) #646 (sundy-li)
- Made
Filter: Send+Sync#645 (jorgecarleitao) - Made API to create field accept
String#643 (jorgecarleitao)
Documentation updates:
- Fixed clippy (coming from 1.58) #763 (jorgecarleitao)
- Described how to run part of the tests #762 (jorgecarleitao)
- Improved README #735 (jorgecarleitao)
- clarify boolean value in DataType::Dictionary #718 (ritchie46)
- readme typo #687 (max-sixty)
- Added example to read parquet in parallel with rayon #658 (jorgecarleitao)
- Added documentation to
Bitmap::as_slice#654 (ritchie46)
Testing updates:
- Improved json tests #742 (jorgecarleitao)
- Added integration tests for writing compressed parquet #740 (jorgecarleitao)
- Updated patch for integration test #731 (jorgecarleitao)
- Added cargo check to benchmarks #730 (sundy-li)
- More tests to CSV writing #724 (jorgecarleitao)
- Added integration tests for other compressions with parquet from pyarrow #674 (jorgecarleitao)
- Bumped nightly in CI #672 (jorgecarleitao)
- Invalidate caches from CI. #656 (jorgecarleitao)
v0.8.1 (2021-11-27)
Fixed bugs:
v0.8.0 (2021-11-27)
Breaking changes:
- Made CSV write options use chrono formatting by default #624
- Add
compressiontoIpcWriteOptions#570 - Made
castacceptCastOptionsparameter #569 - Simplified
ArrowError#640 (jorgecarleitao) - Use
DynComparatorforlexsortandpartition#637 (yjshen) - Split "compute" feature #634 (jorgecarleitao)
- Removed unneeded trait. #628 (jorgecarleitao)
- Sealed 2 traits to forbid downstream implementations #621 (jorgecarleitao)
- Simplified arithmetics compute #607 (jorgecarleitao)
- Refactored comparison
Operator#604 (jorgecarleitao) - Simplified dictionary indexes #584 (jorgecarleitao)
- Simplified IPC APIs #576 (jorgecarleitao)
- Simplified IPC stream writer / remove finish on drop from stream writer #575 (jorgecarleitao)
- Simplified trait in compute. #572 (jorgecarleitao)
- Compute: add partial option into CastOptions #561 (sundy-li)
- Introduced
UnionModeenum #557 (simonvandel) - Changed DataType::FixedSize*(i32) to DataType::FixedSize*(usize) #556 (simonvandel)
New features:
- Added support to write timestamps with timezones for CSV #623 (jorgecarleitao)
- Added support to read Avro files' metadata asynchronously #614 (jorgecarleitao)
- Added iterator for
StructArray#613 (illumination-k) - Added support to read snappy-compressed Avro #612 (jorgecarleitao)
- Added support to read decimal from csv #602 (jorgecarleitao)
- Added support to cast
NullArrayto all other types #589 (flaneur2020) - Added support dictionaries in nested types over IPC #587 (jorgecarleitao)
- Added support to write Arrow IPC streams asynchronously #577 (jorgecarleitao)
- Added support to write compressed Arrow IPC (feather v2) #566 (jorgecarleitao)
- Added support for ffi for
FixedSizeListandFixedSizeBinary#565 (jorgecarleitao) - Added support for
asynccsv reading. #562 (jorgecarleitao) - Added support for
bitwiseoperations #553 (1aguna) - Added support to read
StructArrayfrom parquet #547 (jorgecarleitao)
Fixed bugs:
- Fixed error in reading nullable from Avro. #631 (jorgecarleitao)
- Fixed error in union FFI #625 (jorgecarleitao)
- Fixed error in computing projection in
io::ipc::read::reader::FileReader#596 (illumination-k) - Fixed error in compressing IPC LZ4 #593 (jorgecarleitao)
- Fixed growable of dictionaries negative keys #582 (ritchie46)
- Made substring kernel on utf8 take chars into account. #568 (ritchie46)
- Fixed error in passing sliced arrays via FFI #564 (jorgecarleitao)
Enhancements:
- Faster
takewith null values (2-3x) #633 (jorgecarleitao) - Improved error message for missing feature in compressed parquet #632 (jorgecarleitao)
- Added
toconversion toFixedSizeBinary#622 (ritchie46) - Bumped
confy-table#618 (jorgecarleitao) - Made
MutableArraySend + Sync#617 (jorgecarleitao) - Removed most of allocations in IPC reading #611 (jorgecarleitao)
- Speed up boolean comparison kernels (~3x) #610 (Dandandan)
- Improved performance of decimal arithmetics #605 (jorgecarleitao)
- Simplified traits and added documentation #603 (jorgecarleitao)
- Improved performance of
is_not_null. #600 (jorgecarleitao) - Added
lento every array #599 (jorgecarleitao) - Added support for
NullArrayat FFI. #598 (jorgecarleitao) - Optimized
MutableBinaryArray#597 (jorgecarleitao) - Speedup/simplify bitwise operations (avoid extra allocation) #586 (Dandandan)
- Improved performance of
bitmap::from_trusted(3x) #578 (jorgecarleitao) - Made bitmap not cache null count #563 (jorgecarleitao)
- Avoided redundant checks in creating an
Utf8ArrayfromMutableUtf8Array#560 (jorgecarleitao) - Avoid unnecessary allocations #559 (simonvandel)
- Surfaced errors in reading from avro #558 (jorgecarleitao)
Documentation updates:
- Simplified example #619 (jorgecarleitao)
- Made example of parallel parquet write be over multiple batches #544 (jorgecarleitao)
Testing updates:
- Cleaned up benches #636 (jorgecarleitao)
- Ignored tests code in coverage report #615 (yjhmelody)
- Added more tests #601 (jorgecarleitao)
- Mitigated
RUSTSEC-2020-0159#595 (jorgecarleitao) - Added more tests #591 (jorgecarleitao)
v0.7.0 (2021-10-29)
Breaking changes:
- Simplified reading parquet #532 (jorgecarleitao)
- Change IPC
FileReaderto own the underlying reader #518 (blakesmith) - Migrate to
arrow_formatcrate #517 (jorgecarleitao)
New features:
- Added read of 2-level nested lists from parquet #548 (jorgecarleitao)
- add dictionary serialization for csv-writer #515 (ritchie46)
- Added
checked_negateandwrapping_negateforPrimitiveArray#506 (yjhmelody)
Fixed bugs:
- Fixed error in reading fixed len binary from parquet #549 (jorgecarleitao)
- Fixed ffi of sliced arrays #540 (jorgecarleitao)
- Fixed s3 example #536 (jorgecarleitao)
- Fixed error in writing compressed parquet dict pages #523 (jorgecarleitao)
- Validity taken into account when writing
StructArrayto json #511 (VasanthakumarV)
Enhancements:
- Bumped Prost and Tonic #550 (PsiACE)
- Speedup scalar boolean operations #546 (Dandandan)
- Added fast path for validating ASCII text (~1.12-1.89x improvement on reading ASCII parquet data) #542 (Dandandan)
- Exposed missing APIs to write parquet in parallel #539 (jorgecarleitao)
- improve utf8 init validity #530 (ritchie46)
- export missing
BinaryValueIter#526 (yjhmelody)
Documentation updates:
- Added more IPC documentation #534 (HagaiHargil)
- Fixed clippy and fmt #521 (ritchie46)
Testing updates:
- Added more tests for
utf8#543 (jorgecarleitao) - Ignored RUSTSEC-2020-0071 and RUSTSEC-2020-0159 #537 (jorgecarleitao)
- Improved parquet read benches #533 (jorgecarleitao)
- Added fmt and clippy checks to CI. #522 (xudong963)
v0.6.2 (2021-10-09)
New features:
Fixed bugs:
- Do not check offsets or utf8 validity in ffi (#505) #510 (NilsBarlaug)
- Made
try_push_validpublic again #509 (ritchie46)
Enhancements:
v0.6.1 (2021-10-07)
Breaking changes:
- Bring
MutableFixedSizeListArrayto the spec used by the rest of the Mutable API #475 - Removed
ALIGNMENTinvariant from[Mutable]Buffer#449 - Un-nested
compute::arithemtics::basic#461 (jorgecarleitao) - Added more serialization options for csv writer. #453 (ritchie46)
- Changed validity from
&Option<Bitmap>toOption<&Bitmap>. #431 (jorgecarleitao) - Bumped parquet2 #422 (jorgecarleitao)
- Changed IPC
FileWriterto own thewriter. #420 (yjshen) - Made
DynComparatorSend+Sync#414 (yjshen)
New features:
- Read Decimal from Parquet File #444
- Add IO read for Avro #401
- Added support to read Avro logical types,
List,Enum,DurationandFixed. #493 (jorgecarleitao) - Added read
Decimalfrom parquet #489 (potter420) - Implement
BitXortrait forBitmap#485 (houqp) - Added
extend/extend_uncheckedforMutableBooleanArray#478 (VasanthakumarV) - expose
shrink_to_fitto mutable arrays #467 (ritchie46) - Added support for
DataType::MapandMapArray#464 (jorgecarleitao) - Extract parts of datetime #433 (VasanthakumarV)
- Added support to add an interval to a timestamp #417 (jorgecarleitao)
- Added support to read Avro. #406 (jorgecarleitao)
- Replaced own allocator by
std::Vec. #385 (jorgecarleitao)
Fixed bugs:
- crash in parquet read #459
- Made writing stream to parquet require a non-static lifetime #471 (GrandChaman)
- Made importing from FFI
unsafe#458 (jorgecarleitao) - Fixed panic in division using nulls. #438 (jorgecarleitao)
- Fixed error writing dictionary extension to IPC #397 (jorgecarleitao)
- Fixed error in extending
MutableBitmap#393 (jorgecarleitao)
Enhancements:
- Some
comparefunction are not exported #349 - Investigate how to add support for timezones in timestamp #23
- Made
hashwork for extension type #487 (jorgecarleitao) - Added
extend/extend_uncheckedforMutableBinaryArray#486 (VasanthakumarV) - Improved inference and deserialization of CSV #483 (jorgecarleitao)
- Added
GrowableFixedSizeListand improvedMutableFixedSizeListArray#470 (jorgecarleitao) - Added
MutableBitmap::shrink_to_fit#468 (jorgecarleitao) - Added
MutableArray::as_box#450 (sd2k) - Improved performance of sum aggregation via aligned loads (-10%) #445 (ritchie46)
- Removed
assertfromMutableBuffer::set_len#443 (ritchie46) - Optimized
null_count#442 (ritchie46) - Improved performance of list iterator (- 10-20%) #441 (ritchie46)
- Improved performance of
PrimitiveGrowablefor nulls (-10%) #434 (jorgecarleitao) - Allowed accessing validity without importing
Array#432 (jorgecarleitao) - Optimize hashing using
ahashandmultiversion(-30%) #428 (Dandandan) - Improved performance of iterator of
Utf8ArrayandBinaryArray(3-4x) #427 (jorgecarleitao) - Improved performance of utf8 validation of large strings via
simdutf8(-40%) #426 (Dandandan) - Added reading of parquet required dictionary-encoded binary. #419 (jorgecarleitao)
- Add
extend/extend_uncheckedforMutableUtf8Array#413 (VasanthakumarV) - Added support to extract hours and years from timestamps with timezone #412 (jorgecarleitao)
- Added
io_csv_readandio_csv_writefeature #408 (ritchie46) - Improve
comparisondocs and re-export the array-comparing function #404 (HagaiHargil) - Added support to read dict-encoded required primitive types from parquet #402 (Dandandan)
- Added
Array::with_validity#399 (ritchie46)
Documentation updates:
- Improved documentation #491 (jorgecarleitao)
- Added more API docs. #479 (jorgecarleitao)
- Added more documentation #476 (jorgecarleitao)
- Improved documentation #462 (jorgecarleitao)
- Added example showing parallel writes to parquet (x num_cores) #436 (jorgecarleitao)
- Improved documentation #430 (jorgecarleitao)
- [0.5] The docs
iomodule has no submodules #390 - Made docs be compiled with feature
full#391 (jorgecarleitao)
Testing updates:
- DRY via macro. #477 (jorgecarleitao)
- DRY of type check and len check code in
compute#474 (yjhmelody) - Added property testing #460 (jorgecarleitao)
- Added fmt to CI. #455 (jorgecarleitao)
- Simplified CI #452 (jorgecarleitao)
- fix filter kernels bench #440 (ritchie46)
- Reduced number of combinations in feature tests. #429 (jorgecarleitao)
- Move tests from
src/compute/totests/#423 (VasanthakumarV) - Skipped some feature permutations. #411 (jorgecarleitao)
- Added tests to some invariants of
unsafe#403 (jorgecarleitao) - Added support to read and write extension types to and from parquet #396 (jorgecarleitao)
- Fix testing of SIMD #394 (jorgecarleitao)
v0.5.3 (2021-09-14)
New features:
- Added support to read and write extension types to and from parquet #396 (jorgecarleitao)
Fixed bugs:
- Fixed error writing dictionary extension to IPC #397 (jorgecarleitao)
- Fixed error in extending
MutableBitmap#393 (jorgecarleitao)
Enhancements:
- Added support to read dict-encoded required primitive types from parquet #402 (Dandandan)
- Added
Array::with_validity#399 (ritchie46)
Testing updates:
- Fix testing of SIMD #394 (jorgecarleitao)
v0.5.1 (2021-09-09)
Documentation updates:
- [0.5] The docs
iomodule has no submodules #390 - Made docs be compiled with feature
full#391 (jorgecarleitao)
v0.5.0 (2021-09-07)
Breaking changes:
- Added
ExtensiontoDataType#361 MonthDayNanoadded to enumIntervalUnit#360- Make
io::parquet::write::write_*return size of file in bytes #354 - Renamed
bitmap::utils::null_counttobitmap::utils::count_zeros#342 - Made
GroupFilteroptional in parquet'sRecordReaderand added method to set it. #386 (jorgecarleitao) - Removed
PartialOrdandOrdof all enums indatatypes#379 (jorgecarleitao) - Made
cargofeatures not default #369 (jorgecarleitao) - Prepare APIs for extension types #357 (jorgecarleitao)
New features:
- Added support for
asyncparquet write #372 (GrandChaman) - Add support to extension types in FFI #363 (jorgecarleitao)
- Added support for field's metadata via FFI #362 (jorgecarleitao)
- Added support for
Extension(logical) type #359 (jorgecarleitao) - Added support for compute to
BinaryArray#346 (zhyass) - Added support for reading binary from CSV #337 (jorgecarleitao)
- Added support for
MONTH_DAY_NANOinterval type #268 (jorgecarleitao)
Fixed bugs:
- Parquet read skips a few rows at the end of the page #373
parquet_readfails when a column has too many rows with string values #366parquet_readpanics withindex_out_of_bounds#351- Fixed error in
MutableBitmap::push_unchecked#384 (jorgecarleitao) - Fixed display of timestamp with tz. #375 (jorgecarleitao)
Enhancements:
- Added
extend_*valuestoMutablePrimitiveArray#383 (ritchie46) - Improved performance of writing to CSV (20-25%) #382 (jorgecarleitao)
- Bumped
lexical-core#378 (jorgecarleitao) - Fixed casting of utf8 <> Timestamp with and without timezone #376 (jorgecarleitao)
- Added
Send+SynctoMutableBuffer#368 (jorgecarleitao) - Improved performance of unary _not_ for aligned bitmaps (3x) #365 (jorgecarleitao)
- Reduced dependencies within
num#353 (jorgecarleitao) - Bumped to parquet2 v0.4 #352 (jorgecarleitao)
- Bumped tonic and prost in flight #344 (PsiACE)
- Improved null count calculation (5x) #343 (jorgecarleitao)
- Improved perf of deserializing integers from json (30%) #340 (jorgecarleitao)
- Simplified code of json schema inference #339 (jorgecarleitao)
Documentation updates:
- Moved guide examples to examples/ #387 (jorgecarleitao)
- Added more docs. #358 (jorgecarleitao)
- Improved API docs. #355 (jorgecarleitao)
Testing updates:
- Moved tests to
tests/#389 (jorgecarleitao) - Moved compute tests to tests/ #388 (jorgecarleitao)
- Added more tests. #380 (jorgecarleitao)
- Pinned nightly in SIMD tests #364 (jorgecarleitao)
- Improved benches for take #348 (jorgecarleitao)
- Made IPC integration tests run tests that are not run by arrow-rs #278 (jorgecarleitao)
v0.4.0 (2021-08-24)
Breaking changes:
- Change dictionary iterator of values from
Arrays of one element toScalars #335 - Align FFI API with arrow's C++ API #328
- Make
*_compare_scalarnot returnResult#316 - Make
io::print,get_value_displayandget_displaynot returnResult#286 - Add
MetadataVersionto IPC interfaces #282 - Change
DataType::Unionto enable round trips in IPC #281 - Removed clone requirement in
StructArray -> RecordBatch#307 (jorgecarleitao) - Fixed error in reading a non-finished IPC stream. #302 (jorgecarleitao)
- Generalized ZipIterator to accept a
BitmapIter#296 (jorgecarleitao)
New features:
- Added API to FFI
Field#321 (jorgecarleitao) - Added
compare_scalar#317 (jorgecarleitao) - Add
UnionArray#283 (jorgecarleitao)
Fixed bugs:
- SliceIterator of last bytes is not correct #292
- Fixed error in displaying dictionaries with nulls in values #334 (jorgecarleitao)
- Fixed error in dict equality #333 (jorgecarleitao)
- Fixed small inconsistencies between
compute::castandcompute::can_cast#295 (jorgecarleitao) - Removed order implementation for
days_ms/Interval(DayTime)#285 (jorgecarleitao)
Enhancements:
- Added support for remaining non-nested datatypes #336 (jorgecarleitao)
- Made
multiversionandlexical-coreoptional #324 (jorgecarleitao) - Improved performance of utf8 comparison (1.7x-4x) #322 (jorgecarleitao)
- Improved performance of boolean comparison (5x-14x) #318 (jorgecarleitao)
- Added trait
TryPush#314 (jorgecarleitao) - Added cast
date32 -> i64anddate64 -> i32#308 (ritchie46) - Improved performance of comparison with SIMD feature flag (2x-3.5x) #305 (jorgecarleitao)
- Added support to read json to
BinaryArray#304 (jorgecarleitao) - Improved
MutableFixedSizeBinaryArray#303 (jorgecarleitao) - Improved
MutablePrimitiveArrayandMutableUtf8Array#299 (jorgecarleitao) - Improved
MutableBooleanArray#297 (jorgecarleitao) - Improved performance of concatenating non-aligned validities (15x) #291 (jorgecarleitao)
- Added support for timestamps with tz and interval to
io::print::write#287 (jorgecarleitao) - Improved debug repr of buffers and bitmaps. #284 (jorgecarleitao)
- Cleaned up internals of json integration #280 (jorgecarleitao)
- Removed
serde_derivedependency #279 (jorgecarleitao) - Simplified IPC code. #277 (jorgecarleitao)
- Reduced dependencies from confi-table and enabled
wasmonio_printfeature. #276 (jorgecarleitao) - Improve performance of
rem_scalar/div_scalarfor integer types (4x-10x) #275 (ritchie46)
Documentation updates:
- Cleaned examples and docs from old API. #330 (jorgecarleitao)
- Improved documentation #306 (jorgecarleitao)
Testing updates:
- Improved naming of testing workflows #315 (jorgecarleitao)
- Added tests to scalar API #300 (jorgecarleitao)
- Made CSV and JSON tests not use files. #290 (jorgecarleitao)
- Moved tests to integration tests #289 (jorgecarleitao)
Closed issues:
- Make parquet_read_record support async #331
- Panic due to SIMD comparison #312
- Bitmap::mutable line 155 may Panic/segfault #309
- IPC's
StreamReadermay abort due to excessive memory by overflowing ausized variable #301 - Improve performance of
rem_scalar/div_scalarfor integer types (4x-10x) #259
v0.3.0 (2021-08-11)
Breaking changes:
- Renamed
sumtosum_primitive#273 - Moved trait
Indexfromarray::Indextotypes::Index#272 - Added optional
projectionto IPC FileReader #271 - Added optional
page_filterto parquet'sRecordReaderandget_page_iterator#270 - Renamed parquets'
CompressionCodectoCompression#269
New features:
- Added support for FFI of dictionary-encoded arrays #267 (jorgecarleitao)
- Added support for projection pushdown on IPC files #264 (jorgecarleitao)
- Added support to read parquet asynchronously #260 (jorgecarleitao)
- Added support to filter parquet pages. #256 (jorgecarleitao)
- Added wrapping_cast to cast kernels #254 (sundy-li)
- Added support to parquet IO on wasm32 #239 (jorgecarleitao)
- Added support to round-trip dictionary arrays on parquet #232 (jorgecarleitao)
- Added Scalar API #56 (jorgecarleitao)
Fixed bugs:
- Fixed error in computing remainder of chunk iterator #262 (jorgecarleitao)
- Fixed error in slicing bitmap. #250 (jorgecarleitao)
Enhancements:
- Improve the performance in cast kernel using AsPrimitive trait in generic dispatch #252
- Poor performance in
sort::sort_to_indiceswith limit option in arrow2 #245 - Support loading Feather v2 (IPC) files with more than 1 million tables #231
- Migrated to parquet2 v0.3 #265 (jorgecarleitao)
- Added more tests to cast and min/max #253 (jorgecarleitao)
- Prettytable is unmaintained. Change to comfy-table #251 (PsiACE)
- Added IndexRange to remove checks in hot loops #247 (jorgecarleitao)
- Make merge_sort_slices MergeSortSlices public #243 (sundy-li)
Documentation updates:
- Added example and guide section on compute #242 (jorgecarleitao)
Closed issues:
- Allow projection pushdown to IPC files #261
- Add support to write dictionary-encoded pages #211
- Make IpcWriteOptions easier to find. #120
v0.2.0 (2021-07-30)
Breaking changes:
- Simplified
newsignature of growable API #238 (jorgecarleitao) - Add support to merge sort with a limit #222 (sundy-li)
- Generalized sort to accept indices other than i32. #220 (jorgecarleitao)
- Added support for limited sort #218 (jorgecarleitao)
New features:
- Merge sort support limit option #221
- Introduce limit option to sort #215
- Added support for take of interval of days_ms #219 (jorgecarleitao)
- Added FFI for remaining types #213 (jorgecarleitao)
Fixed bugs:
- Filter operation on sliced utf8 arrays are incorrect #233
- Fixed error in slicing bitmap. #237 (jorgecarleitao)
- Fixed nested FFI. #212 (jorgecarleitao)
Enhancements:
- Avoid materialization of indices in filter_record_batch for single arrays #234
- Add integration tests for writing to parquet #80
- Short-circuited boolean evaluation in GrowableList #228 (ritchie46)
- Add extra inlining to speed up take #226 (Dandandan)
- Removed un-needed
unsafe#225 (jorgecarleitao)
Documentation updates:
- Add documentation to guide #96
- Add git submodule command to correct the test doc #223 (sundy-li)
- Added badges to README #216 (sundy-li)
- Clarified differences with arrow crate #210 (alamb)
- Clarified differences with arrow crate #209 (alamb)
* This Changelog was automatically generated by github_changelog_generator