v4.5.0

Pr review (#3121)

* add claude pr review

* edits

* edits

v4.4.5

• Improved Deeplake support for NULL
• Fixed issue with simultaneous row and column creation

v4.4.4

• Deeplake API now has cmake and pkg-config files for better integration
• Improved storage access performance
• Revisited PG ingestion. Improved batch ingestion

v4.4.3

First C++ library release.

v4.4.1 🌈

Release Notes

🎉 New Features & Enhancements

Storage & API Improvements

• Storage list_dirs API - New API endpoint for listing directories by @khustup2
• Added mesh type support - Extended data type capabilities by @khustup2
• PLY visualization and local file listing - Enhanced node server with visualization tools by @azat-manukyan
• Simple visualizer - New visualization component for data inspection by @khustup2
• Credentials key property - Added property to expose creds key for read-only datasets by @activesoull

Data Management

• Efficient version-locked read-only view - Implemented optimized read-only dataset views by @ashotshakhkyan
• Library preloading - Improved performance with preloaded library support by @vahan-activeloop

⚡ Performance Improvements

• 30% LRU cache insertion runtime improvement - Significant speedup in cache operations by @ashotshakhkyan
• Cache clearing on dataset deletion - Minor performance optimization on cleanup by @ashotshakhkyan
• Refactored async_prefetcher - Simplified batch prefetch logic for better performance by @ashotshakhkyan

🐛 Bug Fixes

Core Functionality

• Fixed progress reporting of rebuild_branch - Corrected progress tracking during branch rebuilds by @khustup2
• Fixed deeplake behavior - Resolved critical deeplake module issues by @activesoull
• Fixed JS build - Resolved JavaScript compilation errors by @ashotshakhkyan

Data Handling

• Summary fix - Corrected summary generation logic by @activesoull
• MP3 header magic bytes - Fixed audio file format detection by @azat-manukyan

Indexing & Storage

• Corrected indexing mode of compactor - Fixed compactor indexing behavior by @ashotshakhkyan
• Duplicate index data handling - Resolved race condition in concurrent index builders by @ashotshakhkyan
• Fixed warning - Addressed compiler warnings by @activesoull

🏗️ Architecture & Refactoring

Code Organization

• Simplified rebuild_branch logic - Eliminated temporary branch creation for cleaner code by @khustup2
• Reduced public headers - Hidden implementation details from public API by @ashotshakhkyan
• Deeplog include cleanup - Organized include statements for better maintainability by @ashotshakhkyan

API Improvements

• Added merge commit message - Enhanced commit metadata by @activesoull

v4.3.4 🌈

🎉 New Features
pg_deeplake Enhancements

CHAR/BPCHAR data type support - pg_deeplake now supports CHAR and BPCHAR data types
Library preloading - Improved performance with preloaded libraries in pg_deeplake
Auto table reload - Tables are now automatically reloaded if not found in storage

Architecture Improvements

Query core module - Separated query engine from execution module for better modularity
Group by execution refactor - Separated group by execution logic for future enhancements

🐛 Bug Fixes

Fixed linked audio type data handling - Resolved issues with linked audio type data
Fixed compilation errors - Resolved compile errors in pg_deeplake

💬 User Experience

Better error messaging - Added clear error messages for missing credentials key

🧹 Code Quality

Cleanup and refactoring - Improved code organization with proper renames and cleanup

🧪 Testing

Enhanced test coverage - Added comprehensive tests for pg_deeplake

v3.9.52 🌈

🧭 What's Changed

• Allow numpy v2. (#3075) @khustup2

⚙️ Who Contributes

@activesoull and @khustup2

v4.4.0

Release Notes

New Features

Indexing Improvements

• Support for JSON data indexing - Added support for indexing JSON data types with proper handling and query optimization
• Generic column index specification - New ability to specify index type on generic columns for more granular control
• Progress bar for index creation - Improved user experience with visual progress indicator during index creation operations
• Index handling unification - Streamlined and unified index handling across the codebase for better consistency and maintainability

Documentation

• Added missing documentation sections - Expanded documentation coverage for previously undocumented features and APIs

Improvements

Performance & Reliability

• Fixed build errors on main branch - Resolved compilation issues affecting the main development branch
• Windows test fixes - Enhanced test reliability on Windows platform with targeted fixes
• Thread-safe cancellation - Improved cancellation mechanism with proper thread safety guarantees
• Rebuild branch internal API - Refactored internal API for better architecture and maintainability
• Fix rebuild indexes during compaction - Corrected index rebuilding behavior during database compaction operations

Platform & Build Support

• Windows Python build support - Added comprehensive support for building Python bindings on Windows
• Updated libtiff dependency - Upgraded libtiff library to latest version for improved compatibility and security

Bug Fixes

• Fixed browser crash - Resolved critical issue causing browser crashes in specific scenarios
• Invalid iterator fix - Fixed invalid iterator handling that could cause runtime errors
• General bug fixes - Various stability improvements and edge case handling

Contributors

Thanks to @activesoull, @vahan-activeloop, @khustup2, and @ashotshakhkyan for their contributions to this release.

v4.3.5 🌈

Data Type Support

Link to bytes support - Added support for link to bytes data type (DEE-299)
Flexible text column handling - Text columns now allow appending data with unknown dtype for better compatibility

🐛 Bug Fixes

deeplake.exists error handling - Fixed exception handling in deeplake.exists to gracefully handle errors instead of throwing exceptions (DEE-300)
Python PNG linking - Resolved Python PNG linking issues

📚 Documentation

Documentation updates - Improved and updated documentation (DEE-36)

🧹 Code Quality

General improvements - Various small fixes and code improvements

v4.3.4

🎉 New Features
PostgreSQL 18 Support

PostgreSQL 18 compatibility - pg_deeplake now supports PostgreSQL 18

Performance Optimizations

Adaptive backoff for worker threads - Implemented an intelligent backoff strategy for idle worker threads to reduce CPU usage
Cgroup-aware concurrency - Hardware concurrency now respects cgroup limits for better container performance

Azure Integration

Azure container name handling - Improved Azure getter with proper container name handling

🐛 Bug Fixes

Build errors resolved - Fixed compilation errors in pg_deeplake
Visualizer fixes - Resolved issues in the visualizer component

🧹 Code Quality

Code cleanup - Improved code organization with proper renames and refactoring
Modular architecture - Continued separation of concerns for better maintainability

🧪 Testing

Enhanced test coverage - Added comprehensive tests for pg_deeplake