June 22, 2025: This Week in OpenDAL

[github-actions[bot]]

This weekly update is generated by LLMs. You're welcome to join our Discord for in-depth discussions.

Overall Activity Summary

The Apache OpenDAL repository has been bustling with activity this week, with contributions spanning language bindings, refactoring efforts, bug fixes, and feature enhancements. A total of 40+ pull requests, issues, and discussions were actively worked on, showcasing the community's dedication to improving OpenDAL. Key themes included expanding presign options, enhancing caching layers, and migrating services to new architectural conventions.

Key Ongoing Projects

• Node.js Bindings Migration to Options API:

  • @kingsword09 contributed feat(bindings/nodejs): Add StatOptions support for new options API #6282, adding StatOptions support, and feat(bindings/nodejs): Add ReadOptions and ReaderOptions support for new options API #6312, implementing ReadOptions and ReaderOptions.
  • Tracking issue Tracking issues for migrating Node.js bindings to RFC-6213 Options API #6281 outlines the migration roadmap for WriteOptions, DeleteOptions, and ListOptions.

• Service Refactoring to Use Access:

  • @Xuanwo led efforts to migrate services like Redis (refactor: Migrate redis from adapter::kv to Access instead #6291), Memory (refactor: Migrate memory service to implment Access directly #6301), and Moka (refactor: Migrate moka from adapter::typed_kv to Access instead #6300) from adapter-based implementations to Access for improved scalability and maintainability.

• Caching Layer Development:

  • @koushiro proposed a Cache Layer design in RFC-6297: Cache Layer #6297, laying the foundation for advanced caching capabilities in OpenDAL.

• CI and Testing Improvements:

  • @yuchanns streamlined Go bindings testing infrastructure in ci(bindings/go): simplify and improve Go bindings test infrastructure #6293, simplifying workflows and improving consistency.

• Presign Options Expansion:

  • Efforts to enhance presign functionality across services are ongoing, with tracking issue Tracking issues of presign options support across all backends #6274 highlighting gaps in support for S3, Azure, and B2.

Priority Items

• Critical Bug Fixes:

  • bug: HDFS write failure may cause infinite loop in HdfsWriter::write #6288: HDFS write failures may cause infinite loops. @oven-yang has submitted a fix.
  • bug: issue with http service #6289: HTTP service fails to read remote XLSX files correctly. Needs investigation.
  • bug: cos test failed for dir should be found in list, but only got #6278: COS async list tests are failing due to missing directory entries.

• Feature Enhancements:

  • Tracking issues of migrating adapter based services #5739: Migration of adapter-based services to Access. Several services remain pending.
  • Tracking issues of presign options support across all backends #6274: Address gaps in presign option support across all backends.

Notable Discussions

• Governance Vote:

  • [VOTE] Accept Reqsign as part of OpenDAL PMC #6261: Proposal to accept Reqsign as part of OpenDAL PMC. The vote passed successfully, led by @Xuanwo.

• Lister Enhancements:

  • new feature: suggestions for Lister #6272: @chitralverma suggested improvements to Lister and Entry APIs, including options for metadata retrieval and better naming conventions.

• File Size Metadata:

  • Is there a way to tell if the file size is 0 or if the service doesn't provide size? #6323: @rick-200 raised a question about distinguishing between zero file size and services that don't provide size metadata.

Emerging Trends

• Language Bindings Focus:

  • Significant contributions to Node.js, Python, and C++ bindings reflect the community's emphasis on cross-language support.

• Refactoring for Scalability:

  • Migration of services to Access and adoption of modular conventions like backend.rs and core.rs are improving code maintainability.

• Presign Functionality Expansion:

  • Contributors are actively working to ensure feature parity across services, enabling advanced presign options.

Good First Issues

• new feature: cpp binding enable feature by user #6250: Enable feature flags in C++ bindings. Ideal for contributors familiar with C++ and OpenDAL's architecture.
• new feature: publish the source and lock files for binaries (oli, oay...) #6198: Publish source and lock files for binaries like oli. A straightforward task for contributors interested in packaging and deployment.
• Tracking issue: Refactor services to use the backend.rs and core.rs convention #5702: Refactor services to adopt the backend.rs and core.rs convention. Suitable for contributors looking to improve code organization.
• Implement Write Returns Metadata for all services #5693: Implement "Write Returns Metadata" for services. Contributors can start with simpler services like fs or s3.
• Tracking issues of RFC-5479: Context #5480: Context implementation tracking. A good entry point for contributors interested in core OpenDAL functionality.