feat: Add Tencent Cloud COS (Cloud Object Storage) support

[XuQianJin-Stars]

Is your feature request related to a problem?

Currently, Daft supports multiple cloud storage backends including AWS S3, Azure Blob Storage, Google Cloud Storage, and Tencent TOS (火山引擎 TOS). However, there is no native support for Tencent Cloud COS (Cloud Object Storage / 腾讯云对象存储), which is one of the most widely used object storage services in China.

Users who want to use Daft with data stored in Tencent Cloud COS currently have to either:

1. Use the S3-compatible API endpoint (which may have limitations and requires manual endpoint configuration)
2. Copy data to a supported storage backend (inefficient and costly)

Describe the solution you'd like

Add native support for Tencent Cloud COS using OpenDAL as the underlying implementation, similar to how Lance implemented COS support.

Why OpenDAL?

• No official Rust SDK: Tencent Cloud COS does not have an official Rust SDK
• Proven implementation: OpenDAL has mature COS support with authentication, retry logic, and multipart upload
• Active community: OpenDAL is an Apache project with active maintenance
• Future extensibility: OpenDAL supports 70+ storage backends, making it easier to add more providers in the future

Implementation Architecture

Since Daft uses a custom ObjectSource trait (not the object_store crate), we need to create an adapter layer:

[codenohup]

Hi, @XuQianJin-Stars ,

I'm very interested in this issue and would like to help solve it.
Did you already started working on it? If you need assistance, I'd be happy to contribute.

Thanks!

[XuQianJin-Stars]

Hi, @XuQianJin-Stars ,

I'm very interested in this issue and would like to help solve it. Did you already started working on it? If you need assistance, I'd be happy to contribute.

Thanks!

I'm doing it.

[universalmind303]

@XuQianJin-Stars, Instead of using opendal under the hood to implement tencent cos, I actually think it'd be much more beneficial to the project if we could come up with a way to just support opendal. We've been getting a lot of feature requests for new object stores. If we could support opendal, then in theory, we get all of those object stores for free.

[XuQianJin-Stars]

@XuQianJin-Stars, Instead of using opendal under the hood to implement tencent cos, I actually think it'd be much more beneficial to the project if we could come up with a way to just support opendal. We've been getting a lot of feature requests for new object stores. If we could support opendal, then in theory, we get all of those object stores for free.

hi @universalmind303 This would be a significant change. My idea is to first introduce opendal by implementing the COS feature, and then gradually modify and refactor it. If the community feels that a refactoring via opendal is acceptable, I can do that.

[universalmind303]

@XuQianJin-Stars I created a PR here that implements a generic opendal backend. Let me know if this works an an alternative. Note that writes are not yet fully supported in this PR.