Skip to content

Conversation

@Eyizoha
Copy link
Contributor

@Eyizoha Eyizoha commented Jan 26, 2026

Purpose

Currently, most APIs require specifying the file system through the Options::FILE_SYSTEM configuration. Even when using a single file system (which should be the primary use case, as cross-file-system tables are quite rare), the implementation still uses ResolvingFileSystem as a forwarding layer. This introduces overhead on every file system access: parsing the path to extract the scheme, acquiring a read lock to obtain the actual underlying file system, and forwarding operations. Creating file systems via Options and forwarding through ResolvingFileSystem introduces unnecessary file system instances and computational overhead.

This change adds methods or parameters to directly specify file system instances for most APIs that require file system operations. This allows external systems to reuse existing file system instances and bypass the forwarding layer, addressing the issues mentioned above.

Tests

Modified existing unit tests to cover the API changes

API and Format

Yes - API Enhancement (Backward Compatible)
Changes to APIs in include/:

  • Added WithFileSystem() methods to builder classes
  • Added file_system parameters (with nullptr default values) to function signatures
  • When file system is not explicitly provided, behavior is unchanged from before

Documentation

None

Copy link
Collaborator

@lxy-9602 lxy-9602 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@lxy-9602 lxy-9602 merged commit 9f513f1 into alibaba:main Jan 27, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants