Support argument discovery

[paulmillar]

This is a somewhat speculative feature request.

My exploration of fsspec suggests that there's currently no standard, machine-actionable mechanism through which a backend implementation can describe the arguments (args, kwargs) that it understands.

The use-case I'm considering is for implementation-agnostic UI systems; for example the Jupyter fsspec extension. Currently, the user is expected to configure the jupyter extension by editing a configuration file, providing the necessary configuration (if any) for that fsspec backend implementation.

Seemingly, the user must discover information about how to configure the backend implementation by somehow finding and reading the backend implementation's documentation.

Fsspec already supports discovering the list of available backend implementations. If it were possible, for a specific fsspec backend implementation, to discover a list of possible configuration options, along with some metadata (input type, a brief description, whether it is required, etc) then the fsspec client (jupyter fsspec extension, in the above example) could build a UI that would allow the user to add fsspec implementations, ideally without forcing the user to discover documentation for the backend implementation.

[martindurant]

There are two principle sources of this information:

• the init signatures of the implementation classes, which may be typed, and their associated docs. This is not ideal, and standardising how to surface this information may indeed be helpful
• many arguments are passed through to other filesystem implementations (for wrappers like caching FSs) or to other libraries (e.g., client options passed by s3fs to botocore).