Datafusion file system access over WASI

[g3blv]

I'm building a WASM module in Rust using DataFusion that needs to read JSON files from a directory shared by the host via WASI. The host (using wasmtime) pre-opens a directory using WasiCtxBuilder, but DataFusion's ListingTableUrl fails to access it with error: No suitable object store found for file:///sources/.

Host code

let wasi_ctx = WasiCtxBuilder::new()
        .preopened_dir("./data", "/", DirPerms::all(), FilePerms::all())?
        .inherit_stdio()
        .build();

sources is located under data (data/sources)

let json_dir = "/sources";

let table_path = match ListingTableUrl::parse(format!("file://{}", json_dir)) {
        Ok(path) => path,
        Err(err) => {
            eprintln!("Error parsing listing table URL: {}", err);
            return Err(err)
                .with_context(|| format!("Failed to parse listing table URL: {}", json_dir));
        }
};

let file_format = JsonFormat::default();

let listing_options = ListingOptions::new(Arc::new(file_format)).with_file_extension(".json");

let resolved_schema = match listing_options
    .infer_schema(&session_state, &table_path)
    .await
{
    Ok(schema) => schema,
    Err(err) => {
        eprintln!("Error inferring schema: {}", err);
        return Err(err).with_context(|| "Failed to infer schema from JSON files");
    }
};

Running the code returns: Error inferring schema: Internal error: No suitable object store found for file:///sources/. See RuntimeEnv::register_object_store.

I can't use the object_store crate because it won't build with cargo component build which is need for wasm32-wasip1

I have the code working in "native" Rust.

Does DataFusion's file access work differently when running in WASM vs native Rust?
Is there a specific way to configure DataFusion's object store system for WASI environments?
Does the file path mapping between host and WASM environment need special handling?
Do I need to implement object_store for the wasifile access? If yes, how would I do that?

[g3blv]

I've read a bit more and my understanding is that an ObjectStore is needed and that there is an ObjectStore implementation for Local File System but it seems like Local File System is disabled when target arch is wasm32,

"Stores are registered based on the scheme, host and port of the provided URL with a LocalFileSystem::new automatically registered for file:// (if the target arch is not wasm32)."(https://docs.rs/datafusion/43.0.0/datafusion/datasource/object_store/trait.ObjectStoreRegistry.html#implementors)

Is my understanding correct? Is there some way I can work around this for my use case?