feat: implement `write` and `write_via_stream` by Sl1mb0 · Pull Request #354 · influxdata/datafusion-udf-wasm

Sl1mb0 · 2026-02-03T22:15:52Z

Describe your proposed changes here.

I've read the contributing section of the project CONTRIBUTING.md.
Signed CLA (if not already signed).

host/src/vfs/mod.rs

crepererum · 2026-02-06T11:46:55Z

host/src/vfs/path.rs

    /// - the segment MUST NOT contain a slash `/` character
    /// - the segment MUST NOT be empty
-    fn new(s: &str, limit: &VfsLimits) -> Result<Self, LimitExceeded> {
+    pub(crate) fn new(s: &str, limit: &VfsLimits) -> Result<Self, LimitExceeded> {


If I read the changes correctly, you only need this method exposed for tests. I wonder if we could do one of these two things instead:

just call PathTraversal::parse(s, limit).unwrap().into_iter().next().unwrap() in this one place in the test

leave the new function private and and a new function just for tests, like:
#[cfg(test)] pub(crate) fn new_test(s) -> Self { Self::new(s, &Default::default()).unwrap() }

My reasoning is that it would be nice if we keep the API exposure tight since it usually simplifies reasoning about the code and allows easier refactorings.

crepererum · 2026-02-06T11:51:15Z

host/src/vfs/mod.rs

+        if self.closed {
+            return Err(StreamError::Closed);
+        }
+        Ok(usize::MAX)


I had to read up what this method actually does:

Returns the number of bytes that are ready to be written to this stream.

Maybe leave a code comment here stating your reasoning:

Suggested change

Ok(usize::MAX)

// Since we don't have a buffer or I/O, we can basically accept an arbitrary number of bytes.

Ok(usize::MAX)

crepererum · 2026-02-06T11:52:52Z

host/src/vfs/mod.rs

+    /// Current write offset.
+    offset: u64,


I wonder if you have an easier time if this is a usize since you need a good amount of back-and-forth conversions in fn write, but maybe that wouldn't help. Just a thing to consider.

crepererum · 2026-02-06T11:55:17Z

host/src/vfs/mod.rs

+                let old_size = content.len();
+                if end > old_size {
+                    if let Err(e) = self.limiter.grow(end - old_size) {
+                        println!("let guest handle space error: {:?}", e);


return error?!

crepererum · 2026-02-06T11:56:13Z

host/src/vfs/mod.rs

+            .ok_or_else(|| StreamError::trap("offset overflow"))?;
+        let end = usize::try_from(end_u64).map_err(|_| StreamError::trap("offset overflow"))?;
+
+        match &mut self.node.write().unwrap().kind {


This code looks very similar to the block in async fn write below. I wonder if that could be moved to a common, shared function or method on VfsNodeKind?

crepererum · 2026-02-06T11:59:18Z

host/src/vfs/mod.rs

+                        .grow(std::mem::size_of_val(&new_file))
+                        .map_err(|_| FsError::trap(ErrorCode::InsufficientMemory))?;
+
+                    match &mut parent_node.write().unwrap().kind {


That's something I've missed when reviewing create_directory_at: You should check the VFS node kind first and in the match branch bump the allocator/limiter data. Because if someone will call this method on the wrong VFS node kind, you're basically "leaking" resources (I mean not really, but the guest cannot get them back).

crepererum · 2026-02-06T12:02:30Z

host/src/vfs/mod.rs

+                let mut guard = desc.node.write().unwrap();
+                match &mut guard.kind {
+                    VfsNodeKind::File { content } => {
+                        content.clear();


So truncate doesn't free any memory? That's a bit unexpected. I would probably call *content = Vec::with_capacity(0); here and then use the limiter to return the resources.

crepererum · 2026-02-06T12:10:39Z

host/src/vfs/mod.rs

+                }
+            }
+        } else {
+            let desc = maybe_desc?;


I was wondering what want_exclusive would do in this case, i.e. when want_create is not set.

Suggested change

let desc = maybe_desc?;

// NOTE: `wants_create` is `false`. We ignore `wants_exclusive` here.

//

// WASI just refers to POSIX:

// https://github.com/WebAssembly/WASI/blob/184b0c0e9fd437e5e5601d6e327a28feddbbd7f7/proposals/filesystem/wit/types.wit#L149-L150

//

// and POSIX just says it's "undefined":

// https://pubs.opengroup.org/onlinepubs/9799919799/functions/open.html

let desc = maybe_desc?;

crepererum · 2026-02-06T12:14:02Z

host/src/vfs/mod.rs

+        let wants_truncate = open_flags.contains(OpenFlags::TRUNCATE);
+
+        let node = if wants_create {
+            match maybe_desc {


Suggested change

match maybe_desc {

if want_directory {

// `wants_create` only works for files, NOT for directories, see:

// https://github.com/WebAssembly/WASI/blob/184b0c0e9fd437e5e5601d6e327a28feddbbd7f7/proposals/filesystem/wit/types.wit#L145-L146

return Err(...);

}

match maybe_desc {

Sl1mb0 force-pushed the tm/write! branch from 906a84e to 29f2ad8 Compare February 3, 2026 22:27

Sl1mb0 requested a review from crepererum February 4, 2026 00:57

Sl1mb0 force-pushed the tm/write! branch from e4c411d to ff327dc Compare February 5, 2026 13:30

Sl1mb0 force-pushed the tm/create-directory-at branch from e55628f to 465ffaf Compare February 5, 2026 13:45

Base automatically changed from tm/create-directory-at to main February 5, 2026 14:32

Sl1mb0 added 2 commits February 5, 2026 07:42

feat: implement write & write_via_stream

f3505c9

test: add some tests!

aa45929

Sl1mb0 force-pushed the tm/write! branch from ff327dc to aa45929 Compare February 5, 2026 15:42

refactor: handle write-like flags in open_at

3f18976

Sl1mb0 force-pushed the tm/write! branch from 4304a5a to 3f18976 Compare February 5, 2026 18:36

crepererum reviewed Feb 6, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: implement `write` and `write_via_stream`#354

feat: implement `write` and `write_via_stream`#354
Sl1mb0 wants to merge 3 commits intomainfrom
tm/write!

Sl1mb0 commented Feb 3, 2026 •

edited

Loading

Uh oh!

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

crepererum Feb 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	Ok(usize::MAX)
	// Since we don't have a buffer or I/O, we can basically accept an arbitrary number of bytes.
	Ok(usize::MAX)

-            let desc = maybe_desc?;
+            // NOTE: `wants_create` is `false`. We ignore `wants_exclusive` here.
+            //
+            //        WASI just refers to POSIX:
+            //        https://github.com/WebAssembly/WASI/blob/184b0c0e9fd437e5e5601d6e327a28feddbbd7f7/proposals/filesystem/wit/types.wit#L149-L150
+            //
+            //        and POSIX just says it's "undefined":
+            //        https://pubs.opengroup.org/onlinepubs/9799919799/functions/open.html
+            let desc = maybe_desc?;

-            match maybe_desc {
+            if want_directory {
+                // `wants_create` only works for files, NOT for directories, see:
+                // https://github.com/WebAssembly/WASI/blob/184b0c0e9fd437e5e5601d6e327a28feddbbd7f7/proposals/filesystem/wit/types.wit#L145-L146
+                return Err(...);
+            }
+            match maybe_desc {

Conversation

Sl1mb0 commented Feb 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Sl1mb0 commented Feb 3, 2026 •

edited

Loading