feat: implement & test `open_at` by Sl1mb0 · Pull Request #363 · influxdata/datafusion-udf-wasm

Sl1mb0 · 2026-02-09T16:38:51Z

Closes #336

I followed the specification described here. Given that we will eventually need to test this against the wasi test-suite, any differences between the aforementioned spec and the test-suite will favor the test-suite; so significant changes may need to be made in the future.

Describe your proposed changes here.

I've read the contributing section of the project CONTRIBUTING.md.
Signed CLA (if not already signed).

host/src/vfs/mod.rs

Copilot

Pull request overview

Implements POSIX/WASI-like open_at behavior in the in-memory VFS so guests can open existing nodes and create/truncate files (a prerequisite for enabling host-side FS writes), and updates unit/integration tests to reflect the new semantics.

Changes:

Refactors VFS path resolution and expands VfsCtxView::open_at to handle CREATE, EXCLUSIVE, DIRECTORY, and TRUNCATE.
Adds extensive unit tests around open_at behavior and updates integration test expectations/snapshots.
Makes Limiter::grow take &self (internally synchronized), simplifying ownership/mutability in component setup.

Reviewed changes

Copilot reviewed 4 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
host/src/vfs/mod.rs	Implements new `open_at` semantics, factors path traversal helper, and adds unit tests.
host/src/limiter.rs	Changes `grow` to `&self` to allow shared use behind a mutex.
host/src/component.rs	Adjusts limiter initialization to match the new `Limiter::grow` signature/usage.
host/tests/integration_tests/python/runtime/fs.rs	Updates Python FS integration test to exercise create-on-open behavior.
host/tests/integration_tests/evil/fs.rs	Updates large snapshot expectations for new VFS/open behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

host/src/vfs/mod.rs

host/tests/integration_tests/python/runtime/fs.rs

Sl1mb0 · 2026-02-10T14:38:38Z

This is a massive PR but most of the diffs are in the tests; the actual implementation is fairly minimal.

host/src/vfs/mod.rs

host/tests/integration_tests/evil/fs.rs

…h traversal

…track_caller]

crepererum · 2026-02-16T10:07:30Z

host/src/vfs/mod.rs

+        let existing = self.get_node_from_start(&path, Arc::clone(&base_node));
+
+        let node = match (existing, create, directory, exclusive, truncate) {
+            (Ok(node), true, _, false, _) => node, // Per POSIX: "If the file exists, O_CREAT has no effect except as noted under O_EXCL below.


what about truncate? I think you may only wanna take this branch of it is false and use the other truncation path down if it is true

Suggested change

(Ok(node), true, _, false, _) => node, // Per POSIX: "If the file exists, O_CREAT has no effect except as noted under O_EXCL below.

(Ok(node), true, _, false, false) => node, // Per POSIX: "If the file exists, O_CREAT has no effect except as noted under O_EXCL below.

crepererum · 2026-02-16T10:16:55Z

host/tests/integration_tests/evil/fs.rs

-    |             | /tmp        | ERR: Read-only file system (os error 69)        |
-    |             | /usr        | ERR: Read-only file system (os error 69)        |
-    |             | /var        | ERR: Read-only file system (os error 69)        |
+    |             |             | ERR: Bad file descriptor (os error 8)           |


why do we have bad file descriptor messages out of the sudden? IMHO this should just fail with not found because the source file doesn't exist.

crepererum · 2026-02-16T10:18:08Z

host/tests/integration_tests/evil/fs.rs

-    | /bin        | /var        | ERR: No such file or directory (os error 44)    |
-    | /bin        | \0          | ERR: No such file or directory (os error 44)    |
+    | /           | /x/..       | ERR: No such file or directory (os error 44)    |
+    | /bin        |             | OK: 0                                           |


/bin doesn't exist, so why can you copy it?

crepererum · 2026-02-16T10:19:36Z

host/tests/integration_tests/evil/fs.rs

One thing just occurred to me: the tests kinda assume that the file system is immutable, i.e. that one copy call or open call for example doesn't pollute the FS for the next test. This isn't really true anymore. Hence I think we gonna need to rewire this a bit. I can do that (PR incoming).

Fixed in #369.

crepererum · 2026-02-16T10:20:12Z

host/tests/integration_tests/evil/fs.rs

-    | .           | ERR: Invalid argument (os error 28)             |
-    | ..          | ERR: Invalid argument (os error 28)             |
-    | /           | ERR: Invalid argument (os error 28)             |
+    |             | OK: created                                     |


I don't think an empty string is a valid path. Hence this should fail.

crepererum · 2026-02-16T10:20:37Z

host/tests/integration_tests/evil/fs.rs

+    |             | OK: opened                                      |
+    | .           | OK: opened                                      |
+    | ..          | OK: opened                                      |
+    | /           | OK: opened                                      |


Can you open a directory with an O_APPEND flag in POSIX?

crepererum · 2026-02-16T10:21:07Z

host/tests/integration_tests/evil/fs.rs

-    | /tmp        | ERR: Read-only file system (os error 69)        |
-    | /usr        | ERR: Read-only file system (os error 69)        |
-    | /var        | ERR: Read-only file system (os error 69)        |
+    |             | OK: opened                                      |


Similar question: is that flag valid for directories?

crepererum · 2026-02-16T10:21:45Z

host/tests/integration_tests/evil/fs.rs

-    | /tmp        | ERR: Read-only file system (os error 69)        |
-    | /usr        | ERR: Read-only file system (os error 69)        |
-    | /var        | ERR: Read-only file system (os error 69)        |
+    |             | OK: opened                                      |


Is O_WRITE valid for directories under POSIX?

refactor: require immutable self reference for Limiter::grow

c98f54c

Sl1mb0 force-pushed the tm/open-at branch 12 times, most recently from a5a3acf to 967f281 Compare February 10, 2026 02:07

Sl1mb0 commented Feb 10, 2026

View reviewed changes

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

Sl1mb0 requested review from Copilot and crepererum February 10, 2026 02:15

Copilot started reviewing on behalf of Sl1mb0 February 10, 2026 02:15 View session

Sl1mb0 force-pushed the tm/open-at branch from 967f281 to c389df8 Compare February 10, 2026 02:19

Copilot AI reviewed Feb 10, 2026

View reviewed changes

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

host/src/vfs/mod.rs Outdated Show resolved Hide resolved

host/src/vfs/mod.rs Show resolved Hide resolved

host/tests/integration_tests/python/runtime/fs.rs Show resolved Hide resolved

Sl1mb0 force-pushed the tm/open-at branch 9 times, most recently from d7bd5c4 to adb81f9 Compare February 10, 2026 04:57

feat: implement & test open_at

23d3040

Sl1mb0 force-pushed the tm/open-at branch from adb81f9 to 23d3040 Compare February 10, 2026 13:36

Sl1mb0 force-pushed the tm/open-at branch 3 times, most recently from c06aaab to eca54f2 Compare February 10, 2026 14:18

refactor: create vfs_test! macro to reduce boilerplate in tests

bc9b47a

Sl1mb0 force-pushed the tm/open-at branch from eca54f2 to bc9b47a Compare February 10, 2026 14:22

Sl1mb0 requested a review from mhilton February 10, 2026 14:52

mhilton approved these changes Feb 10, 2026

View reviewed changes

crepererum reviewed Feb 11, 2026

View reviewed changes

Sl1mb0 added 2 commits February 11, 2026 13:26

refactor: return current or parent directory depending on Stay/Up pat…

c39e69a

…h traversal

fix: tests

58a5de5

Sl1mb0 force-pushed the tm/open-at branch 7 times, most recently from 162426f to 2a96695 Compare February 12, 2026 02:34

refactor: use match to clarify code paths

1f00b30

Sl1mb0 force-pushed the tm/open-at branch from 2a96695 to 1f00b30 Compare February 12, 2026 12:29

Sl1mb0 added 2 commits February 13, 2026 06:20

Merge branch 'main' into tm/open-at

f48c596

fix: test_copy

c4faf2b

Sl1mb0 force-pushed the tm/open-at branch from fd02c5a to f8d8073 Compare February 15, 2026 13:28

refactor: remove boiler-plate macros in favor of a single type and #[…

f0afb3c

…track_caller]

Sl1mb0 force-pushed the tm/open-at branch from f8d8073 to f0afb3c Compare February 15, 2026 13:44

crepererum reviewed Feb 16, 2026

View reviewed changes

crepererum mentioned this pull request Feb 16, 2026

test: isolate "evil" file system tests #369

Open

	(Ok(node), true, _, false, _) => node, // Per POSIX: "If the file exists, O_CREAT has no effect except as noted under O_EXCL below.
	(Ok(node), true, _, false, false) => node, // Per POSIX: "If the file exists, O_CREAT has no effect except as noted under O_EXCL below.

Conversation

Sl1mb0 commented Feb 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Sl1mb0 commented Feb 10, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Sl1mb0 commented Feb 9, 2026 •

edited

Loading