You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: meetings/2019/WASI-11-21.md
+22Lines changed: 22 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -39,3 +39,25 @@ Installation is required, see the calendar invite.
39
39
1. Closure
40
40
41
41
## Meeting Notes
42
+
43
+
Attendees:
44
+
45
+
Pat Hickey
46
+
Jan Falkin
47
+
Alex crichton
48
+
Jay Phelps
49
+
Andrew brown
50
+
Luke Wagner
51
+
Alon Zakai
52
+
Dan Gohman
53
+
Mark McCaskey
54
+
55
+
Meeting notes:
56
+
57
+
DG: WASI Snapshot 1 is out.
58
+
59
+
DG: WASI-SDK and WASI-libc. Currently in Cranestation orginization. They will move under the WebAssembly orginization, that was approved at the last CG meeting. We just have to work out some minor licensing issues for that to move forward.
60
+
61
+
DG: Modularization update. I don’t have an update, haven’t had enough time. The current PR shows the shape of it, but it needs to be updated to the current set of APIs and witx.
62
+
63
+
DG: Call for volunteers. I want to emphasize that WASI is a big opportunity, and now is the time to make changes. Please make PRs against the ephemeral directory
DG: If people have comments on the process we’re following with these meetings, please give feedback.
65
+
66
+
PH: I’ll second the agenda
67
+
68
+
DG: First item, stack trace API. Thomas Lively proposed this, but he’s not in the meeting at this time. Let’s table this until he’s here.
69
+
70
+
DG: Character encodings. This issue has been open for a while and there’s no easy answer. There’s a shape of a proposal in the issue that most engines are following - we say that everything is UTF-8. Is there any disagreement with that consensus here?
71
+
72
+
(none)
73
+
74
+
DG: For things like environment variables, it would simplify things a lot if we could guarantee those are UTF-8 strings.
75
+
76
+
DG: What about filesystems with file names that are unencodable? Can we find a way to tunnel that through? We currently use pointer+length to pass all filenames. No filesystem permits null in a filename, so we can encode a non-UTF8 string by putting extra information after the null. The trouble comes when you have to manipulate the path.
77
+
78
+
PH: What about interface types interaction?
79
+
80
+
DG: The stuff after the null will still be valid UTF-8 which encodes the non-UTF-8 part of the filename. So any library code that expects UTF-8 will still work.
81
+
DG: We’ll take this as consensus, and go forward with this design for now. We can change it if problems happen or we have objections during implementations.
82
+
83
+
AB: Can you clarify what is being encoded after the null?
84
+
85
+
DG: Two things: in the part before the null we want a normal-looking string, so any bytes that cant be translated get the unencodable-character (uffd). Then, after the null, we put the information that was lost. Something like, index of the string, then the missing bytes. We want something that is reversible. I agree it is goofy. Does anyone know how to encode an arbitrary integer in a utf-8 byte pattern? Obviously we could invent something, like using ascii digits. If theres an existing way to encode it, we’d want to follow that.
86
+
87
+
AB: So we can’t just say we support all unicode strings?
88
+
89
+
DG: The problem is that there are names out there that can’t be expressed in unicode. And from people I’ve talked to, its rare in practice. So the goal here is that if you get a path from one wasi api, and treat it as a utf-8 string without manipulating it, and then pass it to another wasi api, it works exactly as is.
90
+
91
+
AB: I misstated my question - is there no unicode format where this reconstruction wouldn't be necessary? Or will there always be some way for there to be an unencodable string?
92
+
93
+
DG: My understanding is there’s no way out of this.
94
+
95
+
Peter H: This would be for bytes which are not valid unicode codepoints.
96
+
97
+
DG: I’m looking for enough agreement that we can keep designing this for now.
98
+
99
+
Peter H: is there currently a part of the spec that allows non-utf8?
100
+
101
+
DG: Witx String type is always UTF-8. We don’t have a byte vector type right now, but there are byte pointer-length pairs. For any string passed to WASI we’ll require it to be UTF-8, and any string coming from WASI would be guaranteed to be UTF-8.
102
+
103
+
DG: Next topic, case sensitivity in filesystems. If you look at issue 72, the big picture strategy is that WASI doesn’t dictate case sensitivity or insensitivity, or what variation of sensitivity it has (because different filesystems have different rules, whether they case map ascii or different versions of unicode). I think the point of the API is to interact with existing filesystems, and we can’t control those, so this is going to be a portability hazard for WASI. We can make tools that help you detect these pitfalls. Does anyone have ideas of how we can make this better?
104
+
105
+
DG: Ok, so to use the wasm scary word, it is nondeterministic to use different cases to access the same file in the WASI api.
106
+
DG: Alright, so the next two issues are about virtual terminal handling. Its a space largely defined by compatibility with existing systems like xterm, unix consoles, and windows consoles. It makes sense to have something simple here. You could imagine unifying terminals with GUIs, modernizing terminals, lots of blue sky stuff here. My sense is that the value here is to find a simple thing that will interoperate with lots of different existing systems.
107
+
108
+
Sbc: Not just to interoperate with existing systems but to allow existing programs to run on many different systems?
109
+
110
+
DG: Yes my goal is something like vi on wasi just works. It looks like lots of people don’t use terminfo to figure out colors and just hard-code them. And windows support is tricky because it doesn’t have exactly the same features as unix. In cases where the terminals differ, should we expose WASI programs to the differences between them?
111
+
112
+
DG: The direction I’m going is we don’t try to do terminfo, the WASI implementation will be responsible for remapping escape sequences and all that. You could imagine having an environment variable TERM=WASI. It would look a lot like xterm, which in turn looks a lot like windows.
113
+
114
+
DG: On that topic, input sequences are a subfield of escape sequences. Input varies way more than output, across platforms. This made me think we could do a cool radical thing like use unicode symbols. But maybe the simplest thing is, just do what xterm does, and if it needs to be remapped by a WASI implementation to fit another console, thats up to them. But there are interesting questions to still ask here like what set of input sequences to support
115
+
116
+
DG: Its tempting to say, we can go and design a modern system that doesn’t have all the legacy baggage. But my instinct now is to pull back from that and pick a reasonable set of defaults based on some of the successful modern implementations. If someone wants to design a whole new system, I won’t stand in their way.
117
+
118
+
SJ: Can you clarify about how vim would work on this?
119
+
120
+
DG: Vim itself wouldnt have to change. Terminfo is a library that gets linked in, but hopefully we could provide a radically simplified version that works on wasi. Inside the wasi implementation, anything like a TTY would have to filter the output coming from WASI program to neutralize all escape sequences, and then encode in terms of whatever its client expects.
121
+
122
+
SJ: I would have assumed that stdin/out/err just map to whatever the host of the WASI implementation expects
123
+
124
+
DG: We’re worried about security, because the terminal interface wasn’t designed to deal with untrusted code running on the terminal. So one example is that you could run a program and you think that it exited, but really its just showing you something that looks like your shell. We want to not allow applications to do things like that, by default. So if you wanted to run vim, you’d have to give it permission to rewrite your entire screen, versus just print ordinary text to stdout, which would be the default. So, controlling cursor position and colors and that would be exposed to wasi as a capability.
Close this PR for now, citing the fact that the spec mechanisms we need to do this aren’t well defined yet, so we’ll focus on the simple thing for now.
75
+
76
+
## POSIX compatibility
77
+
DG: people are surprised that WASI does not align with POSIX. We see justified reasons for this, but it means that applications will have to port to WASI and can’t just recompile for a different target. For compatibility we can have an emulation layer, but for more performance usage to the native syscalls will be beneficial.
78
+
Sbc: agree that we need a compatibility layer for easier transition to WASI
79
+
DG: once we have a better solution for accessing files, a lot of concerns will go away
80
+
81
+
82
+
## What should poll for no events do? https://github.com/WebAssembly/WASI/pull/193
83
+
DG: As WASM doesn’t have signal handlers and might never have, does it make sense to poll indefinitely with no subscribers?
84
+
PH: Personally I think it should return `EINVAL` see (https://github.com/WebAssembly/WASI/pull/193#issuecomment-572737431)
85
+
AB: what happens to existing programs if we change this?
86
+
PH: if we start returning EINVAL from this, probably no one’s checking for it
87
+
DG: we could expose the change through a link failure with helpful documentation references. We could also put the feature back in once we have signals
88
+
89
+
## Framebuffer API https://github.com/WebAssembly/WASI/issues/174
90
+
AT:
91
+
Sbc: Is this using an mmap’ed file?
92
+
DG: I don’t know the API, but could we mmap the file into the runtime
93
+
Sbc: it probably needs an extra copy for copying between WASI and host memory
94
+
AT: Is this a good idea or do we need WebGL, and only that?
95
+
DG: we don’t have to wait for use cases which require WebGL, and we can implement the simple thing (framebuffer) now
96
+
AT: are there documents on how to get started with new syscalls?
97
+
DG: A C header file would be enough to get started.
98
+
AT: Cool, so I’ll get started on that
99
+
DG: are we going to have an audio API too?
100
+
AT: probably
101
+
DG: we’ll have to be careful about synchronizing the data flow
102
+
Sbc: Will consumers call these functions directly as opposed to using a libc wrapper
103
+
DG/Sbc: (...) probably?
104
+
DG: It’ll be great to have this so other API functions can follow this trail, like bulk copy, clearing the screen. So we can figure out questions like if we should pull those APIs in the WASI SDK, etc.
105
+
106
+
## Presence on the upcoming in-person meeting?
107
+
Sbc to file an issue.
108
+
109
+
110
+
111
+
112
+
PE: Will be presenting on an embedder api for wasi at the cg meeting.
113
+
Sbc: maybe we should open an issue to collect items and see if it justifies an in-person meeting
114
+
115
+
## Status on networking syscalls
116
+
DG: fastly and DG had an in-person meeting to discuss HTTP proxy API. Lots of pressure on network API due to asynchronous requirements, which depends on mechanisms which we don’t have (standardized) yet, e.g. coroutines, expressing synchronicity in interface types, etc.
117
+
PH: where’s this work tracked?
118
+
DG: there aren’t specific issues
119
+
SBC: Isn’t libevent which is synchronous the defacto standard?
120
+
DG: async depends on which level in the stack you’re looking at. Something which looks async to the application using callbacks, might be using a poll loop a layer below.
121
+
DG: we could have a single event loop for various events which is great for API builders, but not for application builders which might want their own event loop
122
+
SJ: what next steps are required to progress here?
123
+
DG: figuring out async, and then we need a proposal. We want to avoid a system which has either all networking or no networking. Ideally we’d like to have handles. Please reach out to me so we can discuss ideas.
Smaller binary size, optimization of combining libraries with Wasm binary (for example libc)
67
+
Frame-buffer API:
68
+
https://github.com/WebAssembly/WASI/pull/229
69
+
The process for designing and submitting new API proposals is still being figured out, so in addition to the proposal, let's also talk about what we can do to make the process easier.
70
+
Aaron: Idea: add a short and sweet markdown document laying out the steps for a proposal
71
+
Aaron: I wouldn’t mind writing this up. It’s already pretty much already in my PR. So it’s just cleaning that up and we would be good to go :)
72
+
Josh Triplett’s talk at Wasm SF was helpful, but a bit outdated even though it was only given a few months ago.
73
+
74
+
75
+
76
+
Crypto API!
77
+
https://github.com/WebAssembly/WASI/pull/231
78
+
What criteria should we apply?
79
+
We can also talk about process.
80
+
Being developed to support running wasm within a trusted execution environment, but intending this crypto API to have broader use cases.
81
+
Enarx: https://github.com/enarx/enarx/wiki
82
+
Prerequisite for sockets support
83
+
Lots of interest in sockets support.
84
+
Sockets dependency on merging read/write with send/recv?
85
+
Don’t over-rotate on POSIX.
86
+
How much appetite should we have for prototyping?
87
+
Feature flags are a great way to support this
88
+
Declare ahead of time what the process is.
89
+
Maybe it’s ok to use alternate names, as long as we document what’s going to happen. And maybe use polyfills to ease transitions.
90
+
Action item: Dan to write up a brief README for how to prototype a new feature. Also add things to watch out for.
91
+
Separate experimental repo? Directory? Maybe just put features in ephemeral/snapshot
92
+
Back on read/write - we don’t want to over-rotate on POSIX, but we do want a general read/write? We are heavily influenced by POSIX here.
0 commit comments