Type-safe wait functions

[dra27]

This is a more invasive alternative to #129 (which also "fixes" #117). I've pontificated about it on my blog.

With this idea, even in a thin wrapping around io_uring, an int in OCaml is only used to represent an actual (positive-or-zero, as it happens) int, not an encoding of something else. For the fast-path (i.e. where an error is not being returned) there is no difference in allocations, and unlike #129, no cost to retrieving the Unix.file_descr value from syscalls which return descriptors.

I didn't try plumbing it all the way back to Eio, but if moving the API in this general direction is an agreeable idea, I'd be quite happy to (it might finally tempt me to get my dirty Windows paws on eio backends as well...).

• Differentiate unit and int?
• Store real IDs in Heap, and simplify 'a job?

[avsm]

I'm a big plus for this! Having ocaml-uring return well-typed Unix.file_descrs is a perfect place to put the abstraction boundary, I feel. It is still useful to be able to convert this to a raw integer for other uses though (like passing it to programs that want an fd number)

[dra27]

Does that fit in with this related thought? Would it be the case that you must pass the fd to a program with its current number, or could you pick one (and have Unix.create_process or whatever handle the dup machinery)?

[avsm]

File descriptors are very much integers on *nix' they're just not on Windows! It's not exactly common, but programs do occasionally just need the fd integer for their own purposes, especially fd passing.

[dra27]

When’s the situation where you need to pass a specific fd as that fd, though? That was the point I was coming around to - if the file descriptor were abstract in C as well, when precisely would that hurt. Process invocation is one, but solvable with a map. Are there concrete cases where it actually can’t be done?

[dra27]

The distinction I was trying to draw is that obviously there are times in C where you need to get at the C representation of the file descriptor - but are there situations other than process spawning where you have to be able to get at it in OCaml?

[dra27]

File descriptors are very much integers on *nix' they're just not on Windows!

Because the hair-splitting is relevant here: OCaml file descriptors are not integers on Windows - the C ones very much are, the OS ones are pointers (where on Unix, the OS and C descriptors share the same thing)

[talex5]

Does that fit in with this related thought?
The fix is easy: pass a (int * file_descr) list. We preserve the abstraction, and specify the mapping.

Note: that is very similar to what Eio already does:

https://github.com/ocaml-multicore/eio/blob/7695d22387af7dc98214d66ad003dfad7b2a38f8/lib_eio/unix/process.mli#L27

[avsm]

@dra27 wrote:

Because the hair-splitting is relevant here: OCaml file descriptors are not integers on Windows - the C ones very much are, the OS ones are pointers (where on Unix, the OS and C descriptors share the same thing)

Ah I understand your point now. Right now we have:

• linux: (int * int:fd )
• windows: (int * struct:fd)

Where fd (Unix.file_descr) can be either an int or a struct.

However, this io_uring library will only ever be run on Linux, and so imposing an overhead of allocating a mapping tuple on Linux-only code seems unnecessary -- the two contents of the tuple will be identical in their representation! We could instead expose a type unix_fd = private int from this library, use that directly in Eio_linux (which will never see a Windows fd) and then turn that into a Unix.file_descr in the portable Eio layers.

[dra27]

Yes, indeed, a private int would be a good alternative - although where in Eio would one of these instances be? I've only had a cursory look at low_level.ml and sched.ml in lib_eio_linux, so I'm probably missing it, but I can see the parts where it converts an int from the result back to a file_descr (via Eio's own fds) but I hadn't yet found the reverse (but I should probably just try pushing this through to see!)?

I've only just started looking at it, but given the remarkable similarities between liburing and ioringapi.h, I'm not sure this library necessarily has to stay Linux only! 🧐

[avsm]

I'm not sure where to plumb it into Eio (which might overuse Unix.file_descr in the Linux backend because it's convenient). But that shouldn't stop us from doing it right in this library, and putting the conversion into Eio (which could eventually have a bigger refactor to make it more amenable to the backend-specific fd tracking)

[avsm]

@dra27 wrote:

I've only just started looking at it, but given the remarkable similarities between liburing and ioringapi.h, I'm not sure this library necessarily has to stay Linux only!

I think they are very similar on the surface (https://windows-internals.com/ioring-vs-io_uring-a-comparison-of-windows-and-linux-implementations/ is helpful), but there are lots of differences as the more advanced pieces of kernel plumbing get involved (like fixed buffer handling). I'd probably create a separate Windows only ioring library, and then do the integration in Eio in the first instance.