-sNODERAWSOCKETS DNS resolution, with general readiness

ChangeLog.md

@@ -20,6 +20,17 @@ See docs/process.md for more on how version tagging works.
 
 6.0.2 (in development)
 ----------------------
+- Added support for `epoll` (`epoll_create1`/`epoll_ctl`/`epoll_wait`/
+  `epoll_pwait`) on the legacy (non-WASMFS) JS filesystem, including
+  level- and edge-triggered modes, `EPOLLONESHOT`, `EPOLLEXCLUSIVE`,
+  `EPOLLRDHUP`, nesting, and blocking waits under `PROXY_TO_PTHREAD`,
+  `ASYNCIFY`, and `JSPI`. Also added `emscripten_epoll_set_callback`
+  (in the new `<emscripten/epoll.h>`, experimental), a non-blocking variant
+  that delivers an epoll set's readiness to a JS callback with no
+  `ASYNCIFY`/`JSPI`. As part of this, the (undocumented) `stream_ops.poll`
+  FS-backend handler signature changed from `poll(stream, timeout)` to
+  `poll(stream)` returning the current readiness mask; out-of-tree custom FS
+  backends with a `poll` handler must update. (#27207)
 - New `-sNODERAWSOCKETS` setting that backs the POSIX sockets API with real TCP
   (`node:net`) and UDP (`node:dgram`) sockets on Node.js, with no `ws`, proxy
   process, or pthreads required. Supports incoming and outgoing TCP, UDP, IPv6,

site/source/docs/api_reference/emscripten.h.rst

@@ -1280,6 +1280,69 @@ Functions
   :param em_socket_callback callback: Pointer to a callback function. The callback returns a file descriptor and the arbitrary ``userData`` passed to this function.
 
 
+.. c:function:: int emscripten_dns_lookup_async(const char *node, const char *service, const struct addrinfo *hints)
+
+  Asynchronous :c:func:`getaddrinfo`. Takes the same ``node``/``service``/``hints``
+  inputs and returns a file descriptor that becomes readable once resolution
+  completes. Wait on it with ``poll``/``select``/``epoll`` or, without blocking, by
+  adding it to an epoll set and using :c:func:`emscripten_epoll_set_callback`, then read
+  the result with :c:func:`emscripten_dns_lookup_result`. The caller owns the fd
+  and should ``close()`` it.
+
+  With ``-sNODERAWSOCKETS`` a hostname is resolved asynchronously via ``node:dns``;
+  otherwise (and for numeric or ``/etc/hosts`` names) resolution is synchronous,
+  as :c:func:`getaddrinfo`, and the fd is simply readable on the next turn.
+
+  :param node: The hostname or numeric address to resolve.
+  :param service: The service name or port string (may be ``NULL``).
+  :param hints: ``addrinfo`` filter (``ai_family``/``ai_socktype``/etc.; may be ``NULL``).
+  :returns: A pollable file descriptor, or ``-1`` on failure to start the lookup.
+
+
+.. c:function:: int emscripten_dns_lookup_result(int fd, struct addrinfo **res)
+
+  Reads the outcome of a lookup started by :c:func:`emscripten_dns_lookup_async`,
+  once its ``fd`` is readable.
+
+  :param int fd: The file descriptor returned by :c:func:`emscripten_dns_lookup_async`.
+  :param res: On success, receives the head of the resulting ``addrinfo`` list (free it with :c:func:`freeaddrinfo`, as for :c:func:`getaddrinfo`).
+  :returns: ``0`` on success, or an ``EAI_*`` error code on failure (``EAI_AGAIN`` if the lookup has not completed yet).
+
+
+.. c:type:: em_epoll_callback
+
+  Function pointer type for the :c:func:`emscripten_epoll_set_callback` callback,
+  defined as: ::
+
+    typedef void (*em_epoll_callback)(int epfd, struct epoll_event *events, int nready, void *userdata);
+
+  ``events`` is a runtime-owned buffer of ``nready`` ready entries, valid only for
+  the duration of the call.
+
+
+.. c:function:: int emscripten_epoll_set_callback(int epfd, int maxevents, em_epoll_callback callback, void *userdata)
+
+  Register a persistent readiness callback on an existing epoll fd (built with
+  :c:func:`epoll_create1`/:c:func:`epoll_ctl`): instead of blocking in
+  :c:func:`epoll_wait`, the runtime calls ``callback`` every time the set makes
+  progress, delivering up to ``maxevents`` ready events. The interest is armed once
+  and reused across every delivery. Unlike :c:func:`epoll_wait` it never blocks the
+  calling stack, so it works without ASYNCIFY/JSPI, and it keeps the runtime alive
+  while armed.
+
+  There is at most one callback per epoll: calling again replaces it. A ``NULL``
+  ``callback`` unregisters; the interest also ends when the epoll fd is closed.
+  Per-fd ``EPOLLET``/``EPOLLONESHOT`` behave exactly as with the blocking
+  :c:func:`epoll_wait`.
+
+  :param epfd: An epoll fd from :c:func:`epoll_create1`.
+  :param maxevents: Maximum number of ready events to deliver per callback.
+  :param callback: Invoked with ``epfd``, the runtime-owned ``events`` buffer, the ready count ``nready``, and ``userdata``. ``NULL`` unregisters.
+  :param userdata: Opaque pointer passed through to the callback.
+  :returns: ``0`` on success, or ``-errno`` (``-EBADF``/``-EINVAL``).
+
+
+
 Unaligned types
 ===============
 

src/lib/libcore.js

@@ -947,53 +947,61 @@ addToLibrary({
     return inetPton4(DNS.lookup_name(nameString));
   },
 
-  getaddrinfo__deps: ['$DNS', '$inetPton4', '$inetNtop4', '$inetPton6', '$inetNtop6', '$writeSockaddr', 'malloc', 'htonl'],
-  getaddrinfo__proxy: 'sync',
-  getaddrinfo: (node, service, hint, out) => {
-    // Note getaddrinfo currently only returns a single addrinfo with ai_next defaulting to NULL. When NULL
-    // hints are specified or ai_family set to AF_UNSPEC or ai_socktype or ai_protocol set to 0 then we
-    // really should provide a linked list of suitable addrinfo values.
-    var addrs = [];
-    var canon = null;
-    var addr = 0;
-    var port = 0;
-    var flags = 0;
-    var family = {{{ cDefs.AF_UNSPEC }}};
-    var type = 0;
-    var proto = 0;
-    var ai, last;
-
-    function allocaddrinfo(family, type, proto, canon, addr, port) {
-      var sa, salen, ai;
-      var errno;
-
-      salen = family === {{{ cDefs.AF_INET6 }}} ?
+  // The encode/mint stage: turn a resolved descriptor ({entries, type, proto,
+  // port}, addr in parsed inetPton form) into an addrinfo linked list and return
+  // the head (0 for an empty list). This is the sole point that mints C memory,
+  // and the whole chain is freed uniformly by freeaddrinfo - one ownership rule.
+  // (A future ring/aio backend would add a sibling encoder here, e.g. one that
+  // writes into a caller buffer, without touching parse/resolve.)
+  $writeAddrInfoList__deps: ['$inetNtop4', '$inetNtop6', '$writeSockaddr', 'malloc'],
+  $writeAddrInfoList: (desc) => {
+    var head = 0, prev = 0;
+    for (var entry of desc.entries) {
+      var family = entry.family;
+      var salen = family === {{{ cDefs.AF_INET6 }}} ?
         {{{ C_STRUCTS.sockaddr_in6.__size__ }}} :
         {{{ C_STRUCTS.sockaddr_in.__size__ }}};
-      addr = family === {{{ cDefs.AF_INET6 }}} ?
-        inetNtop6(addr) :
-        inetNtop4(addr);
-      sa = _malloc(salen);
-      errno = writeSockaddr(sa, family, addr, port);
+      var sa = _malloc(salen);
+      var errno = writeSockaddr(sa, family, family === {{{ cDefs.AF_INET6 }}} ? inetNtop6(entry.addr) : inetNtop4(entry.addr), desc.port);
 #if ASSERTIONS
       assert(!errno);
 #endif
-
-      ai = _malloc({{{ C_STRUCTS.addrinfo.__size__ }}});
+      var ai = _malloc({{{ C_STRUCTS.addrinfo.__size__ }}});
       {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_family, 'family', 'i32') }}};
-      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_socktype, 'type', 'i32') }}};
-      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_protocol, 'proto', 'i32') }}};
-      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_canonname, 'canon', '*') }}};
+      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_socktype, 'desc.type', 'i32') }}};
+      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_protocol, 'desc.proto', 'i32') }}};
+      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_canonname, '0', '*') }}};
       {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_addr, 'sa', '*') }}};
-      if (family === {{{ cDefs.AF_INET6 }}}) {
-        {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_addrlen, C_STRUCTS.sockaddr_in6.__size__, 'i32') }}};
+      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_addrlen, 'salen', 'i32') }}};
+      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_next, '0', 'i32') }}};
+      if (prev) {
+        {{{ makeSetValue('prev', C_STRUCTS.addrinfo.ai_next, 'ai', '*') }}};
       } else {
-        {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_addrlen, C_STRUCTS.sockaddr_in.__size__, 'i32') }}};
+        head = ai;
       }
-      {{{ makeSetValue('ai', C_STRUCTS.addrinfo.ai_next, '0', 'i32') }}};
-
-      return ai;
+      prev = ai;
     }
+    return head;
+  },
+
+  // Shared getaddrinfo core. Allocates nothing: returns a resolved descriptor
+  // {entries, type, proto, port} (entries are {family, addr} with addr in parsed
+  // inetPton form), a negative EAI_* code on failure, or - under NODERAWSOCKETS,
+  // for a hostname needing DNS - a {node, family, type, proto, port} descriptor
+  // (no entries) for the caller to resolve. The result is minted from a
+  // descriptor by writeAddrInfoList at the point ownership passes to the caller.
+  $getAddrInfo__deps: ['$DNS', '$inetPton4', '$inetPton6', 'htonl', '$UTF8ToString',
+#if NODERAWSOCKETS
+    '$nodeSockHelpers',
+#endif
+  ],
+  $getAddrInfo: (node, service, hint) => {
+    var addr = 0;
+    var port = 0;
+    var flags = 0;
+    var family = {{{ cDefs.AF_UNSPEC }}};
+    var type = 0;
+    var proto = 0;
 
     if (hint) {
       flags = {{{ makeGetValue('hint', C_STRUCTS.addrinfo.ai_flags, 'i32') }}};
@@ -1063,9 +1071,7 @@ addToLibrary({
           addr = [0, 0, 0, _htonl(1)];
         }
       }
-      ai = allocaddrinfo(family, type, proto, null, addr, port);
-      {{{ makeSetValue('out', '0', 'ai', '*') }}};
-      return 0;
+      return { entries: [{ family, addr }], type, proto, port };
     }
 
     //
@@ -1096,9 +1102,7 @@ addToLibrary({
       }
     }
     if (addr != null) {
-      ai = allocaddrinfo(family, type, proto, node, addr, port);
-      {{{ makeSetValue('out', '0', 'ai', '*') }}};
-      return 0;
+      return { entries: [{ family, addr }], type, proto, port };
     }
     if (flags & {{{ cDefs.AI_NUMERICHOST }}}) {
       return {{{ cDefs.EAI_NONAME }}};
@@ -1107,6 +1111,22 @@ addToLibrary({
     //
     // try as a hostname
     //
+#if NODERAWSOCKETS
+    // /etc/hosts resolves synchronously (read fresh through emscripten's FS).
+    var hosts = nodeSockHelpers.readHosts(node).filter((e) =>
+      family === {{{ cDefs.AF_UNSPEC }}} || e.family === family);
+    if (hosts.length) {
+      var entries = hosts.map((e) => ({
+        family: e.family,
+        addr: e.family === {{{ cDefs.AF_INET6 }}} ? inetPton6(e.addr) : inetPton4(e.addr),
+      }));
+      return { entries, type, proto, port };
+    }
+    // A real hostname needs a DNS lookup; hand the request back to the caller to
+    // resolve asynchronously (getaddrinfo suspends under JSPI / returns
+    // EAI_AGAIN otherwise; emscripten_dns_lookup_async drives the poll-fd flow).
+    return { node, family, type, proto, port };
+#else
     // resolve the hostname to a temporary fake address
     node = DNS.lookup_name(node);
     addr = inetPton4(node);
@@ -1115,9 +1135,45 @@ addToLibrary({
     } else if (family === {{{ cDefs.AF_INET6 }}}) {
       addr = [0, 0, _htonl(0xffff), addr];
     }
-    ai = allocaddrinfo(family, type, proto, null, addr, port);
-    {{{ makeSetValue('out', '0', 'ai', '*') }}};
-    return 0;
+    return { entries: [{ family, addr }], type, proto, port };
+#endif
+  },
+
+  getaddrinfo__deps: ['$getAddrInfo', '$writeAddrInfoList',
+#if NODERAWSOCKETS && ASYNCIFY == 2
+    '$nodeSockHelpers',
+#endif
+  ],
+  getaddrinfo__proxy: 'sync',
+#if NODERAWSOCKETS && ASYNCIFY == 2
+  // Under JSPI a hostname miss suspends the wasm stack on the real node:dns
+  // lookup (returning a promise) rather than reporting EAI_AGAIN. A resolved
+  // descriptor (numeric/hosts) or error does not suspend.
+  getaddrinfo__async: true,
+#endif
+  getaddrinfo: (node, service, hint, out) => {
+    // parse -> (resolve) -> mint. One descriptor threads through all three.
+    var desc = getAddrInfo(node, service, hint);
+    if (typeof desc === 'object') {
+      if (desc.entries) {
+        {{{ makeSetValue('out', '0', 'writeAddrInfoList(desc)', '*') }}};
+        return 0;
+      }
+#if NODERAWSOCKETS && ASYNCIFY == 2
+      // JSPI: suspend on the real node:dns lookup, which fills desc.entries, then
+      // mint from the same descriptor.
+      return nodeSockHelpers.resolveAddrInfo(desc).then((eai) => {
+        if (eai) return eai;
+        {{{ makeSetValue('out', '0', 'writeAddrInfoList(desc)', '*') }}};
+        return 0;
+      });
+#elif NODERAWSOCKETS
+      // No synchronous DNS available: numeric and /etc/hosts names resolve above,
+      // anything else must be resolved out-of-band via emscripten_dns_lookup_async.
+      return {{{ cDefs.EAI_AGAIN }}};
+#endif
+    }
+    return desc;
   },
 
   getnameinfo__deps: ['$DNS', '$readSockaddr', '$stringToUTF8'],

src/lib/libfs.js

@@ -134,6 +134,15 @@ FS.staticInit();`;
       readMode = {{{ cDefs.S_IRUGO }}} | {{{ cDefs.S_IXUGO }}};
       writeMode = {{{ cDefs.S_IWUGO }}};
       mounted = null;
+#if USE_CLOSURE_COMPILER
+      // Closure (@struct) requires these declared ahead of time. The readiness
+      // wait-queue is populated lazily, and only on nodes that derive real
+      // readiness (sockets, pipes, an epoll's own node).
+      /** @type {Set<?>|null} */
+      listeners = null;
+      /** @type {number} */
+      exclTurn = 0;
+#endif
       constructor(parent, name, mode, rdev) {
         if (!parent) {
           parent = this;  // root node sets parent to itself
@@ -164,6 +173,48 @@ FS.staticInit();`;
       get isDevice() {
         return FS.isChrdev(this.mode);
       }
+      // The per-inode readiness wait-queue. The node carries a Set of listener
+      // entries {cb}; producers (SOCKFS, PIPEFS) call notifyListeners on a
+      // readiness transition, and poll()/epoll consume it. It lives on the node
+      // (not the fd) so dup'd fds share one queue. Only nodes that derive real
+      // readiness (sockets, pipes, and an epoll's own node) ever use this -
+      // always-ready types (regular files, ttys) never register or notify.
+      addListener(cb, exclusive = false) {
+        var entry = {cb, exclusive};
+        var listeners = (this.listeners ??= new Set());
+        listeners.add(entry);
+        return {listeners, entry};
+      }
+      notifyListeners(flags) {
+        // Iterates the set without copying, which is safe ONLY under a
+        // load-bearing contract that every internal listener must honour:
+        //   1. A listener must not run user code synchronously (a poll waiter only
+        //      resolves a Promise; an epoll registration only re-lists +
+        //      re-notifies; the epoll callback only schedules a tick). User code
+        //      runs on a later tick, never inside this loop.
+        //   2. A listener may delete entries only from ITS OWN waiter, never from
+        //      a sibling node's set that may be mid-iteration. (Deleting an entry
+        //      of the set being iterated here is fine - a Set tolerates removal of
+        //      a not-yet-visited entry mid-iteration; mutating a *different* node's
+        //      set is fine because that set is not being iterated.)
+        // Violating either gives silently skipped wakeups that are near-impossible
+        // to reproduce. Any new producer/listener must preserve it.
+        if (!this.listeners) return;
+        // Fire every non-exclusive listener. Among EPOLLEXCLUSIVE registrations
+        // (one fd watched by several epolls) wake only one, rotating round-robin
+        // per node, to avoid a thundering herd. (Only epoll registrations are ever
+        // exclusive; poll waiters and a node's own consumers are not.)
+        var excl;
+        for (var entry of this.listeners) {
+          if (entry.exclusive) (excl ||= []).push(entry);
+          else entry.cb(flags);
+        }
+        if (excl) {
+          var i = (this.exclTurn || 0) % excl.length;
+          this.exclTurn = i + 1;
+          excl[i].cb(flags);
+        }
+      }
     },
 
     //