Cygwin: pty: Return EMFILE when too meny ptmx are opened

Previously, opening /dev/ptmx fails without setting errno when it
is opened too many times. With this patch, return EMFILE in that
situation.

Addresses: https://cygwin.com/pipermail/cygwin/2025-March/257786.html
Fixes: 09738c3 ("Cygwin: pty: setup new pty on opening the master, not in constructor")
Reported-by: Christian Franke <[email protected]>
Signed-off-by: Takashi Yano <[email protected]>
(cherry picked from commit 7fc7d8b)

Author: tyan0

winsup/cygwin/fhandler/pty.cc

@@ -1933,7 +1933,10 @@ int
 fhandler_pty_master::open (int flags, mode_t)
 {
   if (!setup ())
+    {
+      set_errno (EMFILE);
       return 0;
+    }
   set_open_status ();
   dwProcessId = GetCurrentProcessId ();
   return 1;

winsup/cygwin/release/3.6.1

@@ -27,3 +27,6 @@ Fixes:
 - Accommodate a change in Windows exception handling affecting software
   generated exceptions.
   Addresses: https://cygwin.com/pipermail/cygwin/2025-March/257808.html
+
+- Return EMFILE when opening /dev/ptmx too many times.
+  Addresses: https://cygwin.com/pipermail/cygwin/2025-March/257786.html

winsup/cygwin/tty.cc

@@ -147,7 +147,7 @@ tty_list::allocate (HANDLE& r, HANDLE& w)
     termios_printf ("pty%d allocated", freetty);
   else
     {
-      system_printf ("No pty allocated");
+      termios_printf ("No pty allocated");
       r = w = NULL;
     }