Support for interactive console application on Windows

[ppastercik]

Description

When developing a console PHP application that works with user input (for example implemented using symfony/console or laravel/prompts), interactive work using console input (for example using arrow keys), is available on Linux environments but is missing on Windows environments. This limits the development of console-based cross-platform PHP applications. It is circumvented either by limiting interactivity (which is the case with symfony/console) or by using a fallback to a non-interactive solution (which is the case with laravel/prompts, which has a fallback to symfony/console).

Specifically, this can be seen, for example, in the list selection. When on Linux we can select values using the up and down arrows.
Alternatively, when typing a password (invisible input), on Windows we need to use a special program to get the input without displaying it in the console, see hiddeninput.exe for symfony/console.

Here is a sample of the interactive application control, which is now not possible on Windows:

On Windows OS this is due to the fact that it is now not possible to update the console input stream mode from user code.

For interactivity, we need to allow developers to set these console modes:

Mode	Purpose	Default	Needed
ENABLE_ECHO_INPUT	Writes input to the sceen buffer	Enabled	Disabled
ENABLE_LINE_INPUT	Waits for carriage return	Enabled	Disabled
ENABLE_PROCESSED_INPUT	System processed Ctrl+C instead of input buffer	Enabled	Disabled
ENABLE_VIRTUAL_TERMINAL_INPUT	Converts input into sequences (for arrow keys, backspace, etc.)	Disabled	Enabled

Proposal

Adding support functions to PHP that allow setting the mode (using the SetConsoleMode function) for the input console stream without having to enable any PHP extension.

The question is the form of these functions.

We can use separate support functions (inspired by sapi_windows_vt100_support function), see the sample user code:

<?php
    $defaultEcho = sapi_windows_echo_input_support(STDIN);
    $defaultLine = sapi_windows_line_input_support(STDIN);
    $defaultProcessed = sapi_windows_processed_input_support(STDIN);
    $defaultVt100 = sapi_windows_vt100_input_support(STDIN);

    sapi_windows_echo_input_support(STDIN, false);
    sapi_windows_line_input_support(STDIN, false);
    sapi_windows_processed_input_support(STDIN, false);
    sapi_windows_vt100_input_support(STDIN, true);

    // Now it reads by characters, not echo input, read "CTRL+C" as character, with arrow keys, etc. as characters.
    $readedChar = fread(STDIN, 1024);

    sapi_windows_echo_input_support(STDIN, $defaultEcho);
    sapi_windows_line_input_support(STDIN, $defaultLine);
    sapi_windows_processed_input_support(STDIN, $defaultProcessed);
    sapi_windows_vt100_input_support(STDIN, $defaultVt100);

Or we can use universal support function with constants, see the sample user code:

<?php
    $defaultMode = sapi_windows_input_mode(STDIN);

    sapi_windows_input_mode(STDIN, $defaultMode & ~SAPI_WINDOWS_ECHO_INPUT_MODE & ~SAPI_WINDOWS_LINE_INPUT_MODE & ~SAPI_WINDOWS_PROCESSED_INPUT_MODE | SAPI_WINDOWS_VT100_INPUT_MODE);

    // Now it reads by characters, not echo input, read "CTRL+C" as character, with arrow keys, etc. as characters.
    $readedChar = fread(STDIN, 1024);

    sapi_windows_input_mode(STDIN, $defaultMode);

Another question here is whether the input stream STDIN needs to be defined, or if it could be omitted, since this can only be used for the console input stream anyway.

Proof of concept

I created a proof of concept using separate support functions.

The implementation of the supporting functions in PHP is here:
https://github.com/ppastercik/php-src/tree/proof-of-concept-laravel-prompts-on-windows

Sample of an interactive console application supporting the Windows platform. It uses the laravel/prompts package, whose implementation is extended with new support functions:
https://github.com/ppastercik/proof-of-concept-laravel-prompts-on-windows

Tested on the following terminals / emulators with php artisan app:test:

Name	Working	Differences
CMD	Yes
PowerShell	Yes
Cmder	Yes
ConEmu - CMD	Yes
ConEmu - Msys2	Yes
Console2	Yes
ConsoleZ	Yes
Git Bash	Yes
MinTTY	Yes
MobaXterm - SSH	Yes	(backspace is 0x08 and not 0x7F)
MobaXterm - Bash	Yes	(run with "winpty php artisan app:test")
PuTTY	Yes
ZOC8 Terminal	Yes

Alternatives

I have been exploring various alternatives within the laravel/prompts issue to see if there is any way to achieve interactive console behavior without interfering with the PHP source code.

External exe file for reading input - working

It is possible to create a support program that reads a character from the command line and returns it to the application (similar to hiddeninput.exe in symfony/console).

But there is the disadvantage that it would have to be run for each character read. Or it would have to be run as a separate process, getting the input through a pipe, and the process would have to be terminated properly at the appropriate time (with the need to prevent multiple processes from running concurrently, for example if another process were to be started for more input in a moment).

Another disadvantage is that in addition to the PHP code, the corresponding exe file would have to be distributed (which can be a problem with antiviruses, etc.).

Using the FFI extension - working

It is also possible to implement this using the FFI extension, where the appropriate mode is set using the SetConsoleMode function. But with this implementation, developers need to enable the FFI extension.

Alternative to Linux stty - not working

Next, I explored the possibility of creating an alternative to Linux stty for Windows. But here I encountered the problem that even though the documentation of Windows console modes states that the mode can be transferred between processes (using same STDIN), in real testing some modes were resetting when starting or ending subprocesses (running within user code) that had the STDIN of a PHP process as input. Thus, this option turned out to be not working.

Possible problems

Returned line ending

The Linux variant using stty returns the line break as a \n character, while the modified Windows variant returns the line break as a \r character. However, this can be resolved in user code.,

Unsupported mode combination

There is a restriction that it is not possible to have the ENABLE_ECHO_INPUT mode enabled with the ENABLE_LINE_INPUT mode disabled (see ENABLE_ECHO_INPUT mode description in SetConsoleMode function). So when we choose separate support function implementation, then when both modes are off, and we want to turn them both on, we need to call the functions in the correct order. Or, if we don't know the default state and the target state is defined by variables, we call one of the functions twice (sapi_windows_echo_input_support, sapi_windows_line_input_support, sapi_windows_echo_input_support).

Codepage of console input stream

Error in the default console input stream codepage in the Windows environment. When setted to UTF-8 (which is the default setting in PHP), the internal PHP implementation for reading data from the console does not read the characters ěščřžýáíé etc. This is due to the read function reading the console stream, which returns a buffer containing test\0\0test for the string testěštest.

Thus, we need to set an 'oem' codepage before reading, like in symfony/console, which will allow those characters to be read as well. But then it needs to be undone, because if the user code wanted to output something between reading each character, it would have the wrong coding. Thus, to read each character, the codepage needs to be reconfigured each time.

To avoid setting the codepage to read every character, it would be useful to consider extending the sapi_windows_cp_set function to allow the codepage to be set only for STDIN so that the output could still be in UTF-8.

Reseting modes after termination of a subprocess (with inherited STDIN)

When a subprocess is started using the current STDIN as input, some modes will be reset after the subprocess ends. Specifically, the following modes will be reset: ENABLE_ECHO_INPUT, ENABLE_LINE_INPUT, ENABLE_PROCESSED_INPUT. However, the setting of the ENABLE_VIRTUAL_TERMINAL_INPUT mode will be preserved. Thus, it would be advisable to point this out in the documentation that in such a situation, the given modes need to be reconfigured.

Breaking change

None. Only adds new functions.

Operating System

Windows

Conclusion

If there is interest in adding these support functions, the question is the form of implementation. And another question is whether to extend the implementation of the sapi_windows_cp_set function to include the ability to set the codepage to STDIN only (which is mentioned in the "Codepage of console input stream" section under "Possible problems").

Is there any next steps I can do?

[nielsdos]

Very nice proposal and very well written. The way forward I think is proposing an RFC to the internals mailing list as it's about adding new APIs to the language. I'm sure some people will have opinions about the design.
We generally use the mailing list instead of GitHub for such discussions. This is so that we can have a discussion on the mailing list with both maintainers and community members, i.e. having a greater reach.
Please see https://wiki.php.net/rfc/howto for a how-to guide for an RFC.

It's also very good that you already have an implementation, it helps if people can have a look at some implementation.

[thg2k]

It would be awesome to have terminal control abstraction in core PHP. I usually get away with use system("stty ..."); but that only works on Linux of course. Big +1

[github-actions]

There has not been any recent activity in this feature request. It will automatically be closed in 14 days if no further action is taken. Please see https://github.com/probot/stale#is-closing-stale-issues-really-a-good-idea to understand why we auto-close stale feature requests.

[Wirone]

Was RFC proposed yet?

[ppastercik]

@Wirone
Not yet, sorry.

In October I wrote an initial description of the proposal for the official PHP mailing list, as a first step before creating the RFC. See: Proposal: Add support for interactive console application on Windows

But then I had other priority things to deal with. Maybe I should have let you know here that I'll come back to it later.

I suppose this month I could get back to it and create an RFC.

However, before creating the RFC, I want to explore the possibilities of some abstraction over the console input, as @thg2k writes above (perhaps by calling the appropriate system functions that are otherwise called by the stty command). Because I'd like to have uniform control of console input independent of platform. But I want to see if this can be done easily, or if it would overcomplicate the RFC and thus delay the eventual implementation.

Plus I still want to explore more in depth working with the codepage when processing console input on Windows. Because my initial attempt showed that it is not possible to read UTF-8 correctly on Windows:

Error in the default console input stream codepage in the Windows environment. When setted to UTF-8 (which is the default setting in PHP), the internal PHP implementation for reading data from the console does not read the characters ěščřžýáíé etc.

So I would like to look into the possibilities, if it would be possible to directly modify the reading of the console input stream on Windows to support UTF-8 codepage correctly.

[github-actions]

There has not been any recent activity in this feature request. It will automatically be closed in 14 days if no further action is taken. Please see https://github.com/probot/stale#is-closing-stale-issues-really-a-good-idea to understand why we auto-close stale feature requests.