ConPTY Version probing

[patrickbkr]

This is more of a question than a feature request.

I'm currently working on adding PTY support to libuv. Bundling ConPTY is not an option, so I have to work with the version kernel32.dll gives me. There are some relevant developments in the last few years/months.

• Support for Overlapped IO (#17510)
• ConptyReleasePseudoConsole (#14544)
• Probably a few more

How do I know which features are available in the ConPTY version bundled with Windows? Is there some version number I can retrieve? Is there a Matrix of Windows version and ConPTY version?

Also: Is there any documentation on how to use these older ConPTY versions? I find it pretty difficult to find any in-depth information on this. For example, here are some of the questions I've stumbled over:

• Do I have to use synchronous pipes for in and out? (I think: Yes in older versions, no in recent versions.)
• Can I use the same (bidirectional) pipe for PTY in and out? (I think: Yes.)
• Do I have to perform stdin writing and stdout reading each on their own thread? (I think: No.)
• What's happening when I close the pty write handle? (I think: ConPTY sends a CTRL-C to the process.)
• How do I properly shut down a ConPTY session? (According to this comment and this remark, it's a pretty delicate dance I have to perform.)

[lhecker]

How do I know which features are available in the ConPTY version bundled with Windows? Is there some version number I can retrieve?

As a good Microsoft employee, I have to repeat our official recommendation against testing for version numbers in place of feature detection. We would never! 😏

Is there a Matrix of Windows version and ConPTY version?

There is not. Is there anything in particular you'd like to know?

Do I have to use synchronous pipes for in and out? (I think: Yes in older versions, no in recent versions.)

"Yes", but for both old and new versions. Support for overlapped IO was only added late last year (#17510) which was after the feature cutoff for Windows 11 24H2 (build 26100). It'll be part of the next big Windows OS update. Maybe that could be 26H2? There are no official dates for that yet.

Can I use the same (bidirectional) pipe for PTY in and out? (I think: Yes.)

I believe so. I've only ever tried that with ConPTY after #17510 merged though. I currently don't see how it wouldn't work with older ConPTY versions though.

Do I have to perform stdin writing and stdout reading each on their own thread? (I think: No.)

You don't have to, but if you have to use synchronous pipes is there another way to do it correctly? After all, your read/write can legitimately block indefinitely (backpressure / pipe full, etc.). It depends on your use case!

What's happening when I close the pty write handle? (I think: ConPTY sends a CTRL-C to the process.)

More specifically, it sends a CTRL_CLOSE_EVENT.

How do I properly shut down a ConPTY session? (According to #15006 (comment) and this remark, it's a pretty delicate dance I have to perform.)

In this case I do recommend feature detection for our new ReleasePseudoConsole function! I just published it and wrote some info about why you want to use it. ClosePseudoConsole also has a new remarks section. 🙂

I recommend doing a GetProcAddress on ReleasePseudoConsole and using it if it's available. Its existence indicates that both ReleasePseudoConsole and ClosePseudoConsole are the newer, safer versions which won't deadlock (#14544, as you mentioned).
For older versions of Windows, I recommend copying our winconpty code and using it directly on the HPCON handle. The HPCON handle may change in future versions of Windows, but it's very safe to assume that old versions of Windows will not change it anymore. The code you need to copy is just a few lines so it should not unnecessarily bloat libuv.

---

Let me know if I forgot anything or if you have any other questions!

[lhecker]

FWIW, we haven't updated the "Creating a Pseudoconsole session" document yet, unfortunately. I'm sorry about that.

[patrickbkr]

@lhecker Thanks for this very extensive reply and for updating the docs! I've adapted my code accordingly.
I have one more issue though. I'm absolutely unable to make ConPTY close the STDOUT handle on me with a minimal test child program that does a single read on STDIN, echoes that on STDOUT and then exits.
I do the following in the parent:

1. Call CreatePseudoConsole (I'm using two synchronous anonymous pipes for in and out.)
2. Call CreateProcess
3. Call ReleasePseudoConsole
4. Call ReadFile(stdout_handle) - blocks
5. Call WriteFile(stdin_handle)
6. Call CloseHandle(stdin_handle)
7. ReadFile(stdout_handle) from step 4. returns with 16 bytes (VT stuff)
8. Call ReadFile(stdout_handle)
9. ReadFile(stdout_handle) from step 8. returns with 107 bytes (expected output)
10. Call ReadFile(stdout_handle) - blocks
11. Process exit is detected, call ClosePseudoConsole - returns instantly

The ReadFile(stdout_handle) from step 10 blocks indefinitely. I would expect it to return with 0 bytes read instead.

ReadFile, WriteFile and ClosePseudoConsole are all executed on separate threads (on a thread pool).

Do you have any hunch what could cause that behavior?

[lhecker]

Perhaps it's a race condition and it just so happens to work for me?

[patrickbkr]

I'm on a pretty fast machine (Ryzen 9 5950X). So that indeed has the potential to run off the beaten tracks WRT race conditions.
Do you suspect the race on the libuv or ConPTY side?

Any idea how I could proceed here?
I've already pondered working around this by manually cancelling the ReadFile via CancelSynchronousIo after some timeout after the child process has died. Knowing that the code actually works on some (most?) systems, makes this idea more attractive.

[patrickbkr]

If this issue is inside ConPTY, then I'll have to find a way to work around it. (Even if it's getting fixed, because I want to be able to work with older ConPTY versions as well.) I believe I can escape the deadlock with CancelSynchronousIo. But the zombie ConPTY will stay around. Is there any chance to deconstruct ConPTY from the outside? Are there processes I'd need to kill or threads to stop?

[lhecker]

The reason it "works" for me is because you hardcoded the uv_run_tests.exe paths in the test. 😅 After fixing that, it then hangs for me as well. Ideally, the test should not complete if the executable cannot be found, of course.

The reason it hangs is because you forgot to close the pty_in_read and pty_in_write handles after calling CreatePseudoConsole. It doesn't take over ownership, so it's your responsibility to close the pipes. (I feel like I should highlight that in our docs...)

[patrickbkr]

Oh wow. I've actually managed to make it work as well. Thank you! The cause is a bit embarrassing, but I'm really glad for your help!

I actually did know about the need to close the ConPTY side handles, but somehow missed to do it. (I originally wrote that code as part of a different project (MoarVM) and then ended up moving it into libuv, I guess the handle closing bit got lost during the transition.) So the docs can't be blamed for my failure to make this work.

[patrickbkr]

All my questions are answered. Thanks lhecker++ once more! I'm closing this discussion.