| Regarding a public API, this GitHub comment suggests that people at Microsoft are working on it.[1] Of course, Microsoft can't commit to anything. As the winpty maintainer, I hope some kind of public API emerges to reduce my maintenance burden and make winpty more robust and efficient. Aside: About a year ago, I studied how the console handles worked (especially in relation to console attach/detach/alloc and CreateProcess), and I discovered a bunch of interesting things. For example, the "fake" handles were inherited regardless of the bInheritHandles parameter to CreateProcess, and an entire "ConsoleHandleSet" was inherited if a process attached to another process' console. Real handles in Win8 seemed to be either "bound" to a particular input/screen buffer or "unbound" so they worked with any attached console. It was possible to have a console open with no attached processes. I tried to formalize the behavior I observed here: https://github.com/rprichard/win32-console-docs In the course of testing, I found some neat bugs like: - A small program that blue-screens Vista - The inheritable flag on a console handle can be changed on any Windows version, except for Windows 7. - CreateProcess has an undocumented(?) behavior where it duplicates stdio handles into the new process rather than inherit them. (The handle values will generally change.) Up until Windows 8.1, a stdio handle of INVALID_HANDLE_VALUE was duplicated into a true handle to the parent process. (Because INVALID_HANDLE_VALUE is actually the GetCurrentProcess() pseudo-handle, you see.) - Assorted inconsistencies in Windows XP and WOW64 (probably fixed in newer OSs). I'm not sure how much confidence I have in the spec. It is backed by a big test suite, but there are so many cases to consider. [1] https://github.com/Microsoft/BashOnWindows/issues/111#issuec... |
Cool. There's a third option that you might want to consider for winpty. (I was experimenting with it before I left Microsoft.)
The basic idea is to use a DLL shim to provide your own console API implementation for a process and its children --- you can replace the system console API with your own by shimming DLL import and export tables. You'd use your own type of pseudohandle for your fake console handles; programs written to work with the old console pseudohandles should work correctly with your pseudohandles, provided you've hooked the relevant HANDLE-using APIs.