[nu11ptr]

Is this new async. I/O feature for Linux only?

I know Windows has IOCP and also now an IORing implementation of its own (Less familiar with macOS capabilities other than POSIX AIO).

https://learn.microsoft.com/en-us/windows/win32/api/ioringap...

Update: Most of the comments below seem to be missing the fact that Windows now also has an IORing implementation, as I mentioned above. Comparison article here:

https://windows-internals.com/ioring-vs-io_uring-a-compariso...

[macdice]

I am not a Windows guy but I (with help) managed to get IOCP working for this in a basic prototype. Will share publicly soon. I also sketched out an IoRing version (if you are interested in helping debug and flesh that out let me know!).

Main learnings: the IOCP version can't do asynchronous flush! Which we want. The IoRing version can! But it can't do scatter/gather AKA vector I/O yet! Which is an essential feature for buffer pool implementation. So actually I am basically waiting for IoRing to add support for that before taking it too seriously (I can see they are working on it because the ops are present in an enum, it's just that the build functions are missing).

So my guess is that in a year or so we should be able to run all PostgreSQL disk I/O through IoRing on Windows. Maybe?

Another complications is that it really wants to be multithreaded (consuming completions for IOs started in another process requires a lot of hoop jumping, I made it work but...) This will resolve itself naturally with ongoing work to make PostgreSQL multithreaded.

The next problem is that IoRing doesn't support sockets! So in future work on async networking (early prototypes exist) we will likely also need IOCP for that part.

[p_ing]

Look into Registered I/O for sockets.

[lfittl]

It depends on the I/O method - as described in the article, "io_uring" is only available on Linux (and requires building with liburing, as well as io_uring to be enabled in the Kernel), but the default (as of beta1) is actually "worker", which works on any operating system.

The "worker" method uses a dedicated pool of I/O worker processes that run in the background, and whilst not as performant as io_uring in our benchmark, did clearly outperform the "sync" method (which is the same as what Postgres currently has in 17 and older).

[nu11ptr]

> "io_uring" is only available on Linux

Windows now also has IORing (see my comment above)

[severino]

Yes, but what's your point? It's not that we can take this new Postgres version and just compile it in a Windows machine supporting IORing, can we? It requires some effort and time -many times by volunteers- and given that most Postgres deployments are running under Linux, it's understandable that the developers focus on that platform first.

[nu11ptr]

Misunderstood - thought you were saying ioring was a feature specific to linux, but you meant specific to linux on Postgres.

[stingraycharles]

Sounds like this feature is based on io_uring which is a Linux feature. I would be surprised if they implemented async io on Windows before they would on Linux given the user/deployment base being very Linux-heavy.

[p_ing]

Yeah, surprise Linux had to play catch up to a Windows 1994 release! Same with the scheduler, I'd argue Windows does OOM better than Linux today...

Windows even had the concept of io_uring before, but network only with Registered I/O back in the Windows 8 (8.1?) days.

Linux still lacks the "all I/O is async" NT has.

The underlying kernel and executive of Windows aren't primitive pieces of trash. They're quite advanced, ruined by PMs and the Ads division throwing crap on top.

And yes, Windows' I/O Ring is a near 1:1 copy of the Linux implementation, but IOCP/OVERLAPPED I/O data structure preceded it since NT's first release.

This isn't a pissing match, just we all hope that kernel devs learn from each other and take the best ideas. Sometimes we, IT, don't get to choose the OS we run -- it's dictated by the apps the business requires.

[cyberax]

> Same with the scheduler

Windows does OOM far better than Linux because it doesn't really overcommit RAM.

But the CPU _scheduler_ in Linux is far, far, far better than in Windows. Linux can even do hard-realtime, after all.

[greenavocado]

How difficult would it be to completely tear out the Windows desktop experience and just use the system and display drivers without the rest? Has anybody attempted such a feat?

[p_ing]

There is Windows Server Core which removes everything but a CLI, but you still have the normal login experience, you still have a "desktop" (no start menu, taskbar, etc), you can still launch normal Win32 apps... for the most part (task manager, notepad, and so on).

Win32 is also responsible for core Services, which means you can't de-Windows-ify Windows and strip it down to an NT API-only. All other personalities (OS/2, POSIX, SFU) have a dependency on Win32, as well.

You're still running the WindowServer of course; it's part of the Executive.

That said, with a bunch of modifications, NTDEV did get Windows 11 down to it's bare minimum, and text only to boot. So I guess it's technically possible, though not useful.

https://www.youtube.com/watch?v=SL6t_iuitxM

[smileybarry]

> There is Windows Server Core which removes everything but a CLI, but you still have the normal login experience, you still have a "desktop" (no start menu, taskbar, etc), you can still launch normal Win32 apps... for the most part (task manager, notepad, and so on).

Yep, they've replaced nearly every UI with text (the login window is a TUI), though there's still some shell DLLs and the whole thing still uses a window manager. That's honestly for the best, since it allows you to migrate full installations with some UI-based apps to a Core installation with them intact.

> That said, with a bunch of modifications, NTDEV did get Windows 11 down to it's bare minimum, and text only to boot. So I guess it's technically possible, though not useful.

Windows has had a "text mode" since at least Windows XP IIRC, but it's really not that useful, if at all. Even for rescue operations you're better off with Windows PE.

[Ericson2314]

There is always https://en.wikipedia.org/wiki/ReactOS ! :)

[password4321]

Free-as-in-beer Hyper-V Server 2019 is in extended support (only security updates) until 2029.

[dboreham]

There are several such things. The Windows installer uses one. X-Box uses another.

[nu11ptr]

> this feature is based on io_uring which is a Linux feature

And now also a Windows feature, see my comment above for info

[zokier]

IoRing != io_uring. They are similar APIs but not the same thing.

[PaulHoule]

For a long time ago there have been APIs to do asynchronous file I/O on the books for Linux but they weren't worth using because they didn't really speed anything up.

[anarazel]

They sped up things for a long time - but only when using unbuffered IO. The new thing with io_uring is that it also accelerates buffered IO. In the initial version it was all through kernel worker threads, but these days several filesystems have better paths for common cases.

[immibis]

IIRC they literally just did the sync I/O on a worker thread.

[DmitryOlshansky]

That’s the POSIX AIO. Linux has had AIO that was only supported for direct I/O and had certain limitations making it awkward to use.

[spwa4]

Yes, although Windows has had async I/O since Windows NT 3.1, their API is still not supported by Postgres.

[anarazel]

FWIW, there are prototype patches for an IOCP based io_method. We just couldn't get them into an acceptable state for PG 18. I barely survived getting in what we did...

[spwa4]

I didn't mean that as a criticism on Postgres, certainly not on Postgres developers. I really look forward to evaluating this change. I'm just reminiscing a bit about admiring Windows NT ... and being incredibly disappointed by the market.

[nu11ptr]

Yes, that was IOCP, however, Windows now also has IORing (see my comment above)

[mijoharas]

Does anyone run postgres on windows?

Given that they still target it there must be a user base. Does anyone know the statistics of usage by platform? Anyone here use it?

Genuinely curious, windows backend dev is something I know very little about.