I think IO Completion Ports [1] in Windows are pretty similar to kqueue [2] in FreeBSD and Event Ports [3] in Illumos & Solaris. All of them are unified methods methods for getting change notifications on IO events and file system changes. Event Ports and kqueue also handle unix signals and timers.
Windows will also take care of managing a thread pool to handle the event completion callbacks by means of BindIoCompletionCallback [4]. I don't think kqueue or Event Ports has a similar facility.
The documentation for the new thread pool APIs in Windows say that it is more performant than the old ones. Do you have any experience transitioning from the old thread pool to the new one, and if so did you notice performance differences?
Last decade I wrote a server based on the old API and BindIoCompletionCallback. I can't comment on the performance differences, but the newer threadpool API is much better to work with.
IMO one of the biggest problems with the old API is that there is only one pool. This can cause crazy deadlocks due to dependencies between work items executing in the pool. The new API allows you to create isolated pools.
The old API did not give you the ability to "join" on its work items before shutting down. You could roll your own solution if you knew what you were doing, but more naive developers would get burned.
You also get much finer grained control over resources.
I'm really lucky to have decided to dig into this stuff (for PyParallel) around ~2011, by which stage Windows Vista (where all the new APIs had been introduced) had been out for like, 5-6 years, so no, I never had to deal with the old APIs, thankfully.
Regarding the differences between IOCP and epoll/kqueue, it all comes down to completion-oriented versus readiness-oriented.
https://speakerdeck.com/trent/pyparallel-how-we-removed-the-...