[willvarfar]

Thank you for making me challenge my assumptions and memory.

I've asked around a bit and am fairly sure of my facts again, and my understanding is:

* kernel mode is cheaper than ever to reach; SYSCALL/SYSENTER etc so its not even an interrupt and there are no hardware threads or anything involved

* in kernel mode, the thread can get straight at the buffer and the locks that protect it; there is nothing that we'd call a 'context switch' in there

* being as this seems to be what is meant by monolithic kernel, surely its the same on freebsd too?

[kqueue]

I totally agree that SYSCALL/SYSENTER/SYSRET are very cheap to execute. But these instructions only takes care of the ring switch and are not executed alone.

When you make a system call, a trap is issued that causes the hardware switch to kernel mode.

The hardware pushes onto the per-process kernel stack the pc, status word, and the kernel code takes care of saving the registers, esp, etc.. this is called a task context switch. Context switching between processes is much more expensive but task context switching is still considered a context switch.

When you are making a system call, it's still much more expensive then most of the work you are doing in your program hellepoll and hence it's your bottleneck. This is why you don't see your process's CPU at 100%.

On a related note, whenever you have a program doing a lot of network IO, you are essentially causing a lot of process context switches because each time you get data on the wire you cause a context switch because the kernel needs to handle this hardware interrupt.

[willvarfar]

So we only disagree in edge-case terminology. I think your trying to worm out of your missclassification, but no worries.

Yes, to get these numbers i have had to minimise syscalls. Thats the advantage of hellepoll. I wrote the http server just to release it, as before it was an rtmp server but it was commercial and couldn't be released. That had write buffers usr side too which helped even more.

And i understand what ab and httperf test, and yes i am counting served pages.

Finally, i spend a lot pf time staring at linux perf reports and timecharts.