Hacker News new | ask | show | jobs
by eloff 3513 days ago
A syscall is a lot cheaper than shown here in terms of direct costs ~ 150 cycles, and also a lot more expensive, depending on the call, when you factor in the full cost of the cache that gets clobbered. More like the full cost of a context switch shown, around 10-30K cycles[1]. For this reason it's important to use system calls that allow you to amortize more work into one call, like preadv, pwritev, recvmmsg, and sendmmsg.

[1] http://www.cs.cmu.edu/~chensm/Big_Data_reading_group/papers/...
1 comments
bogomipz 3513 days ago
>"and also a lot more expensive, depending on the call, when you factor in the full cost of the cache that gets clobbered."

But a syscall doesn't necessarily mean a context switch right? It's just a mode switch if the kernel is servicing the syscall on behalf of the same process. The cache wouldn't get clobbered then because the kernel is mapped into the top of every process, for just this reason.

Or am I misunderstanding what you are saying?

link
fdsaaf 3513 days ago
You are correct. The author is considering the cost of accessing kernel mode code and data, which I don't think is fair
link
eloff 3513 days ago
It absolutely is fair. For a long time after a syscall your program will run slower thanks to the increased cache misses from the kernel clobbering your stuff in cache. That's a real price that you pay for a syscall and it would be wrong not to count it. However, if you found some other way of doing the work, e.g user code, kernel bypass, amortized bulk syscall, those will also have a (lesser) effect on the cache. So to be fair you compare a syscall against that, not against zero.
link
JoshTriplett 3513 days ago
If you have enough CPU cache, the CPU will cache both kernel and user code/data for your process. If you run long enough and access enough user data that the kernel bits get evicted, then you'll take more of a performance hit, but the same thing applies to accessing enough user data that different parts of your user data get evicted.
link
eloff 3513 days ago
Maybe this is more reflective on the kinds of software I work on, but I don't find that I have enough cache. So syscalls always come at a steep price. In fact I am tempted to break up one program into multiple processes and use the cache partitioning in Xeon v4 to prevent the different parts of my program from clobbering it's own cache.
link
bogomipz 3513 days ago
Oh I see, that makes sense. Agreed. Thanks.
link