Hacker News new | ask | show | jobs
by blibble 4878 days ago
for reading files and transferring over the network: the limit is always going to be the IO devices, not an extra context switch and a few extra copies...
1 comments

s/always/usually/. Depending on the disks involved and your network hardware (either a super-fast link, or even a system with a slow CPU and cheap network hardware that expects the drivers to do the heavy lifting in software), it actually could end up CPU-constrained.

But for the usual case where it is I/O-constrained, you can often get files across more quickly by throwing CPU at reducing the total amount of bandwidth required by, e.g., using something like bzip or pbzip:

  pbzip2 < file | nc $host $port
And on the other side:

  nc -l $port | pbzip2 -d > file