Hacker News new | ask | show | jobs
by paulsutter 2515 days ago
Programs should really use unbuffered i/o for large files read only once (yes i know Linus doesn’t like unbuffered i/o but he’s wrong)

> This comes into play when you copy or access huge files that are going to be read exactly once

1 comments

Readahead is still useful for large files read sequentially once, and that needs to be buffered. Such programs should use posix_fadvise().
you can readahead as far as you like with unbuffered io
If you are reading unbuffered (ie. O_DIRECT) then you are reading directly into the memory block the user supplied, so you cannot read ahead - there's nowhere to put the extra data.
Of course you can read ahead using multiple buffers, you can issue as many reads as you want concurrently
I think it is pretty clear I was referring to kernel-mediated readahead. Sure, you can achieve the same thing in userspace using async IO or threads.
I think it’s pretty clear that I was referring to reading large files using direct io, and your comment was totally irrelevant