[namkt]

I would have thought that allocations in managed languages like Go/Python would have been the "fast" part of the processing. Isn't technically the GC that's slowing you down, and not the allocation per se? For one-shot input/output programs like these I guess you could tune the GC to kick in with less frequency.

You also note that reading a file sequentially from disk is very fast, which it is, but there is no guarantee that the file's contents are actually sequential on disk (fragmentation), right? We'd have to see how the file was written, and I guess at worst you'd be reading sequential chunks of a hand-wavy 4KB or something depending on the file system and what not. I'm sure others can fill in the details.

Just nit-picking here.

[PaulKeeble]

Files aren't stored sequentially in an SSD anyway, they are scattered all over the place physically on different blocks just due to the way SSDs work. This doesn't hurt their sequential read performance at all since they have no seek time they already have a lookup table from the physical block the OS sees to a virtual location within themselves.

However one thing I found out a few years ago is that old data can be slow to read as a lot of error correction kicks in. Additionally a lot of fragmentation at the operating system level in Windows has quite a bit of overhead. It can seriously degrade performance to about 50MB/s sequential reads. In practice defragmentation/rewriting of certain high write files may be necessary on SSDs because Windows read performance degrades at high levels of fragmentation.

[wtallis]

> You also note that reading a file sequentially from disk is very fast, which it is, but there is no guarantee that the file's contents are actually sequential on disk (fragmentation), right?

Correct. And there are actually two layers of fragmentation to worry about: the traditional filesystem-level fragmentation of a file being split across many separate chunks of the drive's logical block address space (which can be fixed by a defrag operation), and fragmentation hidden within the SSD's flash translation layer as a consequence of the file contents being written or updated at different times.

The latter can often have a much smaller effect than you might expect for what sounds like it could be a pathological corner case: https://images.anandtech.com/graphs/graph16136/sustained-sr.... shows typically only a 2-3x difference due to artificially induced fragmentation at the FTL level. But if the OS is also having to issue many smaller read commands to reassemble a file, throughput will be severely affected unless the OS is able to issue lots of requests in parallel (which would depend on being able to locate many file extents from each read of the filesystem's B+ tree or whatever, and the OS actually sending those read requests to the drive in a large batch).