[nortiero]

Hi! Memory is definitely an issue today, especially with big fat server processors...

Here is the small program I wrote, first allocates an array, then writes a random placed linked list that touches all the array cells (e.g. start->|4|6|2|5|3|end , so it goes to cell 1, then 4, then 5, then 3,2,6 and done.

Then it reads the same array from start to finish, just to sum contents. This is fast and goes to 6-8 GB/s, depending on unrelated memory pressure from video and os tasks. This is the advertised speed. So I don't think it's a swap issue (and my SSD is way faster :-) I've also tried with smaller and bigger arrays. Just ensure it stays in real memory.

Moving around in a big array is very hard on the memory controller: - no prefetch possible; - row and column changes at every access (almost), so commands have to be issued to burst terminate, close row, close bank, precharge maybe, open bank, activate row, fetch address, and who knows; - no advantage from interleave; - no advantage from a fat 64 or 128 bit bus;

It is the combined time needed for each and every memory read (due to randomization) that cause performance to drop.

This is the code I've been using. I've modified it to work on unices (OSX has an issue with timeval struct), but have not tested on them. It compiles, may need to be altered a bit. Launch it with ./a.out <array_size> <random seed>. You will note that, as soon as the array moves out of caches, hell breaks loose. Works with gcc or clang.

https://godbolt.org/g/qX39tL

[logicallee]

Thank you. This is very good and thank you for sharing. You should probably copy the usage information from your comment here to the top of the file, so they're not separated.

I tried it and got the same result you suggested (I used a code running site but it shouldn't matter, the VM it runs in should have virtualized operations that shouldn't slow it down by a complete order of magnitude so I roughly trust the results I'm seeing.)

FYI, here is the output with 10m items. (It times out if I try to increase it further, but that is just because the code-running site kills programs that run longer than a set time)

  ht_size: 10000000
  sizeof(int): 4
  seed: 256
  start run
  roll 64: -2004260032
  time (seconds:+nanoseconds) 1s:270464 ns
  sequential run: roll 64: -2004260032
  time (seconds:+nanoseconds) 0s:281244 ns

so if that is 1.27 sec for 10mill it should obviously be around 12 seconds for 100 million. Just brutal. I had no idea things could be so bad.

Thanks so much for sharing your program with us!