I don't find that surprising at all: these are very large files, and they aren't probably not scrubbing around in them back and forth but streaming them off the disk in a single pass; so you could have 128GB of RAM and you are unlikely to get much caching on the files themselves... the only thing that matters is the directory index structures, and I wouldn't be shocked if they needed more than 16GB for that (because these are large files).
You don't need 10GbE and Cat6 for that. 1GbE and Cat5/5+ are fine. The SOHO NAS from Synology or QNAP with ARM CPUs, 1-2 GB RAM and 1x 1GbE ports are achieving such performance.
That's what caught my eye too. 100MB/s is nothing. The author claims that he's getting half that with SMB, which means something is horrifically broken with his system.
Anything worth using as a file server should have no trouble doing 100MB/sec with pretty much any protocol.
No, something is horrifically broken in Apple SMB client implementation. That's probably why they never actually phased out AFP, because it's the only way to move data fast on MacOS.
Paradoxically, the speeds that I'm getting from my rMBP13 for SMB transfers are comparable over Ethernet (both Apple Thunderbolt adapter and rangom assortment of TB2 docks) to Linux and Windows machines, but over Wi-Fi, rMBP is much faster than any Linux or Windows machine I have. The Mac machine is 802.11ac 3x3 MIMO by Broadcom, while the others are only 2x2 by Intel, but I'm not sure that the speed difference could be explained just by this factor.
Yes, there were small bugs in the past, like waiting for some timeout when browsing shares on the server when the auth is done via Kerberos, but they were ultimately fixed.
I was with C-COR/ARRIS when we bought N-Cube for their VOD streaming technology (cable television). You'd be very surprised at the difference between the amount of content stored and the 5 minute buffered content currently being streamed. I'm not surprised at the amount of memory OR that the server isn't sweating under the workload.
I guess now that clients can't attach, it's technically in the warm-down phase of its workout.