Hacker News new | ask | show | jobs
by lossolo 3325 days ago
> The new prototype has 160 TB of shared memory spread across 40 physical nodes, interconnected using a high-performance fabric protocol.

So basically something similar to RDMA scaled to 40 nodes, 4 TB RAM per node.

4 comments

the article is pretty weak, people have built rdma machines that size before, and there have been architectures that allow for direct memory addressability of that size before.

so i have to assume the latter. that seems to be borne out by the little information i can find...addressable persistent memory is clearly a theme. but i haven't found any discussion of what kind of latency hiding mechanisms might be at play and what kind of consistency model is being used. will keep looking for anything detailed an authoritative

(edit - this seems to be pretty relevant https://www.labs.hpe.com/publications...but i don't know how much of it was speculative and how much was built...after all the security papers, there are some about concurrency control)

The Next Platform has had many articles with bits of detail on HPs Machine. Unfortunately, they don't really have any tagging mechanism, but you can search the phrase "the machine" or look at related articles there.

https://www.nextplatform.com/2017/01/09/hpe-powers-machine-a...

Great resource. Based on that, it looks like the fabric has a bandwidth of 600Gb/s (or 1.2Tb/s full duplex).

The interconnect seems to be the real innovation here.

I think the interesting part is the programming model and not the amount of RAM. This computer has no separate storage area. If you want to manipulate data, you can do it in-place instead of reading it into a RAM-like working memory and then writing the result to a disk-like storage.
I think the interesting part is the programming model and not the amount of RAM. This computer has no separate storage area.

You can already mount a filesytem backed by ram, or mmap files that exist on disk. What existing distinction between RAM and storage is holding anything back?

I see a reason some techniques will get more popular, but I don't really seen the difference. Saving a load operation? Not having to worry about flushing to long term storage? Am I missing something?

Based on HPE's website, it looks like they're going to be making their fabric part of this:

http://genzconsortium.org/faq/gen-z-technology/

Underwhelming - you could build the same exact thing with a bunch of white box servers and a pair of Infiniband switches.