Hacker News new | ask | show | jobs
by elchief 2585 days ago
Hadoop? Spark? Are those not systems software?
2 comments

Beowulf? MPI? Were these not already things 30 years ago?

https://github.com/intel/spark-mpi-adapter

Oh look, a paper; "For example, a recent case study found C with MPI is 4.6–10.2×faster than Spark on large matrix fac-torizations on an HPC cluster with 100 compute nodes"

Does it sound like large data analytics would have horribly stagnated?

Hadoop was not an academic project although Spark was.