Hacker News new | ask | show | jobs
by kyllo 4733 days ago
It's not a speed optimization. It just allows you to process a much larger dataset than you can fit in main memory. It will be slow because it's hard disk I/O, but you can do a sparse matrix multiplication on terabytes worth of data in a database with SQL. You couldn't do that in Python or Java without running out of memory, unless you used a MapReduce job distributed to many worker machines--and then you'd get a bill for the EC2 time or whatever.