Hacker News new | ask | show | jobs
by CHY872 4086 days ago
I'd be very wary of criticising Spark based on the technologies it's built on.

In particular, a decent garbage collector will give you performance dependent on the number of live objects (typically low) and not on the number of allocations and deallocations, as you might see in a non-garbage collected language. This gives great allocation performance and reduces overheads.

The disadvantages can be (potentially long) GC pauses and higher overall memory requirements, but in practice this isn't usually a problem for non-interactive systems.

Of course, if you do have a device with low memory and low tolerance for GC pauses (i.e. mobile gaming) there might be a problem.

The main disadvantage seems to be less predictable performance; which could be a problem in domains which require good timing performance, but that's not really Spark's problem.

A GC'd language is also generally easier to program in, since one doesn't have to (in general) worry about memory management, so it's generally a lot easier to program very large systems with lots of moving parts.