Hacker News new | ask | show | jobs
by orthoxerox 744 days ago
Because it's a drop-in replacement that lets you (theoretically) spend O(1) development effort on speeding up your Spark jobs instead of O(N).

I say theoretically, because I have no idea how Comet works with the memory limits on Spark executors. If you have to rebalance the memory between regular memory and memory overhead or provision some off-heap memory for Comet, then the migration won't be so simple.

1 comments

and this is unfortunately the case for Velox.
care to elaborate?