|
|
|
|
|
|
by marginalia_nu
960 days ago
|
|
|
Yeah I've started using it with my search engine as well. It's fantastic how versatile it is for data all manner of data munging. Just the other day I used it to transform an unordered 60 GB CSV file with links and texts into a 3 GB parquet file that's so fast I can create a projection for the relevant data of each partition in like a minute (which then fits in memory). It has some minor stability issues so I'm not sure I'd build a full blown application on top of it, but for data transformation tasks it's amazing. |
|
|