Hacker News new | ask | show | jobs
by umren 1505 days ago
Your friends haven't tried R and Julia?

I don't see how elixir can replace python any time soon because it's DS ecosystem is lacking rn sadly

3 comments

Oh, he definitely tried R but was not very impressed. Says it's only moderately better than Python's otherwise excellent bindings to a lot of libraries, and that he had no patience for Julia. He isn't exactly a programmer, he's just a very practical guy that is open to learning some programming to get his stuff done quicker and with less mistakes but that's about it. He has a short fuse for ecosystem deficiencies -- tbf I think we the programmers have to learn from such people. :D

And yeah, Elixir's scene in this area is just now starting. The NX project opened a lot of possible doors but the community just started stepping up. It's going to be a while, granted, but I am optimistic because Elixir's overall community values quality and centralization (with all the sometimes problematic collaboration efforts that entails) over rushing to have 10 competing implementations.

Time will tell. I have no horse in the race either way, just sharing various observations really.

As an R user who can't stand working in Python, where Elixir I think has the most potential is not the data science part but data engineering. The whole world of ELT/ETL pipeline and workflow tools written in Python is a mess IMO, and Broadway/Flow + Ecto + what's going on with Explorer etc I think can quickly replace a whole mess of python/scala/EMR type workflows in a language and environment that feels way better suited to the problem space, in my experience.
Yeah, can confirm, I did a ton of Elixir work in the ETL space and it's amazingly well-suited for it. If you don't get an OCD over the fact that it obviously can't work as fast as Rust (realistically it's likely anywhere from 30x to 100x slower even) then the ergonomic and intuitive syntax combined with the transparent parallelism create an all-around win.

So far I've never regretted using Elixir for ETL, but I do confess that for two projects I was mighty tempted to try a rewrite in Rust because ingesting several dozens of billions of records in a dynamic language isn't exactly fast.

But, again, the ergonomics trump everything else 99% of the time for me. Plus for one-off ingestion tasks nobody cares if it takes 30 minutes or the whole night.

of course they do, R much more than Julia tbh, I think on the prototyping side of the venture Elixir experience is more polished, even though slower ( Nx IMO is promising in that space)