Hacker News new | ask | show | jobs
by krastanov 1135 days ago
Could you elaborate on the ecosystem problem? For my corner of the world, Julia probably has one of the highest quality ecosystem (differential equations, physics modeling, autodiff through very complicated code, probabilistic programming, SIMD/multithreading, and wonderful plotting libraries (the Makie.jl ecosystem) and good data wrangling capabilities (the Dataframes.jl ecosystem)).

I am curious what are the fields where it is less well developed?

1 comments

> I am curious what are the fields where it is less well developed?

Data engineering and cloud integration is a big one. It has very few tools in that domain, and I say this as a heavy Julia user (hobby).

I've heard about cloud integration as an issue before, but what is "data engineering"?
working with databases, streaming data around with stuff like kafka or snowflake, integration with orchestrators like prefect or dagster, robust interfaces for spark, being able to read a directory of parquet files without the GC going insane, this kind of thing

I know there are some existing packages that nominally do some of these things, but generally they are understaffed and not fully mature. I love Julia as a language, so I hope this will improve over time. I think it's one of those problems that just requires more adoption before it can be fixed, and things like precompile TTFX improvements in 1.9 are a good way to get that!