[awaythrowact]

Congrats to the Julia team.

I am a python developer who has dabbled with Julia but it never stuck for me.

I think Julia was built by academics for other academics running innovative high performance computing tasks. It excels at the intersection of 1) big data, so speed is important, and 2) innovative code, so you can't just use someone else's C package. Indeed, Julia's biggest successful applications outside academica closely resemble an academic HPC project (eg Pumas). I think it will continue to have success in that niche. And that's not a small niche! Maybe it's enough to support a billion dollar company.

But most of us in industry are not in that niche. Most companies are not dealing with truly big data, on our scale, it is cheaper to expand the cluster than it is to rewrite everything in Julia. Most who ARE dealing with truly big data, do not need innovative code; basic summary statistics and logistic regression will be good enough, or maybe some cloud provider's prepackaged turn key neural nets scale out training system if they want to do something fancy.

I think for Julia to have an impact outside of academia (and academia-like things in industry) it will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so! The world would be better off with more cool data science packages.

But I think the sales pitch of "it's like Pandas and scikit but faster!" is not going to win many converts. So is Jax, Numba, Dask, Ray, Pachyderm, and the many other attempts within the Python community of scaling and speeding Python, that require much less work and expense on my part for the same outcome.

Again, congrats to the team, I will continue to follow Julia closely, and I'm excited to see what innovative capabilities come out of the Julia ecosystem for data scientists like me.

[jjoonathan]

> The next PyTorch needs to be written in Julia. Will that happen? Maybe! I hope so!

The Two Language Problem makes this more likely than one might think. Those high level python packages that plaster over python's many mediocrities have to be written and maintained by someone, and while extremistan bears the brunt of the pain and has done a remarkable job shielding mediocristan, it's extremistan that gets to decide which language to use for the next killer app.

Of course, python has more inertia than god, so it won't go quietly.

[stillwater919]

Your comment almost reads like a poem

[ssivark]

> do not need innovative code

I think this is a good deciding factor. Not just for “big data”. In my experience, with its combination of flexibility and raw speed, Julia makes implementing new algorithms (from scratch) a breezy experience. The strong research/academic presence in the community also helps towards encouraging decent Julia libraries for a lot of cutting edge work.

So if you are working in an area where that could make a significant difference, it’s an excellent reason to use Julia.

> will need to develop killer app packages. The next PyTorch needs to be written in Julia. Will that happen?

If enough cutting-edge work happens in Julia, it’s likely that a few great tools/platforms will emerge from that. We’re already seeing that in the scientific modeling ecosystem (as an example), with Differential Equations infrastructure and PumasAI.

[krastanov]

There is another important niche I am particularly excited about: programming language research geeks and lisp geeks. The pervasive multiple-dispatch in Julia provides such a beautiful way to architecture a complicated piece of code.

[vletal]

> The pervasive multiple-dispatch in Julia provides such a beautiful way to architecture a complicated piece of code.

On the other hand it makes function discovery almost impossible [1]. Combined with the way how 'using' keyword exports a predefined subset of functions, this makes the language doomed from larger adoption outside of academia at least as long as there is no superb autocompletion and IDE support.

[1] https://discourse.julialang.org/t/my-mental-load-using-julia...

[mccoyb]

Have you seen Shuhei Kadowaki's work on JET.jl (?)

If you're curious: https://github.com/aviatesk/JET.jl

This may seem more about performance (than IDE development) but Shuhei is one of the driving contributors behind developing the capabilities to use compiler capabilities for IDE integration -- and indeed JET.jl contains the kernel of a number of these capabilities.

[pjmlp]

Yep, that is my niche.

[ampdepolymerase]

Those two niches don't pay. They are effectively useless outside of the occasional evangelism on HN.

[pjmlp]

Who do you think puts money to action on LLVM, GCC, Swift, Rust, .NET (C#, F#, VB), Scala,....?

[UncleOxidant]

There's a Compiler and Runtime Engineer position listed at https://juliacomputing.com/jobs/ which presumably pays money.

[trenchgun]

Money does not create code, humans do.

[xiaodai]

Money don't do jobs, humans do.

You see the logical fallacy in ur argument now?

[systemvoltage]

Great summary. I've worked with scientists that love Julia and more power to them. As a software engineer, there are still rough edges in productionizing Julia (yes, I know there are a few examples of large scale production code). As soon as you take Julia out of notebooks and try to build moderately complex apps with it, you realize how much you miss Python. Having used Julia for last 4 years and having to maintain that code in production environment, I am strongly convinced that Julia has a niche but it is not going to be a contestant to Python/Java/C++ depending on the use case. Which really is a shame - I want one goddamn language to rule them all. I want that and tried to give a fair chance to Julia.

[amkkma]

> As soon as you take Julia out of notebooks and try to build moderately complex apps with it, you realize how much you miss Python

Why's that? What features or lack thereof of Julia contribute to that experience?

[systemvoltage]

There is too much to talk about and I’d want to give an objective impression with examples in a blog post, but one of the major grips I have is how little information Julia provides you with stack traces. Debugging production problems with absolutely zero clue of what/where the problem might be is one of the most frustrating aspects. I’ve spent so many hours debugging Julia using print statements. Debugger is janky, Juno/Atom support is not very good. Nothing feels polished and robust. Dependencies break all the time. We are stuck with Julia 1.2 and too much of an undertaking to go to latest version. Package management is an absolute disaster - this is the case with Python but Julia is worse. Julia Registry has many issues (compat errors). Testing and mocking is also underwhelming. I think startup times have improved but still not very developer-friendly. Sorry not an objective answer you’re looking for. There also things that Python has such as excellent web-dev ecosystem and system libs that are missing in Julia. Python has everything. Want to generate a QR code image? Easy in Python. Want to create PDF reports with custom ttf fonts? A breeze in Python.

[leephillips]

This is the first time I’ve heard somebody say that Julia’s package management is worse than Python’s! For most people who have spent years grappling with the succession of unofficial attempts to supply Python with something like package management, including virtual environments, etc., and the resulting dependency hell, Julia’s integrated and sophisticated package management (even with its own special REPL mode) is refreshing. I don’t doubt your experiences are real, but suspect you have just had really bad luck.

1.2 is pretty ancient. Current, or even recent, versions of Julia have a fraction of the startup time (https://lwn.net/Articles/856819/). Package management has been refined further, as well.

[systemvoltage]

I’m a tortured soul, my opinions might be biased and I hope things improve with Julia.

We absolutely cannot upgrade Julia version right now, dozens of repos full of complicated scientific code. Management doesn’t care as far as it barely runs. I don’t think it’s fair to blame Julia for it but it just shows how much more it needs to go. That should be looked at as a positive thing.

I have one more complain to Julia community - please don’t be too defensive. Accept and embrace criticisms and turn that into a propellant for improving the language. I’ve seen a cult-like behavior in Julia community that is borderline toxic. Sorry, it’s the truth and needs to be said. Speed isn’t everything and people are magnetized by the benchmarks on the Julia website, especially non-software engineers.

[amkkma]

>I’ve seen a cult-like behavior in Julia community that is borderline toxic. Sorry, it’s the truth and needs to be said. Speed isn’t everything and people are magnetized by the benchmarks on the Julia website, especially non-software engineers.

I think all languages have this dynamic...I've seen it with python and R. To some extent it's fed by what we perceive as criticisms from people defending their favorite incumbent language with arguments that aren't at all informed- such as a focus on speed and how numba achieves parity there.

In the same vein, I and many Julia users are enthusiastic precisely because of thing other than speed, such as the type system, abstractions, differentiability and other things that make programming more joyful, fluid and productive.

Agree though, that we could always improve on acceptance of criticism.

[leephillips]

Well you’re right about the libraries—Python really does have everything (although sometimes those libraries are not great quality, but at least they mostly work). But Julia makes it easy to use Python libraries with Pycall. And there are big things that just don’t exist yet in the Julia world, such as a web framework. I recently tried to create a web project using Genie.jl, which advertises itself as a ready-for-prime-time web framework for Julia, and I gave up after a few days. It’s just not in the same universe as something like Django, plus it’s barely documented.

[amkkma]

I really don't understand your point about package management.

Instead of Pip, virtualenv, conda, etc etc there's one package manager that resolves and installs native and binary dependencies, ensures reproducibility with human readable project files, is accessible from the REPL etc.

You can get an entire GPU based DL stack up and running on a windows machine in under 30 min, with a few keystrokes and no special containers or anything like that. Don't even have to install cuda toolkit. It's a dream, and I've heard the same from recent python converts

[awaythrowact]

I second most of this.

A lot of problems are fixable with time and money. Maybe the Series A will help!

But some problems might be related to Julia's design choices. One thing I really missed in Julia is Object Oriented Programming as a first class paradigm. (Yes I know you can cobble together an approximation with structs.)

OOP gets a lot of hate these days. Mostly deserved. But in some large complex projects it's absolutely the right abstraction. I've used OOP in several Python projects. Most of the big Python DS/ML packages use OOP.

Maybe you think PyTorch, SciKit, etc are all wrong, and would be better off with a more functional style. I know it's fashionable in some circles to make fun of the "model.fit(X,Y); model.predict(new_data)" style of programming but it works well for us busy professionals just trying to ship features.

I don't think Julia is wrong for enforcing a more functional style. It probably makes it easier for the compiler to generate great performance.

But Python has achieved its success because of its philosophy of being the "second best tool for every job" and that requires a more pragmatic, multiparadigm approach.

[amkkma]

How is "model.fit(X,Y)" better than "fit!(model,X,Y)"?

Julia is object oriented in a broad sense, it just uses multiple dispatch which is strictly more expressive than single dispatch, so doesn't make sense to have dot notation for calling methods because types don't own methods.

For giving up some facility in function discover, you get speed, composability, generic code...and a net gain in usability because you can have one array abstraction for GPUs, CPUs etc etc, which is just an instance of having common verbs across the ecosystem (enabled by MD). Instead of everyone having their own table type or stats package, you have Tables.jl or Statsbase.jl that packages can plug into and extend without the issues inherent in subclassing, monkeypatching etc.

This is a much better, more powerful and pleasant experience

Closing the gap in Method discovery will simply require a language feature with tooling integration, where you write the types and then tab to get the functions. There's already an open issue/PR for this

[east2west]

I don't miss OOP in Julia but I do feel there need to be more ways to abstract things than multiple dispatch and structs. One thing I do miss is interfaces, which can group functions for a common purpose. I understand it may be not feasible in a dynamic language, but hopefully there will be something above functions as an abstraction mechanism.

[awaythrowact]

I'll give you my two cents, recognizing that I very well might just be ignorant about Julia and multiple dispatch, and if so please continue to educate me.

Consider if we want to run many different types of models. Logistic regression, gradient boosting, NNs, etc. We want the ability to easily plug in any type of model into our existing code base. That's why model.fit(X,Y) is attractive. I just need to change "model = LogisticRegressionModel" to "model = GradientBoostingModel" and the rest of the code should still Just Work. This is a big part of SciKit's appeal.

But all these different models have very different training loops. So with "fit!(model,X,Y)" I need to make sure I am calling the compatible "fit" function that corresponds to my model type.

You might now say "Ah! Multiple dispatch handles this for you. The 'fit' function can detect the type of its 'model' argument and dispatch execution to the right training loop sub function." And I suppose that's theoretically correct. But in practice I think it's worse.

It should be the responsibility of the developer of "model" to select the "fit" algorithm appropriate for "model." (They don't have to implement it, but they do have to import the right one.) The developer of "fit" should not be responsible for handling every possible "model" type. You could have the developer of "model" override / extend the definition of "fit" but that opens up its own can of worms.

So is it possible to do the same thing with "fit!(model,X,Y)"? Yes of course it is. It's possible to do anything with any turing complete language. The point is, which system provides the best developer ergonomics via the right abstractions? I would argue, in many cases, including this one, it's useful to be able to bundle functions and state, even if that is in theory "less flexible" than pure functions, because sometimes programming is easier with less flexibility.

[chrispeel]

> One thing I really missed in Julia is Object Oriented Programming as a first class paradigm.

Julia peeps would tell you that the multiple dispatch used by Julia is a generalization of OOP. And that they like multiple dispatch

[cbkeller]

Yes indeed. While Julia does generally eschew the "model.fit(X,Y); model.predict(new_data)" style, it's not because it's functional, it's because it's dispatch-oriented, which is arguably a superset of class-based OO, and even perhaps arguably closer to Alan Kay's claimed original vision for OO than modern class-based OO is.

[xiaodai]

> one of the major grips I have is how little information Julia provides you with stack traces

as opposed to Python and JVM traces? LOL

Also, python has been around for alot longer. So perhaps that Python has "everything" is a feature of time and user base.

Maybe Julia will grow to have a large enough user-base to have those things.

[mbauman]

> We are stuck with Julia 1.2

Gosh, that's really old and it's not even an LTS version. I fear many of your woes stem from that. Julia 1.6 has huge improvements, and the community has rallied around VS Code now that Atom seems to be dying.

It really shouldn't be too bad to update.

[amkkma]

Thanks for sharing! I think much of it has to do with the version you're stuck on. (ie startup and compile times are way way better with 1.6)

Python having a broader ecosystem is a very good point, but I've found pycall to be very helpful

[habibur]

This matches with my experience with an old version of Julia too. But lets not focus on the negative aspects here. People have their different sets of priorities.

[snicker7]

What warts did you encounter? What language features / tooling would make Julia easier to productionize?

[queuebert]

> The next PyTorch needs to be written in Julia.

If a major company would pick up Flux.jl and fill out the ecosystem that would be AMAZING.

PyTorch and Tensorflow feel like duct tape and chewing gum all day, every day.