[cookieperson]

What do you mean how? Something in base changes, the author of a package didn't put strict enough compatibility on their package and now you can't use the package. It's especially great when this sort of thing happens after spending ten minutes waiting for precompilation an hour for your time to first gradient and three days for a run to complete and you trusted the tool to let you write the results to a CSV using the CSV package(one of the most widely used packages). Then you file a bug report and get gas lit, so you just patch your local Julia version and compile it yourself so you don't wait 4 months for an upstream fix. Good times.

[cjalmeida]

Counter anecdote, I had to go back to a quite complex project that a was developed in 1.5, 3 years ago, hundreds of dependencies.

Upgrading to 1.9 RC2 required changing a single dep (Light raps to the drop-in Graphs) and a single line (a library changed the return from a String to a StringView)

[cookieperson]

You're incredibly lucky. Even minor version bumps have left me up shit creek in production with deadlines.

[cjalmeida]

Julia core is backwards compatible all the way to 1.0, libs not so much.

Now, something as fundamental as numpy dropped support for Python<3.6 in a minor release: https://numpy.org/devdocs/release/1.19.0-notes.html

And I've been bitten by relevant python libs breaking stuff even in patch releases.

It happens and it's far from particular to Julia. I just make sure to test stuff property before going to production.

[cookieperson]

Pythons breakages have also been security centric. I'm not a big fan of it, but yea Julia isn't 30 yrs old and it's done things like this every minor release without anouncement. To be clear though I'm not talking about a conscious researchable decision to drop support after 5 years. I'm talking about bugs that pop up over night. Anyway has Julia ever been audited for security? Most of the netstack was written by a single person... Makes ya wonder...

[freilanzer]

Why do you update a programming language version in production with deadlines in the first place?

[cookieperson]

Because it has a new feature that's save the day(if it worked), fixes bugs in a janky patched Julia version, and production in the data science world looks different in research and development phases then it does for software. Production in r and d can simply be, boss wants to see the pros and cons next week with a successful run. Only with Julia have I run into these kinds of predicaments

[dunefox]

Other languages and libraries break as well so that's just bad practice.

[cookieperson]

Not as frequently. Having a broken release for a programming language is also bad practice.

[Sukera]

Can you be more specific? As is, it's hard to follow what the issue was that you encountered.

[cookieperson]

At this point the only thing I can say is. Have you ever been a Julia user? Or are you actively developing Julia? If you are actively developing Julia, yea sure pull down the feature branch after someone's reported a bug, hot fix your toml to point too some specific version after reading ten diffs to be sure you have the right one, maybe stand up your own package server or pay 50k for one from julia computing, recompile Julia itself and everything is totally fine! If it's not no big deal just stomp over the type with your own code, yay. Maybe that's even part of your job. If you're a user expecting to do development and receive updates on packages or language, all I'll say is best of luck.

[amval]

I run in production several Julia projects and I honestly can't say I know what you are referring to.

I do use some libraries through PyCall. This is a relatively frequent source of trouble, with libraries breaking because of 3rd party dependencies, behaviour changing, or even somehow installation suddenly stops working. This all happens in the python side, to the point that I am currently replacing all the libraries even if it means porting some of them or even a loss of functionality.

[ChrisRackauckas]

You can use CondaPkg.jl (https://github.com/cjdoris/CondaPkg.jl) to setup Python dependencies with version control. I haven't played with it too much but it seemed to work out for what I tried. Indeed the reason why I haven't had too many test cases is because in SciML we removed all Python dependencies since they were the main source of instability. PyDSTool.jl, FEniCS.jl, SymEngine.jl (through ParameterizedFunctions.jl and ModelingToolkit.jl) were the biggest development burdens because of the Python deps changing function names all of the time until they were deprecated and replaced by pure Julia packages, so I definitely know the pain.

[cookieperson]

Is it fair to say that it is your career to rewrite code in Julia to write academic papers about julia?

[ChrisRackauckas]

Most of my time isn't academic or writing code. Most of it is focused on enhancing algorithms to achieve what's required for new methods to work deployed in production environments such as the Pumas clinical pharmacology work, and now working on what's required to get scientific machine learning point-and-click GUI ready. I always leave a little bit of time in mornings and weekends though for some coding, and a little bit a day goes a long way after years.

I stay away from code rewrites like the plague, which is why SciML maintains so many wrapper packages. It's less work and if there's nothing new then rewriting is thankless work. Unless we have some kind of angle for something new in the algorithm, we just recommend someone use the same old wrappers (for instance, IDA with fully-implicit DAEs). We won't beat the C/Fortran code without something new. However in most areas there's lots of interesting research to be done. I could "sabbatical as an academic" and spend the next 3 years just improving explicit Runge-Kutta methods: those for example are probably still 4x away or so from what I think is theoretically possible, but that's not going to happen any time soon since we have some real applications to focus on.

Those package deprecations mentioned above were deprecated by community efforts (BifurcationKit.jl replaced PyDSTool wrappers and was done by Romain Veltz, FEniCS wrappers were largely replaced by Gridap, and SymEngine was replaced by Symbolics which is largely the work of one of the Julia Lab PhD students Shashi). Even those had a largely new element to them though, with BifurcationKit focusing a lot on recent algorithms that other bifurcation software don't have (like the deflated Newton methods), and Shashi's Symbolics.jl work focusing on generality of term rewriting systems to allow for alternative algebras.

[cookieperson]

So much for solving the 2 language problem. Everytime I ended up in a similar situation the Julia code was rewritten either by me or someone else in c++ for practical/performance reasons. Hope you consider the same. The libraries for doing mathematics in C++ are surprisingly good and you might be surprised to find how easy parallel processing and fine tuning is in c++. In some ways it's less cognitive load than Julia because it's more explicit...

[amval]

My problem is not with Julia, my problem is with the Python ecosystem and its dependency management. Not sure how you can read what I wrote and reach that conclusion.

[kukkukb]

We run Julia code in production. These are compute-heavy services called by other parts of the app. Version upgrades are normally easy. Upgrade, run the unit tests, build the Docker images and deploy to K8s. Maybe we've been lucky, but since 1.3 we've never had big issues with version upgrades.

[cookieperson]

That's wild, I've seen huge performance regressions from changes to Base, among commands being dropped, irreconcilable breakages with basics of the ecosystem, spurious bugs in the networking libraries that are nearly impossible to chase down, and the menagerie of compatibility with essential packages break. I stopped caring around 1.7 when I realized this was the norm and it wasn't going to change. You must be honed in on some specific packages with a lot of custom code.