[dnautics]

what's the problem with deploying julia in production in inference? Some occasional piece of data that looks wrong in an unanticipated way causes a runtime type fault? People deploy high uptime websites with django - how do they do it? Well you use kubernetes (or, gasp, systemd) and have restart and load balancing logic. Even if you were typecheck-compiled, you can't guarantee some other developer logic or system error, or an errant bit flip from a cosmic ray, won't take your setup down. Static checker doesn't really matter. If you're at the point where you're ready to deploy, you're probably good for at least 95-99% of the data you'll ingest. The rest of the gap can be closed using rolling update.

[shele]

On the other hand, Julia can do the right thing dynamically. Your matrix happens to be symmetric? Julia will chose an appropriate factorisation and will propagate that knowledge through dynamic dispatch.

[skohan]

"Automagical" solutions tend to work until they don't. I would much rather have my solution be provably correct at compilation time than to depend on high-overhead runtime systems.

[dnautics]

You don't know what you're talking about.

[skohan]

Tell me where I'm wrong

[ychen306]

You are not wrong. For some "highly dynamic" applications, say, optimizing compiler IR where there are many different subclasses of IR nodes, dynamic dispatch is nice. But when you are running an ML model, scientific application where you already know which sparse matrix format you need, etc, you can do all of that statically with less overhead and performance predictability. This is in no way an argument against Julia; the point is that you don't need dynamic dispatch if you can statically determine what needs to happen.

[KenoFischer]

That is basically the core insight behind julia. The really performance sensitive parts of your application are already static just by the nature of that code, so we can extract that static information to make it really fast. We can also make use of the same static information for static error messages or static compilation and get the best of both world (dynamic during development, static when you're done), but the tooling for that is a bit less developed at the moment.

[dnautics]

Julia can and does make 'provably correct' decisions at compile time, it's just that at the the default typesystem settings are not quite correct for machine learning apps.

Also it's not a high overhead runtime. The runtime itself is compiled to highly optimized machine code (it can even compile, say the derivative of f(x) = 5x+3 down to the machine immediate "5" at compile time).

There is a lot of lifting to get that compilation framework into place, so there is a load-time overhead.

[ychen306]

Which you don't need most of the time in production because you can statically determine which operation needs to happen without the overhead of dynamically choosing the operation.

[skohan]

Just because you can deploy production code without static checking does not make it a good idea. And to me if you need to rely on an external process to restart your application for NPE's etc. it is a sign your application is not that robust.

It's possible to write production code in brainfuck if that's what you really want to do. Statically checked code is easier to implement correctly, easier to modify, and easier to maintain.

[dnautics]

> Statically checked code is easier to implement correctly, easier to modify, and easier to maintain.

If that were true literally no one would program in python. Static checking is not the end all to uptime and stability. I wrote an elixir program in three days that served as a testbench for a senior's go program (which took him six months to write). This senior believes in static typechecking for everything and doesn't write unit tests. The testbench handles thousands of parallel async requests without a hiccup and even survives operating system resource exhaustion, where the go program falls over and panics.

Erlang is not statically checked (there is a static typechecker, but it's not fully typed). I promise you a well written erlang program has much higher stability than a well written go program. There is a reason why kubernetes exists, after all.

[skohan]

> If that were true literally no one would program in python.

There are many cases in which people don't choose the optimal language. But I would say the size of the python community has more to do with inertia, the breadth of libraries available, and a relatively shallow learning curve than it says about its strengths as a tool for writing good software.

I actually find that Python has some rather serious warts: the whole story around environment/version management is a mess, and the less I have to work with Python in a serious capacity the better.

> I wrote an elixir program ... The testbench handles thousands of parallel async requests without a hiccup and even survives operating system resource exhaustion, where the go program falls over and panics.

Well Erlang is specifically designed for concurrency and stability: if you want to judge your result on those two metrics I hope it is going to perform well.

I never made the claim that static typing is the "end all to uptime and stability" - static typing makes it easier to reason about your code, and to provably eliminate many issues. It's very nice that you implemented a test harness quickly, but come back to me when you've worked on a complex codebase with several other people over an extended period of time.

[ychen306]

The problem comes down to how you use your model, which is all that matters. If there is a mismatch between the language in which your model is embedded in and the language in which your main application/interface is written in, then ideally you want a way to extract the model statically from the dev environment. It's not about type-checking per se, which is nice, but how you get the model out without having to, say, start a python interpreter in your server process just to do the inference (which is why in some aspect Tensorflow is more convenient).