Hacker News new | ask | show | jobs
Differentiable programming for gradient-based machine learning (forums.swift.org)
34 points by undefinednull 2040 days ago
3 comments

Has anyone got experience in using other things than python for machine learning? What are the rivals out there?
Particularly outside deep learning, R remains popular especially for those closer to the stats world.

Matlab and Mathematica both have interesting machine learning features but are probably orders of magnitude lower usage levels compared to Python.

In my experience: Julia, Java, C++. Lack of libraries in a pain; OTOH the quality of most libraries (in general) tends to be poor.

Examples of library issues: there is C++ support in tensor flow, but not for training.

For anything involving differential equations, Julia is way better than anything else. I also wouldn't be at all surprised if Julia catches up to python for way more machine learning within the next year
Agree to that completely. It is also hard for the libraries to spend their energy / time in building for languages that covers only a few percentage of total users. :(
Julia doesn't lack libraries and the quality imo is pretty good. Did you look at PyCall.jl?
Interesting! I did not yet! will take a look. I remember swift was working on something similar to seamlessly call Python.
There is a rather active community for Go in data science and machine learning. +1700 members on the #data-science channel on the Gopher slack, a few books on the topic, and a few libraries:

Books

Machine learning with Go - https://www.amazon.com/Machine-Learning-Classification-Time-...

Go Machine Learning Projects - https://www.amazon.com/Go-Machine-Learning-Projects-Ultimate...

Libraries

https://gorgonia.org/

https://www.gonum.org/

A good talk (although from a from years back), explaining why Go for ML could be a good idea:

https://www.youtube.com/watch?v=D5tDubyXLrQ

I'm using Julia instead of Python for NLP and it's great. If I need a python library I just use PyCall and so far it has worked flawlessly.
Can you say a bit about the advantages of each? I'm just starting to learn deep learning and planning to use it for NLP. I'm taking the python-based fast.ai course. But I'm tempted by the elegance of Julia.
If you're starting you should use Python and when you have a more solid grasp you can switch to Julia. That's mostly due to the fact that there are much more tutorials and resources for dl in python.
Is anyone still excited about differentiable programming? At least in NLP it seems like a lot of the energy has shifted to large scale overparameterized models like BERT, e.g. you don't need arbitrary control flow in your model, all you need is attention.
The differentiable programming framing is quite useful in the scientific simulation space, because it lets you couple traditional physics simulations to either neural surrogates or some sort of bayesian model. Enough of it is happening in the Julia world that were very actively improving the compiler to support it.
Here's a stunning recent example from computer graphics: http://rgl.epfl.ch/publications/NimierDavidVicini2019Mitsuba...
Attention is cool but not to the level of higher-order cognitive architecture. Further, there are many uses of differentiable programming besides AI
This talk by Eric Meijer got me intrigued of the subject ... (Based on this talk, I think I almost understood it for an hour or two):

"The principles behind Differentiable Programming - Erik Meijer" https://www.youtube.com/watch?v=lk0PhtSHE38