As a huge Lisp/Scheme fan, I don't find it surprising that Python became the language of choice for neural networks over Lisp.
NN in general just boil down to doing lots of linear algebra, which is a lot about mutating very large matrices. For this you really just want a wrapper around BLAS/LAPACK, so you can leverage existing optimized libraries.
Working with matrices in Lisp always feels a bit clunky, and doubly so when you want to do a lot of stateful operations on them. The real compitetor in this space to Python is something like Matlab, which has probably the best interface for doing linear algebra (but is worse at everything else).
The one area where Lisp and Python both shine is the ability to perform automatic differentiation. Lisps are great for this task since it's all symbolic manipulation. However, this is only important once you have a solid interface for working with matrices.
If you want to get a feel for the difference I would suggest reading through the surprisingly excellent The Little Learner. I think you'll find that while it really demonstrates the power of Scheme (Racket in this case) in areas where it excels, you wouldn't want to use the framework in that book for anything other than toy examples.
> But is there is something inherent to Lisp that makes it "clunky" to work with matrices?
s-exps are basically coding at the level of the AST, which for many programming tasks is a powerful level of both abstraction and control.
But this interface tends to work much better on tree-like structures (the most simple of which is the list, which in it's simplest case is a cons cell, Lisp's most fundamental particle).
Most programming tasks can boil down to tree manipulation, but large matrices feel a bit out of place with this interface because you are basically to working with arrays (which in it's simplest cases is a pointer to a memory address, which is C's fundamental particle). C like languages (I know, technical ALGO-like), tend to feel more natural for these tasks.
But of course if you can manipulate trees easily, you can manipulate code easily which gives you...
> Lisp can be at least as high level and expressive as any other language.
In practice, for Lisp this is also a weakness. Again, I love Lisps in general, and of course any Lisp and especially Common Lisp (with it's exceptionally excellent macro system) can do anything you want.
However, my (and many others) experience has been that this leads to is it becomes very easy to create code that is amazing for the original developer, but very difficult for new developers to get their head around. Hence the adage "Lisp is optimal for team sizes of one."
You could undoubtable build the worlds most elegant interface to working with matrices in Lisp, but now you essentially have a new language.
I think matlab has the most succint language for writing matricies
[1 2 3; 4 5 6; 7 8 9]
then lisp
#2A((1 2 3) (3 4 5) (7 8 9))
then python comma freak show
[[1,2,3],[4,5,6],[7,8,9]]
python's greatest achievement was bringing open source to matrix calculations and dethroning matlab. python is what it is today because it offered 99% of what matlab offered in open source and for $0 to uni students
I was curious about how Julia does 2D matrices and arrays, given they target mathemeticians. They seem to have no shortage of methods of creating them.
octave was always seen as subpar version of matlab and r is a statistics programming language. in comparison python was already back then a proper practical programming language that people were using for things other than math
Lisps have n-dimensional arrays since decades. Often their implementations are a bit less efficient than the ones found written in C and used by Python.
> You could undoubtable build the worlds most elegant interface to working with matrices in Lisp, but now you essentially have a new language.
My understanding is that enough time had passed between the AI winter of the late 1980s and the rise of big data and deep neural networks in the 2010s to where a relatively tiny minority of modern machine learning practitioners came from the old-school AI community, which tended to prefer Lisp or Prolog. When machine learning became hot in the past 15 years or so, many new entrants to ML who had no previous AI knowledge used the tools they’re familiar with, which are generally C, C++, Java, JavaScript, and Python. In addition, many in the natural sciences use Fortran and Matlab. Perhaps had the AI winter of the late 1980s not happened (which took out the Lisp machine market) and had Common Lisp not been relegated to certain niches, perhaps there would’ve been more usage of the language, which may have led to Python being less dominant today.
I love Lisp and Smalltalk, but I work as a machine learning researcher as my day job, and I code in Python at work. Admittedly I miss Common Lisp whenever I code in Python, and I think Jupyter notebooks are a poor man’s Smalltalk environment, but when comparing Python to C and C++ (and I used to work primarily in systems programming, so I know my way around C), Python is a major productivity boost.
well, if we categorize python and js as non-lisp languages, i'd say it's mostly because python has been the language of choice at google since last millennium, at least for situations where flexibility outweighs performance
last millennium when google started up, no lisp was a good option. clojure didn't exist, schemes were mostly for teaching, cmucl was in disrepair, and though there were proprietary common lisps, they were proprietary and wanted per-cpu license fees, which would have meant disclosing how many cpus google had—a closely guarded secret
moreover, common lisp's approach of putting a uniform but somewhat repulsive veneer over all operating systems was a pure drawback in google's all-linux environment, and s-expression syntax is harder to read than python's (if easier to edit)
also the python community was friendly and welcoming, while the scheme community was tiny and fragmented, and the common lisp community had degenerated into an ego trip for a world-class asshole named erik naggum
this is guessing from the outside; i've never worked at google, which is why i'm allowed to write this
I like that you're framing this as if it was between python and lisp and not python being the language of choice by a huge margin followed by a dozen other languages, then lisp.
Maybe when Common LISP gets its next standard and stops being a geezer on the deathbed ? The world has changed since 1994. So has the computing landscape. If a programming language is static for more than a human generation, then it cannot expect any sort of adoption.
It's unlikely the ANSI standard will ever get updated - it's a very expensive process.
Python doesn't have a standard, but it has a canonical implementation, so maybe a "defacto standard". Not having an official standard didn't matter, though.
Common Lisp has defacto "standard libraries" for things like threading. Other things, such as async, are less clear (to me), but libraries do exist. Just because something isn't in the standard, though, doesn't mean it can't exist. Also, implementation such as SBCL pick up a lot of the slack, filling in some of the missing gaps.
Python evolves through PEP's https://peps.python.org. Can you kindly correct my deplorable ignorance and point me to the Common LISP enhancement proposals ? Maybe I am missing something elementary.
And? What has changed so much since 2013 that would require new proposals? Do you need the language to change every month? You keep moving goalposts, and I'm beginning to think that you’re not arguing in good faith here.
Really ? We are simply going to ignore the entire Python web-app landscape ? Django, Flask, Pyramid, etc. The deployments of those outnumber CL deployments by so many several orders of magnitude. No, ML didn't "save" Python. It only accelerated its usage further above all competitors.
Django and other web apps pale in comparison to Python's dominance in numerical computing. Im not saying they are not important, in fact I believe Python's use in areas other than math drove its adoption in the scientific community, and that one growth fed the other, but without ML there are far too many competitors to Python. However, almost everyone agrees thag py2->py3 was a mess. I like that when I run CL code written 10/20 years ago, and even older!, I dont have to worry too much if it is going to run today
NN in general just boil down to doing lots of linear algebra, which is a lot about mutating very large matrices. For this you really just want a wrapper around BLAS/LAPACK, so you can leverage existing optimized libraries.
Working with matrices in Lisp always feels a bit clunky, and doubly so when you want to do a lot of stateful operations on them. The real compitetor in this space to Python is something like Matlab, which has probably the best interface for doing linear algebra (but is worse at everything else).
The one area where Lisp and Python both shine is the ability to perform automatic differentiation. Lisps are great for this task since it's all symbolic manipulation. However, this is only important once you have a solid interface for working with matrices.
If you want to get a feel for the difference I would suggest reading through the surprisingly excellent The Little Learner. I think you'll find that while it really demonstrates the power of Scheme (Racket in this case) in areas where it excels, you wouldn't want to use the framework in that book for anything other than toy examples.