I think this depends on what part of the statistics universe you're working in.
For example, within Lisp-Stat the statistics routines [1] were written by an econometrician working for the Austrian government (Julia folks might know him - Tamas Papp). It would not be exaggerating to say his job depending on it. These are state of the art, high performance algorithms, equal to anything available in R or Python. So, if you're doing econometrics, or something related, everything you need is already there in the tin.
For machine learning, there's CLML [2], developed by NTT. This is the largest telco in Japan, equivalent to ATT in the USA. As well, there is MGL [3], used to win the Higgs Boson challenge a few years back. Both actively maintained.
For linear algebra, MagicCL was mention elsewhere in the thread. My favourite is MGL-MAT [4], also by the author of MGL. This supports both BLAS and CUBLAS (CUDA for GPUs) for solutions.
Finally, there's the XLISP-STAT archive [5]. Prior to Luke Tierney, the author of XLISP-Stat joining the core R team, XLISP-STAT was the dominate statistical computing platform. There's heaps of stuff in the archive, most at least as good as what's in base R, that could be ported to Lisp-Stat.
Common Lisp is a viable platform for statistics and machine learning. It isn't (yet) quite as well organised as R or Python, but it's all there.
what do you mean a one-off project? some of these have corporate/institutional backing but thats not the point. if you know how to implement machine learning models then there should be nothing (except maybe availability of pretrained models) preventing you from using common lisp. if you dont want to dig deep and are happy using just the front end (perfectly acceptable) then python is a much better bet
all the needed ingredients are there to build a custom production class ml solution in common lisp: vectorization, cuda, blas, lapack
Do any of those libraries linked above provide an example of training a resnet or bert on a GPU? I’m not looking for anything fancy, or production quality, just a basic example.
the rally call of lisps is to get a lot done with less, so it is definitely possible that it can compete in some sense. also common lisp packages such as these can cater to the programmer-first type of data scientist by allowing greater (interactive) tinkering with the underlying code
>> cater to the programmer-first type of data scientist
Exactly. Especially those who want to dabble with lisp by playing with familiar problems and applications. I find it much more instructive to play with linear regression code in a new language than with, say, a game development engine, because I have a pretty good idea about the strengths and quirks of `lm` in R.
So I can easily imagine packages like this becoming widely adopted /within/ the Lisp community.