[mark_l_watson]

Wow, that is great, thanks for putting it together! I have been using CL since the mid 1980s, but cumulatively have just used the language for perhaps five years.

While I find myself using Python a lot because it wraps so many machine learning libraries and Java because of the ecosystem, there are other languages like CL, Ruby, and Haskell that all just make me feel good when I use them.

[aaron-lebo]

I really wish I could use CL more. It's just that in almost every domain, the new libraries are elsewhere. You could use lisp for scientific computing, for example, but it'd be crazy to do so because you'd be reinventing so much that's getting built elsewhere. Similarly with games, C++ just has a lot more libraries available (you could FFI I guess, but that's never as bulletproof).

Not sure lisp is better at any one thing than being a lisp, which is great but I want to be able to justify reaching for it. 10 years ago PG had me believing that there'd be a lisp revival one day, but despite getting more and more polished, there's so much going on in other ecosystems.

Sorry for the rant, I know the lisp community has been building great stuff for decades, lisp is great.

[grault]

You don't have to reinvent a thing, just use some ffi (foreign function interface) implementation and call into whatever you want.

[flavio81]

To add to this reply, interfacing in C with Common Lisp is pretty easy today. There are at least two portable libraries to do this.

Also, you can also interface with Java libraries on the JVM by using the ABCL lisp implementation.

[vram22]

>There are at least two portable libraries to do this.

What are they?

[flavio81]

One is CFFI, widely known and well documented, with tutorial and tooling:

https://common-lisp.net/project/cffi/

The other was UFFI, older; but then there are others. In fact, there are a ton of other projects for doing FFI in Common Lisp!

http://www.cliki.net/FFI

And then you can also use the FFI functionalities provided by the particular Lisp implementation (i.e. SBCL, LispWorks, ABCL, etc.)

However, the use of a portable library like CFFI means that you can take your code that runs correctly in SBCL, and then run the very same code in CLISP (other Lisp implementation) with no change at all.

CFFI works for the following Lisp implementations or "compilers": ABCL, Allegro CL, Clasp, CLISP, Clozure CL, CMUCL, Corman CL, ECL, GCL, LispWorks, MCL, SBCL and the Scieneer CL.

That's a lot of implementations!

[vram22]

Sure is a lot. Thanks!

[e12e]

I'd guess one is cffi:

https://www.quicklisp.org/beta/UNOFFICIAL/docs/cffi/doc/Intr...

Maybe with the help of cl-autowrap:

https://github.com/rpav/cl-autowrap/blob/master/README.md

There are others... :

http://www.cliki.net/FFI

[vram22]

Interestingly there is a CFFI module in Python too. Maybe they got the idea from Lisp.

[guiomie]

Are there performance implications using FFI ?

[dreamcompiler]

There can be, but they mostly have to do with moving things around in memory and garbage collection. An FFI has to marshall in-memory data structures from Lisp into the form C expects, and then move C's results back into Lisp's world. Sometimes your program will have to do this manually but the FFI usually takes care of simple cases like pointers and atomic types. Still, it's not cost-free because at minimum the removal and insertion of tags has to happen.

A related problem is word alignment; getting this right is important for passing data from Lisp to vector processors (and to a GPU I presume, but I haven't done that).

The other issue is GC: When the C function is running, it's important that Lisp's garbage collector not move the memory C is using. Some Lisp implementations do the laziest thing and just stop Lisp until C returns. Others are more sophisticated and keep C's heap separate -- but that requires more copying. Still others mark the memory C uses in a way that tells the GC "don't touch" but that can lead to pathological fragmentation in extreme cases.

The above makes it sound worse than it is. 99% of my use of a Lisp FFI has resulted in a performance increase because C is in general about twice as fast as CL for low-level stuff--which is one of the reasons to use an FFI in the first place.

[flavio81]

> You could use lisp for scientific computing, for example, but it'd be crazy to do so because you'd be reinventing so much that's getting built elsewhere.

However, it has the potential to be just great for scientific computing. And it has been used for scientific computing extensively in the past.

First because of the fully interactive experience. Even a running program can be modified while running, in an easy and safe way.

Second, because of Lisp supporting easily, with no use of special libraries or special operators, the following: Arbitrary length numbers, complex numbers, fractions, integers, and arbitrary long integers. You work with them with all the built-in operators (+,-,/, etc) and all the built in functions like sine, cosine, etc. They all work correctly with the provided datatype.

Let me quote a part of the "History of Lisp" PDF by Gabriel:

---------------- QUOTE ------------------------

The S-1 was initially intended to be a fast signal processor. (...) Infuenced by S-1 Lisp, Common Lisp provides an expanded system of floating-point data types to accommodate such architectural variation. The inclusion of complex numbers in Common Lisp was also an inheritance from the S-1. This was something of a sticking point with Scott Fahlman. A running joke was an acceptance test for nascent Common Lisp implementations developed by Steele. It was in three parts.

First you type T; if it responds T, it passes part 1.

Second, you define the factorial function and then calculate

    (/ (factorial 1000) (factorial 999))

If it responds 1000, it passes part 2.

Third, you try

    (atanh -2) 

If it returns a complex number, it passes; extra credit if it returns the correct complex number. (...) Gerald Sussman and his students (including Gerald Roylance and Matthew Halfant) became interested in numerical applications and in the use of Lisp to generate and transform numerical programs [Sussman, 1988; Roylance, 1988]. Sussman also spent a fair amount of time at MIT teaching Lisp to undergraduates. Sussman thought it was absolutely crazy to have to tell students that the quotient of 10.0 and 4.0 was 2.5 but the quotient of 10 and 4 was 2. Of course, nearly all other programming languages have the same problem (Pascal [Jensen, 1974] and its derivatives being notable exceptions), but that is no excuse; Lisp aspires to better things, and centuries of mathematical precedent should outweigh the few decades of temporary aberration in the field of computers. At Sussman's urging, the / function was defined to return rationals when necessary, so

    (/ 10 4) 

in Common Lisp produces 5/2. (This was not considered a radical change to the language. Rational numbers were already in use in symbolic algebra systems. The developers of Common Lisp were simply integrating into the language functionality frequently required by their clients, anyway.)

-----------------------------------------------

I think any programmer that has had to work with huge numbers, complex numbers, floats and integers in C or C++ will appreciate the above information.

> Similarly with games, C++ just has a lot more libraries available (you could FFI I guess, but that's never as bulletproof).

True, however it can be done in Lisp as well. Some commercial games were written in the past in Lisp. Some famous and frankly groundbreaking games of the past, like King's Quest, Space Quest, Police Quest and Leisure Suit Larry, were written in an special interpreter (AGI) which was basically... a minimal Lisp.

Currently, on the Lisp reddit there is a long series of tutorials on this topic, called "Pushing Pixels with Lisp".

[flavio81]

I have used Python, Java and C# in commercial projects for years, however i've started in CL some months ago and from my experience, it is much more suitable to general purpose development, including, definitively, commercial software development.

There have been voices that lament the supposed lack of libraries in CL, however i've found libraries for most of the things I would need anyways on a project, like ORMs, database access, compression, encryption, communication, etc.

There are also other voices that are scared by the lack of static typing, however CL is very strongly typed (it will strongly enforce types at runtime), and popular compilers like SBCL will accept type declarations that will also do many static-time type checks, plus these declarations will also greatly speed up the code.