The author almost immediately writes off Common Lisp for lack of "frictionless access to a rich ecosystem of code written in the same language as your software", then recommends Clojure. Most of the ecosystem you have access to in Clojure is, in fact, not built in Clojure, and most Clojure libraries were/are wrappers around Java or Javascript.
Footnote 1 makes no sense, suggesting that the only way to get access to "a bunch of other useful code" is to embed it into a C program and that understanding your dependencies is somehow easier in Javascript.
In reality, Quicklisp[1] offers effortless access to over 1200 libraries and programs. It's nowhere near the 200 thousand packages in npm, but the overall quality is good and these libraries cover a surprisingly large number of things.
The post tries to display Clojure as "Lisp, but with access to open source". Clojure is a fine programming language, but it's also completely different from Common Lisp. CL has high quality optimizing compilers performing extensive type checking, a powerful object system, easy access to C libraries, etc. If you need Java interop, you can just use ABCL[2].
Seeing Common Lisp fans arguing against Clojure fans is funny since your market share combined is insignificant compared with the mainstream choices. You should learn from each other and grow that pie together.
I also don't agree with you. Intuitively Lisp might be a secret weapon for startups, but not for open source where the communities of the mainstream choices are more productive, simply because of popularity. Java is a huge ecosystem of working and mature solutions. If you don't depend on Java, you depend on C. And for the kind of problems Lisp is suited for, I'd rather take Java. You're also missing the point on JavaScript. Say what you will about the quality of npm packages, but ClojureScript can reach people that your favorite CLisp cannot, due to the browser.
> Intuitively Lisp might be a secret weapon for startups, but not for open source
> where the communities of the mainstream choices are more productive, simply because
> of popularity
Open source is not a popularity contest. In fact, open source, and specially free and libre software, is a great place for leftfield technologies to flourish. In fact, Lisp had a place in free software before Github even existed, with various flavours being the scripting language of choice in popular projects like Emacs, Gimp or Gnome.
It is rather in the proprietary corporate world where there is often resistance to adopt languages outside of the mainstream. The more supply of developers for a technology is available, the lower cost of labour. It's hard for the merits of a technology to argue against this fact in the face of management.
That 200k figure isn't very comparable to other languages either as there are a lot of pacakages that just do a tiny thing and a hell if a lot of overlap in packages.
While that's got upsides I must admit I prefer one or two relatively complete and stable packages over way more numerically but leas stable complete packages.
But I come from a desktop background so I'm spoilt in that regard I guess.
I view every additional depency as a weakness a year from now.
I love Common Lisp, and as you say, there are many outstanding libraries that Quicklisp gives you easy access to. ABCL hasn't been "marketed" enough to the Java community, so it has less mind share. When I last looked at ABCL the Java interop was not as clean as Clojure's, so while it was there it felt more clunky. This was at least a year ago, so things are probably better.
I use Clojure to implement a portion of a semantic search engine: it is packaged as a REST servlet that responds to requests from a medical POC application. Because of the Java interop I utilize our internal libraries and from the ops perspective they are just deploying a WAR file that plays nicely in their reporting infrastructure.
I've been using Clojure since its first release and am comfortable with it and its ecosystem. ABCL is a contender, but I have yet to do a performance comparison, both for native and interop tests. And in reality the only part of Common Lisp I really miss is the condition system. Clojure as an implementation of CL's format and I was never a big user of loop, so...
Good point, thanks! I've removed the footnote, and I'm going to rewrite that paragraph to steer away from Common Lisp a bit.
My point wasn't Clojure > Common Lisp. I was simply anticipating the "but npm has 200k modules" argument against Lisps in general and I wanted to illustrate why it doesn't matter if you're using Clojure.
It does seem like a somewhat valid case against Common Lisp, but I don't care and it detracts from my main point.
Agreed, I wish people stop selling the JVM/JS libs as the solution to all Clojure library needs, is not really smooth sailing as most paint it out to be.
If you're asking yourself what kinds of problems is a Lisp good for you should probably watch this: https://www.youtube.com/watch?v=8X69_42Mj-g (A computational chemist built himself his own Common Lisp implementation because no other language available was powerful enough for his needs...)
It basically boils to: if you're solving a truly new and interesting problems for which the current libraries and ecosystems don't matter that much to you anyhow (because what you do is too bleeding-edge / ahead of everyone else, so you'll write your own better stuff anyhow), than you might want to choose a Lisp.
And if your problem is so bleeding-edge and exotic that you also need to build your own programming-language for it, than you might just as well build yourself a Lisp (like the guy in the video did) and add your needed features to it, because this would be easier than any other approaches and allow you to spend more time on your problem and not on the language...
I can't believe an article about Lisp left out QuickLisp. It's the most popular Common Lisp package manager and has over 1200 open source libraries in it, and most of them are written in Common Lisp, or are bindings to native C/C++ libraries. Using libraries written in the same language you're using has a lot of advantages. Using Java libraries from a Lisp seems clunky.
Also, judging solely by the number of available packages is not a good measurement. Browse through NPM, and it's pretty obvious there are a LOT of duplicates, and that people have published libraries for the silliest little things. Search for "PNG", for example, and in the first page of results there are two different png diff libraries, a few libraries for determining if a buffer contains a PNG byte-stream, multiple stream and un-stream libraries, etc. That's all done with a single library in CL.
And finally, I don't think Clojure is that great. It's better than Java, and it's what I'd use if I had a lot of legacy Java code to work with, but it wouldn't be my first choice for a new project written in Lisp. I would hazard a guess that it's popular because it's less clunky than Java, without the over the top type system of Scala.
You might want to check out Kawa: http://www/gnu.org/software/kawa .
It's mature (since 1996); runs on the JVM; compatible with Scheme standards (basically full R7RS except for continuations); great Java/JVM interop; trivial and flexible deployment (just a jar, unlike Clojure which assumes its own eco-system); extremely fast start-up (easily beats ABCL and Clojure); good compile-time and run-type error-reporting; and best-of-class runtime speed (comparable to Java/Scala with appropriate use of optional type specifiers).
This seems like a little bit (probably a lot) of hyperbole around clojure.
While I love clojure, selling it as a "secret sauce" you just sprinkle on your startup and it succeeds is definitely not the message I think the community should be trying to send.
Lisps have benefits, they're pretty well documented (just the fact that lisp is multi-paradigm in and of itself is a large benefit), but they're not infallible. If you give macros to the wrong programmer, you will slow your entire team down.
The author seems to not know that there is open source in Common Lisp as well as most every other language. Open Source per se is orthogonal to whether Lisp or Language X is a great language to have in one's toolkit.
Clojure mind share is quite low compared to scala as far as languages built on top of the JVM go. Personally I think clojure and clojurescript is pretty cool but not so cool so far as to tempt me away from python, javascript and CL.
For a startup don't even worry about what language your competitors, if any, are using. Simply produce the best product you can and get it to market and break even before you even begin to think about these other things. HINT: in this age there is not one or a few areas and a bunch of competitors but millions of unique cool niche ideas and products. Build a better system for pushing out new app and product ideas consistently on the side. It isn't like the case of web store competitors at all. Also today there is a large market to buy startups that produce some webapp product that grows enough legs to more than break even.
You don't predicate your business on a secret but on producing a good product people want at a price point they are happy to pay. Secrets are hard to keep. Particularly when the author just told us about his secret weapon and advises as many as possible to use the same "secret".
Back in the "old days" we would joke that if we wanted to derail our competition then we should simply give them our source code. Their best geeks would be bemused and befuddled for months. Today we are drowning in reams of "free code" that may or may not be suitable. Then we have a side helping of hundreds of web APIs to pick from for various subparts of a product. So much time is spent understanding and adapting open source and choosing among and massaging our data to use so many external APIs that it is a wonder anything gets done. :P
Aside, the size of the NPM database ... is that really indicative of the amount of JS code out there? For example, how many trivial modules like "isarray" are there, whose sole active code is this single line?
> Clojure hit the scene as a modern lisp that embraced the JVM.
> By now, Clojure is the de facto standard lisp for new applications.
What's with the whole, "modern," meme when people talk about Clojure? SBCL only forked off from CMU CL in 1999. Clozure CL is still well supported and actively developed. It's not like CL was written in the 1960s and had never changed ever since. The final standard was published in 1994. The alpha and beta of the JDK weren't released until a year later. Given that Clojure leverages so much of the JVM what exactly makes it the modern, de-facto standard for new applications? Don't we already have an ANSI standard for such a Lisp?
Is it modern because it can invent without restriction of a specification? That's a good thing and plenty of other Lisps are doing that... but what makes it, "modern," and why is it the standard? Is its relative immaturity a feature when you're taking on the world in a startup?
Either way it's not much of a secret anymore but I still don't see many startups advertising that they use Lisp -- even Clojure is rather rare. I hope more people give Lisp a try and kick butt.
As a long time Common Lisper, allow me to disagree. The STM and concurrency features of Clojure are what brought me in, and they certainly haven't been "done before". Common Lisp's concurrency story was always poor.
You could also argue that the effects of lazy evalutation and transducers could have been achieved before using SERIES package, but then you surely haven't tried to actually use and debug SERIES code.
> The STM and concurrency features of Clojure are what brought me in, and they certainly haven't been "done before". Common Lisp's concurrency story was always poor.
It's the JVM that does most of the work here for Clojure. You get the same features when working in Java if you wanted them.
Most CL implementations provide access to native threads and kernel polling facilities (kqueue, select, epoll, etc).
If "modern," means, "leverages the libraries and implementation of a well-funded ecosystem" then I suppose you have a point. Clojure is quite modern. And that is confusing.
> You could also argue that the effects of lazy evalutation and transducers could have been achieved before using SERIES package, but then you surely haven't tried to actually use and debug SERIES code.
I'd much rather debug CL code any day. Even at its worst it is still far ahead of Java's tracebacks and having to mentally map them back to where the error occurred in your Clojure code.
Orbitz tackled a hard technical problem with Common Lisp. I don't know if it was the "secret weapon", but it must have been fairly critical. They needed efficiency.
Viaweb is not even an example of that. They're a startup that succeeded while using a niche language. That's correlation, not causation, despite everything pg says.
Looks like a subtle ad for Clojure. Which I don't mind because I am slowly falling in love with Clojure. I am not sure if it's a "secret" or we should "write all the things in Clojure" but I'd love to see more companies use it, including start ups. I wouldn't mind working for one. Combine JVM's power with Om/Reagent and I think it's a solid stack. Although I am not advanced Clojure programmer. I wonder how it's doing in production?
I've never quite bought into the lisp idea that maximum "power" or "expressiveness" is always good. I could see it being an advantage for library authors who could present better, safer interfaces to the library's functionality. But if you have a startup, how much time do you have to build libraries and refine their interfaces?
I doubt that these qualities are so good that they would be a material advantage for application developers. And certainly not material enough that it would affect the outcome of a startup (unless it makes the engineers more motivated).
That being said, I hope clojure succeeds, and it's on my list to learn sometime (though not in the top 3).
But we need types people! Properly static ones that can be used to encode your intentions and handle inconsistency before even running the program. And yes I know that Clojure has an optional type system. But optional means that you cannot rely on having types in the libraries that you use. So you can't fit your stuff together with their stuff and have reasonable expectations that it will work once the user does that one thing you didn't think would be a most reasonable thing to do (of course it is, what were you thinking! And tell me again why you didn't just let the computer do that thinking for you?).
Types do have a lot of benefits but they also come with a big cost: some programs are very hard to express in modern type systems. For example, consider a pattern that is becoming more common today in UI programming (even in plain javascript): keeping your UI in one big tree data structure, and then working on arbitrary paths (represented by a sequence of keys) into that data structure. That is not a pattern you can encode in a type system.
Types also come with a lot of inherent complexity that you don't get with dynamic programs. Recently, I wrote a Type Provider for web Routing in F#, and it is orders of magnitude more complicated than what a dynamic approach would have been. It is nice that it is not possible to express a program that uses route parameters the wrong way, or provide a route that is unreachable, etc, but it takes a lot more work than a dynamic version with no such safety.
One domain where I think static typing is worth it is for parsing. After having used FParsec to parse a structure, and also having tried it dynamically, I felt the static typing added little complexity for all the safety it brought, so it was very much worth it. But there are also some dynamic parsing libraries that much easier to get started with, such as Instaparse, so YMMV.
I use a pattern matcher that can match against types in Scheme, a
language without static typing. Static types are just not the crucial
feature that some people make them out to be. I believe this
criticism of Pascal and C from SICP applies to other languages with
static types (sans the pointer stuff):
Pascal and C admit structures whose elements are structures. However,
this requires that the programmer manipulate pointers explicitly, and
adhere to the restriction that each field of a structure can contain
only elements of a prespecified form. Unlike Lisp with its pairs,
these languages have no built-in general-purpose glue that makes it
easy to manipulate compound data in a uniform way.
With static typing, it's also difficult to write procedures that operate on arguments of which the type is none of the procedure's business.
Well, I was definitely not mentioning Pascal and C. Haskell's type system is much easier to work with.
The thing is that any operation you are using on those arguments (of which the type is none of the procedure's business) can be described by an interface which the argument implements.
You're much safer knowing in compile time that the operations you'll use will be valid, instead of hoping something doesn't crash in runtime.
That pretty much describes my Angular data models, yes. My internal helper functions get passed sub-structures from the model, but Angular bindings get strings like "top.area.sub.fld" to name a variable.
Doubtless, there's another way to do this without a global / dynamic data tree, but this is working, at least for me.
So use F#'s dynamic typing parts? Or make everything obj and cast away? Unless most solutions don't work with types then it seems that you're better off having them in the language and using escape hatches as necessary.
Perhaps the ideal language would be one where every library is statically typed, either through social or technological means, but the application language is optionally typed. This would be a pretty big departure from most normal languages, but with opinionated languages like Go (which comes with its own code formatter) existing, it's not too crazy.
But again no massive ecosystem of libraries to simply plug in so unsure if 'secret weapon'. Also I have experienced the nightmare of trying to port non typed libraries to typed Clojure and falling into a rabbit hole of errors propagating. Haven't had the same problem with Racket but ecosystem is not even close to Clojure so anecdotes worthless.
One day we will find that perfect type system (and associated compiler/IDE/Interactive front end) and then everyone will take one big sigh of relief. And then we all will start fixing problems in a way that makes them stay fixed.
I agree types prevent people from shooting in their foot. Types are what make program ide friendly, and programmer friendly. Got that.
But types are also what cause us to think with in the box i.e to say powerful type hierarchies, the ones that save us from writing boiler-plate are also the ones that are difficult to understand, and in Java, somewhat weird. Not to mention they are needlessly specific - Unless you have ML style parameterized types.
Also, note that if you are working in type safe language, does not mean you can escape writing tests. You still have to write tests.
Just like human body is not all bone, though it gives structure, nor a bridge is all steel, even though that is what keeps it together, likewise I don't believe all aspects of a project should be written in a strictly and statically typed language. Dynamic types have their niche, and the nice thing about dynamic types is you are just concerned with ideas aka abstractions instead of type algebra and its mechanics.
Once you have protected your interfaces(with a type safe language), you are past "Idk what i am reading" stage, and hence in a safer place to do type agnostic things - and the sweet thing with clojure and lisps, yeah sweet if you get it, you can almost always break what you think your ceiling of capabilities is.
Flamewar in the comments here... Just throwing in my 2c.
I've been programming Clojure on the side for about a year now. It all started when I did the exercises in the Clojure for the Brave and True-book, and also simultaneously picked up Emacs and Cider (a nice IDE indeed, and keyboard-based).
Clj is just code like anything else. Anything you can do in Clojure you can do in Java. But with that language, I find myself creating things which I could not in other languages. Solutions come easier. Programs don't become long, but they get a lot done, and are performant.
I dig it, and really recommend it to anyone who is bored with their day job.
Javascript will probably be around for a while but Python and Ruby have been constantly shrinking in mind share and buzz these past years (mostly because of Go but also because there are now faster, statically typed languages that subsume them).
The simple truth is that statically typed languages have been much better at improving in areas where dynamically typed languages used to be the only ones shining than the other way around.
In a few years, we'll look back at dynamically typed languages with the same fondness as we do Pascal: it was a good idea at the time but we do better now.
> Javascript will probably be around for a while but Python and Ruby have been constantly shrinking in mind share and buzz these past years
At least in the scientific and data analysis communities, Python is exploding. I'd say it's neck and neck with R. R's infrastructure outside data analysis is weak but improving. Go isn't even a part of the conversation.
Can't speak to Ruby, but as the "big data" hype cycle continues Python will continue to blow up.
> Why would you pick Lisp or Clojure over say Elixir, Scala, or even F#?
They are all great choices!
F# is an excellent choice if you are happy living within the .net ecosystem. I'd highly recommend it if you are a c# dev. Tooling is great. Ecosystem is really good etc. Performance is really good.
Elixir? Its a really good option for high concurrency apps. It's based upon Erlang and has access to heavily battle tested libraries. Tooling and ecosystem not as good for many kinds of applications as the jvm/clr.
Scala? Another good option somewhat limited by its complexity. I personally prefer F# and I think it has better tooling. Nevertheless it's a great option for the jvm crowd.
Why did I choose clojure? Mainly for its good defaults, a community focused on simplicity, data, and composability, a great jvm AND JavaScript story, macros, amazing tooling, and ease of learning.
With Lisp your program is data, like any other data, so it is easy to transform with macros etc. (Disclaimer I have never written a lisp program)
On the other hand languages like Haskell have a powerful type system that largely negates the need for macros. However when you do want to macro in Haskell you have template Haskell and it is quite ugly.
This was a great read. I've always heard about Clojure and now I have a better idea as to what it is. I'm still going to stick with what I've heard all along: it doesn't matter what language you write in, it matters how you write it.
In the expressjs example, I wonder how practical is to do all that interop instead of just using plain JS? I mean, is awesome that Clojurescript can leverage all those JS libs, but its worth writing lots of interop just to use Clojurescript? Of course, as Clojurescript libraries grow this will not be an issue, but it is nowhere close to JS for web dev.
For me, a more practical approach for server side would be to use Clojure instead of Clojurescript, the problem is Clojure does not have the rich ecosystem of web libraries as nodejs does.
If you're starting from from scratch it looks really awesome. No callback hell and in believable tooling. Watch these talks to see what I mean about tooling:
> In the expressjs example, I wonder how practical is to do all that interop instead of just using plain JS?
It depends. Interop in clojurescript isn't any more difficult than using plain JS but if your code is very interop heavy I don't think you gain a lot by using Clojurescript.
For a web application though interop code is going to be a tiny fraction of the overall code.
I tried Clojure, but immediately got the impression that the run-time type-system is ill-designed. I cannot recall the details, but I can remember odd things like different run-time types used for empty lists than for non-empty lists. A simple program doing a "switch" on some run-time types turned out to be really cluttered. But of course, there's always the possibility that I was doing something wrong :)
In a dynamic language you're not supposed to rely on types, but on (dynamic) interfaces. Also Clojure, as a design choice, converts your collection into a lazy list when applying operators, then you need to convert it back if you want by using "into".
Here are some heretical thoughts. The language is irrelevant. The text editor is irrelevant. The OS is irrelevant. The size of your monitor is irrelevant. All your productivity hacks are irrelevant. The only relevant thing is your ability to formulate and solve problems.
You might say the language can help with both the formulation and the solution but I'd say that just comes down to what language you're most comfortable with and how good of a problem solver you are. So you can use lisp and I'll use ruby and at the end it'll all be a wash because the fundamental bottleneck will always be the speed at which you can formulate and solve problems and how quickly you can respond to market dynamics. All other choices are accidents of history.
Graham and his buddy were just good problem solvers.
Irrelevant might be too strong of a word. Yes, it might not be as significant as your own problem solving skills but they are still important. You have to remember that tools chosen by you to solve problems.
You wouldn't say what tools you use to build a house is irrelevant only carpenter's skills.
Picking the right tool for the right problem is part of problem solving. Sometimes picking the right tool might boil down to what you are most comfortable with, or sometimes it might be learning a tool that you are not familiar with.
(Of course, you shouldn't flock to a particular tool just because someone else said so.)
Anyways, picking the right tools are important part of your problem solving skills.
Graham and his buddy were good problem solvers who picked the right tool for the problem that they were solving.
> but I'd say that just comes down to what language you're most comfortable with and how good of a problem solver you are.
Techniques and concepts influence your ability to reason about problems.
If you don't understand dictionaries (hashes), or trees, or macros, or reflection, or first class functions etc, then you will think about a problem very differently.
If you understand the concepts then you can usually get away with not having those features readily available in your language.
For example if your language doesn't have strong macro support you can get away with using code-gen instead.
Likewise if your language doesn't have built in dictionary support, you can get away with rolling your own, or finding a library that adds it.
Or if your language doesn't support first class functions you can often get away with patterns like the command pattern instead.
But if you don't understand the concepts then your ability to think about the problem becomes much more limited.
But why would you want to use a language that didn't have such things? Not having lexical scope, first-class functions, and macros is a huge hinderance to me. Sure, I can work around it, but it's a terribly inefficient and unsatisfying way to work.
Anyone with a good CS background can think abstractly in terms of algorithms and data structures, instead of focusing in language X with feature list Y.
But languages differ in how much of that they abstract away from you. If the language already does a lot of those algorithms and data structures with minimal syntax, the CS guy can think on an even higher level of abstraction.
Algorithms and data structures are on a too low level of abstraction in order to be useful in solving real world problems. Most problem domains are not expressed in terms of algorithms and data structures, they're rather formalised as languages.
Heretical or not you are dead wrong. Don't think so? Go program in Forth or Assembler instead of your current main language for a while and get back to me. There would have been no reason to invent any HLL or even assembler if you were correct. Put aside your ruby and try to do the same work in vanilla C and get back to me.
I think you're taking his point too literally. Yes, you can find a pathological interpretation of his words and beat that up. But his point is that most of the arguments people have here over language choice are irrelevant, because whether you're using PHP or Python or Ruby or Scala or Go is not the bottleneck in the system.
Personally, I agree. I've never seen a bad team succeed because of a good language, and I've never seen a good team fail because of a bad one. We argue over languages not because it makes or breaks companies, but because that's the most obvious feature of our days. We form tribes around languages the way non-programmers do around sports teams, and as with sports teams, minor differences get blown up into major social divisions. It's our own version of the streetlight effect: https://en.wikipedia.org/wiki/Streetlight_effect
I'm all for people arguing about language features when they are actually designing languages. But as far as building businesses around software, I think "Language X is a secret weapon" is absurd. If a good team picks something they're happy working in, then the biggest problem with a project will almost never be the language they picked.
I think that's wrong for at least some contexts, because - for example - Python is so slow that it can literally become the bottleneck on any non-toy server. So unless you're running a one-person blog or don't carDjango should probably not be your first choice for a scalable production server.
People argue about languages because they all have different trade-offs for performance/efficiency, initial development time, developer cost, developer availability, and maintenance.
The differences are real, and they have real effects on the profitability and the prospects of success/failure of any project.
Management quality and culture do too, obviously. But you can assume that for a given level of management competence, different languages will still create different business outcomes.
Can you give an example of a company that failed because somebody chose Python? Or even one with materially worse business results, say a 20% reduction in revenue?
My guess is no, because people make fine businesses out of nominally unscalable technologies all the time. Rails is a notorious performance pig, for example, but there are plenty of successful businesses built in it. When server bottlenecks become an issue, modest performance tricks and ops magic are applied, and suddenly per-server performance is no longer the problem. And still, despite being a pig, Rails is a great place to start, because its real power is not CPU efficiency, but ease with which you can get something workable up and then iterate.
There is no language that is so fast that it's infinitely scalable out of the box. Every software company that grows hits performance problems with naive uses of the tools at hand. Every successful one figures out how to make things work. And making technologies serve the human purpose at hand isn't a characteristic of good languages; it's a characteristic of good teams.
Indeed, a strange one to pick. I'd even say that if portability isn't a concern, working with x86_64 assembly is quite pleasant. If you avoid the lure of premature optimization, and stick with either function oriented, or at least procedure oriented programming, without too many long jumps -- I think you could get on quite fine in assembly.
The biggest issue, might be one that (sadly too) many low-level languages share: with the move to UTF-8 as the only sane choice for text, there are no string/char datatypes any more. No reason one can't use procedures/functions to deal with strings in assembly, but it would require a bit more planning than in a language that have "native" UTF-8 support.
[ed: I think Kollibri OS proves both points: it's a working OS, but they've yet to port from 32bit to 64bit even though they upstream they forked from (Menuet OS) has a 64 bit version (that's not Free software/open source).
Forth should not be in this list, it is a meta-language. It can be quickly turned into something equally or more powerful than you current Blub (pick any, does not matter).
5-6kloc is not "tons", it's just several days worth of work. And then I can build a compliant JavaScript implementation on top of this, in order to be able to parasite on 200k npm packages (actually, did this already).
>5-6kloc is not "tons", it's just several days worth of work.
To buy you what? Not even what the Java SDK has as built-in libs. And 5 kloc in "several days"? I doubt it.
>And then I can build a compliant JavaScript implementation on top of this, in order to be able to parasite on 200k npm packages (actually, did this already).
Link or it didn't happen.
Besides, there's no point in using Forth if you're gonna end up running your stuff in a "compliant JavaScript implementation" (which takes much more than "several days" anyway).
" I think Lisp still has an edge for larger projects and for applications where the speed of the compiled code is important. But Python has the edge (with a large number of students) when the main goal is communication, not programming per se.
In terms of programming-in-the-large, at Google and elsewhere, I think that language choice is not as important as all the other choices: if you have the right overall architecture, the right team of programmers, the right development process that allows for rapid development with continuous improvement, then many languages will work for you; if you don't have those things you're in trouble regardless of your language choice."
I doubt this. Looking at some stuff done in C - a ton of time is simply wasted shuffling bytes around and fixing the resulting memory issues.
If your point is that language alone isn't gonna decide much, that's probably mostly right. FB used/uses PHP. Harry Potter was written on pen and paper. I don't think either is much of an argument about tools.
You are absolutely right. However, if you use a cluttered language with a dozen choices for a data structure or concurrency (as far as I know Ruby is not one of them), you'll run into many accidental problems before you even get to solving your business domain problem.
It goes hand in hand. Back in the days people said lisp in newbies hands will be backfire, but handled by those with abstraction skills it will give you ability to express things you cannot with other systems.
Many here agree that technology should not matter, but it does at one point. Some trait of the culture behind and around languages are very very important. Lisp tree recursion, "lexicality" and ad-hoc genericity is not present in other languages. Immutability in the ml land (even though side effects are permitted in some) is also something few people have to live in and understand.
Both these things give you new idioms, hints, point of view about how to solve problems. It also sheds light on the other ones.
I'm not really a programmer. All my training is in abstract math so whenever I see people arguing over language X and Y it always seems weird to me. I don't write code until I've sketched things out with pencil and paper and by necessity I have to disentagle the syntax of the language from the essence of the problem. Turns out when I do things that way translating the solution back into a language X ends up being the easy part and it really doesn't matter what the target language ends up being. Adding macros and types just shifts how I encode the solution and in no way affects its abstract formulation.
So I don't think it goes hand in hand. The fundamental bottleneck is always what's in your head.
And this is actually the most important thing in industry software development.
It affects:
1. how easy it is to modify an existing codebase: eg., can you add feature X before your competitor even after you app grew to 1M LOC?
2. how easy it is for a new developer to start working productively on you codebase
3. how stable your software is: do modifications introduce subtle bugs that make the whole thing more and more unstable until it becomes unusable?
4. how secure your software is: how easy (read "cost effective") is it to write secure code without having to take 10x longer to do this?
From a business perspective, the "fundamental bottleneck is always what's in your head" problem can always be solved by temporarily hiring a few super-smart consultants with "bigger heads" that can get over this bottleneck, the rare times when you have the problem...
The rest of the time, the "shape of your code" is 100x times more important than the abstract formulation of an elegant solution. Because what you do 99% of the time is "re-shaping" existing code to do slightly different versions of what it already does, not solving interesting problems...
So yeah, at this level, languages can make all the difference!
This is approach is great for greenfield development, but when it comes to software maintenance I find myself having to think in terms of the target language so that I can formulate the right long-term solution for the codebase in question.
Yes, this is true. For maintenance projects I've noticed that before I can sketch things out I have to translate the existing codebase into the proper abstract domain but in my experience this is just an extra step and does not negate the general approach of thinking in a domain that is completely free of programming language related issues.
But depending on the language you intend to use, you may approach the problem from a different angle. A Lisp solution may look very different from a Python or C solution.
A way from your formal specification of a solution down to a particular language set of features can be enormously long.
A trivial example can be translating a formal grammar - codified as just few lines of BNF - into a low level language (say, C, or Java). Good if your grammar can be expressed as recursive descent directly, and even in this case you'll end up spitting out hundreds of lines of code, most likely making a lot stupid mistakes.
So, language does matter a lot. BNF is a right language for this task. C or Java are not right, period. Programmers should not waste their precious time translating abstract concepts down to the boring languages minutiae - they must always use the same level of abstraction as their problem domain requires.
> I'm not really a programmer. All my training is in abstract math
Back in mainframe days, that training would've been the easiest way to get work as a programmer. Nowadays, much "programming" is gluing together pre-written open source stuff using glue languages like Bash, Javascript or Groovy by staff who entered programming as business people transferring from other departments then left a big mess for someone else to clean up when they used their embellished experience to get another job, probably with "architect" in its job title, somewhere else. If you've done any hobbyist stuff using something like Lisp or Haskell, then think twice before saying you're "not really" a programmer.
> and by necessity I have to disentangle the syntax of the language from the essence of the problem.
Doing that is why programmers prefer different languages, some language allow that to be easier than others do. Some language allow a much more direct expression of the essence of the problem.
> Adding macros and types just shifts how I encode the solution and in no way affects its abstract formulation.
Sure it does, it allows you express the abstract formulation more directly with less syntactic fluff. Ideally you wind up with a DSL that allows the problem to be solved as tersely yet readably as possible.
In my experience, the more interesting reasons to choose languages come from the "platform and ecosystem" categories rather than syntax.
Lua: It has clean, clear syntax but its real value is the implementation, which is geared towards use as an embedded language.
Javascript: The language isn't bad but were it not for being the browser scripting language it never would have gotten so popular and tools like node.js probably wouldn't even exist.
Java: Native language on the powerful JVM.
C/C++: Direct access to various operating system and hardware libraries not available anywhere else.
Perl: Deployed widely on unix systems making it very useful for scripting and system administration.
SQL: For querying relational databases.
Granted, within each category, you might have syntax preferences. I would never use Ruby or Perl if I could use Python, for example. I prefer JSON for serialization(robust and simple) but for manually encoding a complex data structure, YAML is far more comfortable. That's all syntax. But picking the right language for a job usually requires going beyond the syntax.
> In my experience, the more interesting reasons to choose languages come from the "platform and ecosystem" categories rather than syntax.
I agree, but that doesn't have anything to do with what I said. I didn't say anything about choosing a language for a particular job, I said what makes programmers prefer a language which is very much syntax.
> Javascript: The language isn't bad but were it not for being the browser scripting language it never would have gotten so popular and tools like node.js probably wouldn't even exist.
If it were not for being the browser scripting language, it would not exist, so any reasoning beyond what it might have been otherwise is fallacious as it would not have been at all.
> Java: Native language on the powerful JVM.
Nope, portability across platforms made it popular.
> But picking the right language for a job usually requires going beyond the syntax.
Not disputed, but still unrelated to my point which is about syntax. I'm discussing syntax, it's off topic to and rude to tell me not to discuss syntax which is basically what you're doing.
The real question is do you understand how a computer works. There's mapping a solution into a programming language, but programming language is just a tool to help with the real goal which is to map the solution to the computer itself.
> The fundamental bottleneck is always what's in your head.
Expressing a correct solution in a programming language is just the first step. It also has to be performant, usable, and accessible. Language choice usually has implications in those categories.
This works for small programs you write on your own. Even then, the kind of problems you even try to solve in Matlab are different from the ones you try to solve in JavaScript.
But in business, you often program within some framework (which makes some things easy, other things hard), glue together tons of open source libraries (what's available in different languages differs a lot), and you work on huge codebases in teams (and different languages get in your way in different ways).
I have a good CS background and follow a similar approach.
We were lucky that back they excel in throwing all the paradigms at the students and have enough lectures about pure algorithms and data structures regardless of the language.
I wonder if Graham was presented with the same problem today, will he pick a lisp and write everything from scratch or pick up a less "expressive" language because it has a strong and mature ecosystem of libs.
The reason Paul Graham used Lisp is it was the only reasonable way at the time to implement RTML, which was the HTML macro language with online editor that Viaweb was built out of. RTML was how every website in Viaweb was constructed, it is what allowed them to move quickly and customers to customize with unequaled flexibility.
It was their key competitive advantage and Lisp was critical to its creation and function. It would be foolish to attempt to implement RTML in something else. Perhaps slightly less foolish in 2015 but not by much.
You can have your cake and eat it. It's possible for Lisps to use library code in less expressive languages, such as C, via a variety of mechanisms (foreign function interface, extending the virtual machine, ...). If you want access to Java libraries, you can use Kawa, Clojure, or Armed Bear Common Lisp.
There's a solid trend to do that, Norvig wrote a lot of recent articles in python, colleges are replacing Java with it too. Lisp genes have spread so it's not as alternative as it used to be. That said, SBCL seems to have possibilities and performance unmatched by all the tiny trendy languages of the day (except for v8 maybe and lua maybe).
At the time when PG was solving a web development problem, web development problems were "new and interesting", so it made sense to use a Lisp.
Nowadays it probably doesn't make sense to use Lisps for web, but it would absolutely make sense if you are solving something that is actually a new and interesting problem nowadays...
If your startup is dependent on solving chicken and egg marketing problems the choice of Lisp vs. something else is fairly irrelevant as you said. This is the case in most startups that take a traditional business (e.g. Taxi) and modernise it (e.g. uber). The level of tech required to do that is not going to be cutting edge. It is quite rote to solve these things in RoR or PHP.
On the other hand there may be some startups where the choice of language really matters. One thing I can think of is how a lot of Enterprise software is a big feature fest with massive complicatedness coming from a mixture of the real-world muckiness of the problem, and poor architectural choices. Such shops require armies of programmers to maintain their bloated wares. Maybe a startup with a much more thoughtful approach, using a Lisp/Haskell/etc. type language and a small number of developers can have a big competitive advantage.
Customer "we need a new feature". Startup "sure we'll add a rule and you'll have that tomorrow". Old business "that will take 3 months and cost $1m"
I think you're making the case for language irrelevance even more. The reason big corp. can't move fast isn't because of their technology choices. It is because of all the bloatware and bureaucracy. You can be bureaucratic and bloated in any language.
Let's say you're talking about Java. A "bad" team will have mountains of code. A "good" team will apply Greenspun's Tenth Rule and use a lisp to compress down the mountain to the extent that "lisp" can compress it over Java.
You may protest that the lisp has angle brackets and that it's inferior to say Clojure, but the machine does not care.
The kind of org that cares about how it writes software will choose the right tool for the job, which may be Lisp. However they may be able to achieve their aims in Java. Whatever they choose it would have been carefully considered.
The kind of org that doesn't care how it writes software is unlikely to choose an outlier language and will use C#, PHP, Java etc. because it is easier to hire devs and you won't get fired for choosing it.
> The only relevant thing is your ability to formulate and solve problems.
The maximum level of abstraction available for your language is indeed a hard upper limit on how you formulate and solve problems. Solving a complex problem on a high level of abstraction can be trivial, and the same problem may be nearly impossible on lower levels.
The real real secret is using a language so obscure that you end up writing the compiler for production use, by studying the compiler that some researcher created for a paper.
It was a serious answer though. If your USP is your technology, using Lisp is meh, you better be using something that is like dynamite to the problem you are trying to solve. And the problem you are solving is more like high-frequency trading or data analysis not a peer-to-peer marketplace for rickshaw riders and their passengers.
Footnote 1 makes no sense, suggesting that the only way to get access to "a bunch of other useful code" is to embed it into a C program and that understanding your dependencies is somehow easier in Javascript.
In reality, Quicklisp[1] offers effortless access to over 1200 libraries and programs. It's nowhere near the 200 thousand packages in npm, but the overall quality is good and these libraries cover a surprisingly large number of things.
The post tries to display Clojure as "Lisp, but with access to open source". Clojure is a fine programming language, but it's also completely different from Common Lisp. CL has high quality optimizing compilers performing extensive type checking, a powerful object system, easy access to C libraries, etc. If you need Java interop, you can just use ABCL[2].
[1]: https://www.quicklisp.org/ [2]: http://abcl.org/